Ya, there are great use cases. I get nervous when companies and governments decide to use them because they have only listened to the good bits. I thinks it's worthwhile to know where they are not good to avoid making big, or just embarrassing mistakes by using their output unchecked.
It's like a Tesla self driving car. It's pretty great and reliable most of the time. Enough you can trust it to make the right decisions...until it kills you one day because you trusted it. LLM's probably aren't going to kill you, but being aware of their limitations is important. And for the developers of them, knowing what needs work is the valuable bit.
Ultimately I think they are flawed for the same reasons humans are. Overconfidence and reliance on knowledge. If the information is wrong, they can never reason out the correct answer. And as we well know, you can't trust everything you read on the internet. The overconfidence can be dialed down, but verifying facts on which decisions are made is a lot harder nut to crack.
|