@ByrdNick indeed, I've been saying this for a while now (I'm not a neurologist though) https://www.kevinmarks.com/confabulation.html
@KevinMarks @ByrdNick that's a great list of toots. I like this one:
"This is my deeper worry about AI - as we demand that the AI justify its decision to us, we create an incentive model for confabulation, just as we have for natural intelligence. Personalised reassuring salesmanship and gaslighting."
Especially: "Personalised reassuring salesmanship and gaslighting" Ouch...
I was familiar with the concept so when I saw LLMs do it that immediately struck me. But it looks like you were predicting something like this years in advance.
I see some of this in the context of machine understandability, with plausible yet false explanations for other processes. That would almost be nice as I suspect a lot of human speech is a bit like that. LLMs don't typically do that - they just explain themselves badly.