Do #LLMs have mental states?
-
@UlrikeHahn @dcm You might argue that intentionality requires full agency. GPT doesn't really have that (by design). It very likely has intentions when it's executing a task, "local" to that task, but not overarching long term intentions about itself.
You could then argue that an SQL engine has aboutness and local intentionality, so that's not really enough for mental states.
(Not my opinion, btw. Just an argument that could be made. I think my that we are too far from a workable definition)
-
@UlrikeHahn Just so I understand, though, how large of a space are you considering "LLM" to denote? Is GPT-2 in this family, or does it need some "reactive" component? For example, the generate method maybe requires mental states but that's a layer of abstraction on top of the model, a model itself
-
@pbloem @dcm
I definitely see going down the route of agency as the best way to deny LLMs mental states among what is present in the discussion here.But I'm unaware of that having historically been an intrinsic part of the notion (presumably for lack of need), so the question is *why exactly* would we want to make that move now?
-
I tried to short circuit the need for too detailed debate by jumping straight to ChatGPT4o and highlighting some of the features that I think speak to past debate on these issues
-
It is not internally driven. Kids learn through RLHF too - but they also learn through RLSF (s=self or subjective). They evaluate/interpret their own performance/knowledge. I donāt think LLMs do that. And, to me, that seems essential to claiming a āmindā - I see the paper Dimitri posted puts a lot more meat on these issues.
I find the ācommunicative groundingā issue particularly interesting as it gets to the heart of the matter - it seems. LLMs canāt collaboratively engage as they donāt bring their own perspective / intentions/ desires to the table - and that short coming makes it hard, intuitively, to credit them with their own knowledge/ understanding - and hence a mind that could have mental states.
-
@icastico hmm, I feel like the 'communication' route has the same way of dissolving into open issues as the one we are currently on, e.g., how is this example exchange I just ran with Ollama not 'communicatively grounded' in some way?
-
Dimitri Coelho Molloreplied to Ulrike Hahn last edited by
@UlrikeHahn @pbloem As Peter said, the notion of intentionality is tied to things like beliefs and desires, which come with the whole package of agency, etc. In an early paper, I distinguish mental states from cognitive states, and intentionality from aboutness for this reason.
So I think the definition in the OP is baking quite a bit more than it might seem. (Worthwhile to mention the odd fact that philosophers of mind and phil.s of cognitive science are categories that don't overlap much. -
It seems a great example of how the LLM is not collaborating actively - it is simply filling in the blanks using vacuous patterns. That second blurge about why it blurges shows how understanding of the conversation is particularly lacking. Do you think the LLM āfearsā anything? Is it really telling you something about itself to guide you to mutual understanding?
-
@icastico no, I don't think it 'fears' anything, and I also don't think it's "over eager" and, yes, the language it uses makes me slightly nauseous (like AI art also does). But it is still managing to incorporate a novel 'word' and concept into the exchange - that's my point.
We can always point to some further thing down the road, and say, well it doesn't do *that* yet.
But why is an ability to incorporate a novel 'word' into an exchange not evidence of 'communicative grounding'?
-
Ulrike Hahnreplied to Dimitri Coelho Mollo last edited by
I can see that one *could* bake in more depending on how one wants to understand intentionality, but say I want to follow a Dretske-style informational account. Is the issue not then done and dusted?
I guess it's not clear to me (as a non-philosopher) that that whole package is necessarily baked in even though it may be on particular accounts.
while try and check out your other paper too!
-
-
Dimitri Coelho Molloreplied to Ulrike Hahn last edited by
@UlrikeHahn @pbloem The thing is that it is rather unclear that the barebones 'intentionality' of Dretske is the 'intentionality' that some have defended is the mark of the mental. A very liberal way to go (that some have taken) is to claim so, but then all sorts of things would count as having mental states, including bacteria, etc. This has a cost in terms of the usefulness of the concept. And would make the claim that LLMs have mental states rather unsurprising, with the bar set so low.
-
Dimitri Coelho Molloreplied to Ulrike Hahn last edited by
@UlrikeHahn @pbloem on belief-desire psychology, there is a complicated matter regarding the relations between folk psychology and scientific psychology, the ways we understand ourselves and others in everyday life and the sort of targets cognitive science and neuroscience have, etc. On this, as usual, I recommend Dennett, 'True Believers', especially.
-
Ulrike Hahnreplied to Dimitri Coelho Mollo last edited by
-
Ulrike Hahnreplied to Dimitri Coelho Mollo last edited by
fair point, but at the same time, I had (mistakenly?) taken (part of) the point of the notion of intentionality to be a detaching from, in effect, just saying "whatever humans have" because then it's not really helpful for elucidating the 'mental'? So the question just becomes 'how much of what humans have is required', no?
Basically, the usefulness vanishes in both directions (bar too high, bar too low)?
-
Dimitri Coelho Molloreplied to Ulrike Hahn last edited by
@UlrikeHahn @pbloem I recommend also Three Kinds of Intentional Psychology, also by him. True Believers especially hinges on dispositionalism about beliefs, desires, etc., which I think is the right account, but of course is debatable.
He was very good at attacking things from unexpected angles, and sometimes dissolving (or trying to) problems, which is a way to solve them
-
Incorporating your word, your name, or your input is how the predictive text generation works. It responded to a specific prompt for a more concise response and repeated the terminology you defined back to you. That is one of the patterns it was trained on. Thatās impressive. People do that.
But the fact that its explanation included āfeelingsā based reasoning- feelings which it doesnāt have - exposes what itās doing; it is āmindlesslyā completing patterns. It is mimicking responses from text produced by people who have communicative grounding - but it isnāt collaboratively working towards mutual understanding the way those people were.
Edit: also, the second blurge, rather than a concise āby design my default response is comprehensiveā shows the lack of communicative grounding related to that novel word.
-
Dimitri Coelho Molloreplied to Ulrike Hahn last edited by
@UlrikeHahn @pbloem arguably non-human animals also have belief-like and desire-like states, agency, etc., so this tends not be a human-only view.
In any case, it is indeed far from easy to figure out how best to conceptualise and theorise about this whole mess. We would be out of a job if it weren't, though, so all is not bad
FWIW, I'm sceptical there are going to be neat bars to clear either way, and some things will go down to choices we make depending on what turns out to be useful
-
@icastico Icastico, I feel like we're talking at cross purposes here. You're trying to explain to the kinds of things that it isn't doing, and that llama 3 at least, likely can't do. I'm on board with those. I don't have beliefs in magical LLM abilities that differ from yours. I'm pointing out that it's incorporating a wholly novel word in such a way that it can respond to it over multiple rounds of exchange. It has assimilated (by whatever means) what 'blurge' refers to in some minimal sense.
-
Did it? If so, it seems it would have avoided the second blurge. I donāt see evidence that it interpreted the meaning of āblurgeā - hence my skepticism.