Do #LLMs have mental states?
-
Ulrike Hahnreplied to Dimitri Coelho Mollo last edited by
"Often something like a belief-desire psychology is considered central for the mental" and there is a slender peg on which to hang things if ever there was
will check out your paper in a moment
-
@icastico how, though - on your version of 'knowing'?
-
Even young children evaluate the result of their intentional acts to see if they were successful and learn from that experience. Their experiences change the algorithm they use the next time based (at least in part) on internal adjustments - not exclusively on external prompts.
-
@UlrikeHahn @philosophy I think when something is capable of representing something it can be perceived of as having a "mental state" I guess, but in the same way that a slime mold might approach food by the shortest possible path
-
Dimitri Coelho Molloreplied to Ulrike Hahn last edited by
@UlrikeHahn yep, this whole debate is rather complex (or muddled), and I think somewhat misguided. We probably should draw finer distinctions between different kinds of cognitive system, rather than care much about marks of this or that vague category.
-
@icastico and why is that not what ChatGPT4o does, for example during RLHF?
-
Ulrike Hahnreplied to Dimitri Coelho Mollo last edited by
@dcm ok, gave your paper a quick skim- it's great!!
so, back to the main theme - if we buy the premise that the representations in GPT4o are, in fact, representations, and thereby have 'aboutness', is there anything you see *in the specific definition* I put up in the OP that would allow one to conclude that LLMs *don't* have mental states?
-
interesting example- I once had a very intense, multi-day email exchange with a PhD student on why I didn't think the framing of slime mold 'making decisions' was meaningful
-
@UlrikeHahn @dcm You might argue that intentionality requires full agency. GPT doesn't really have that (by design). It very likely has intentions when it's executing a task, "local" to that task, but not overarching long term intentions about itself.
You could then argue that an SQL engine has aboutness and local intentionality, so that's not really enough for mental states.
(Not my opinion, btw. Just an argument that could be made. I think my that we are too far from a workable definition)
-
@UlrikeHahn Just so I understand, though, how large of a space are you considering "LLM" to denote? Is GPT-2 in this family, or does it need some "reactive" component? For example, the generate method maybe requires mental states but that's a layer of abstraction on top of the model, a model itself
-
@pbloem @dcm
I definitely see going down the route of agency as the best way to deny LLMs mental states among what is present in the discussion here.But I'm unaware of that having historically been an intrinsic part of the notion (presumably for lack of need), so the question is *why exactly* would we want to make that move now?
-
I tried to short circuit the need for too detailed debate by jumping straight to ChatGPT4o and highlighting some of the features that I think speak to past debate on these issues
-
It is not internally driven. Kids learn through RLHF too - but they also learn through RLSF (s=self or subjective). They evaluate/interpret their own performance/knowledge. I don’t think LLMs do that. And, to me, that seems essential to claiming a “mind” - I see the paper Dimitri posted puts a lot more meat on these issues.
I find the “communicative grounding” issue particularly interesting as it gets to the heart of the matter - it seems. LLMs can’t collaboratively engage as they don’t bring their own perspective / intentions/ desires to the table - and that short coming makes it hard, intuitively, to credit them with their own knowledge/ understanding - and hence a mind that could have mental states.
-
@icastico hmm, I feel like the 'communication' route has the same way of dissolving into open issues as the one we are currently on, e.g., how is this example exchange I just ran with Ollama not 'communicatively grounded' in some way?
-
Dimitri Coelho Molloreplied to Ulrike Hahn last edited by
@UlrikeHahn @pbloem As Peter said, the notion of intentionality is tied to things like beliefs and desires, which come with the whole package of agency, etc. In an early paper, I distinguish mental states from cognitive states, and intentionality from aboutness for this reason.
So I think the definition in the OP is baking quite a bit more than it might seem. (Worthwhile to mention the odd fact that philosophers of mind and phil.s of cognitive science are categories that don't overlap much. -
It seems a great example of how the LLM is not collaborating actively - it is simply filling in the blanks using vacuous patterns. That second blurge about why it blurges shows how understanding of the conversation is particularly lacking. Do you think the LLM “fears” anything? Is it really telling you something about itself to guide you to mutual understanding?
-
@icastico no, I don't think it 'fears' anything, and I also don't think it's "over eager" and, yes, the language it uses makes me slightly nauseous (like AI art also does). But it is still managing to incorporate a novel 'word' and concept into the exchange - that's my point.
We can always point to some further thing down the road, and say, well it doesn't do *that* yet.
But why is an ability to incorporate a novel 'word' into an exchange not evidence of 'communicative grounding'?
-
Ulrike Hahnreplied to Dimitri Coelho Mollo last edited by
I can see that one *could* bake in more depending on how one wants to understand intentionality, but say I want to follow a Dretske-style informational account. Is the issue not then done and dusted?
I guess it's not clear to me (as a non-philosopher) that that whole package is necessarily baked in even though it may be on particular accounts.
while try and check out your other paper too!
-
-
Dimitri Coelho Molloreplied to Ulrike Hahn last edited by
@UlrikeHahn @pbloem The thing is that it is rather unclear that the barebones 'intentionality' of Dretske is the 'intentionality' that some have defended is the mark of the mental. A very liberal way to go (that some have taken) is to claim so, but then all sorts of things would count as having mental states, including bacteria, etc. This has a cost in terms of the usefulness of the concept. And would make the claim that LLMs have mental states rather unsurprising, with the bar set so low.