Do #LLMs have mental states?
-
interesting! But the LLM is far beyond the thermostat. It uses what (for us) is a symbol in a flexible way that not only includes many different contexts, it is linked to different modalities (and in the case of ChatGPT4o has representations formed simulatenously across dimensions) and it supports a whole host of flexible actions (and in that sense reaches out into the world).
The thermostat does almost none of that. The LLM is vastly more powerful.
-
@qruyant @philosophy but regardless of whether those differences matter or not, structurally, your point would be that the standard definition of mental state is derivative of something that is not actually expressed in the definition itself, namely that the agent in question has intentions (or desires, or something) in a meaningful way?
-
@UlrikeHahn @philosophy
I would say no, or perhaps 'impossible to say'. Nobody knows how humans recognise a table, but its likely to be very different from the way that a computer does it. One problem: neuroscientists are every bit as susceptible to hype as #AI -
@UlrikeHahn @philosophy surely a mental state is nothing to do with intentionality? If I'm happy or stressed it is not (necessarily) because I intend to be?
I guess the nearest to a mental state that a computer model has is if the processor is working hard, a little like high cognitive workload in people. -
@david_colquhoun @UlrikeHahn @philosophy I think it could be useful to separate the concepts of "thought/thinking" from consciousness. It could be that LLMs or complex enough systems can hold thoughts that are about things in the world.
-
@david_colquhoun David, this thread is not about hype and as a cognitive scientist who wrote a very long, boring PhD about categorization I'm aware that we don't fully know how humans recognise a table. That's not my question here. This is a very specific question about the technical definition of mental state that I put up in a lecture every year, and how it relates to LLMs.
-
@UlrikeHahn I have become convinced that it's very hard to rule out LLM mental states while ruling in human and animal mental states.
-
@compthink @david_colquhoun
yes, the standard definition of mental state (in my OP) separates consciousness from other mental states like 'beliefs' in order to capture the fact that beliefs needn't be part of our current awareness (you and I, for example, believe the Earth to be round, but we also believed this 10 min ago before I mentioned it). -
Andrew, I don't quite understand your point? What I gave in the OP is (to the best of my knowledge) a textbook philosophical definition of mental state and involves intentionality ('aboutness').
-
@UlrikeHahn @philosophy yes, I don't deny any of this, I just think these differences in complexity are irrelevant in so far the LLM wouldn't have intentionality without us using it in some way for some purpose, whereas it seems to me that to the intentionality of a mental state is more constitutive of it. It's a vague intuition, not sure how to express well. Maybe the intentionality is merely metaphorical in the case of tools? Not sure what you mean about what's not expressed in the definition
-
@qruyant not expressed in the definition in the sense that you seem to me to be appealing to some richer definition of what it means to intend something than is (necessarily) implicated in the sense of 'intentional' state or 'intentionality' as aboutness?
-
here is another angle on all of that, though: does it then become important for your line of argument whether LLMs can engage in `deceptive behaviour' (putting that in scare quotes as part and parcel of the same issue & not wanting to sneak in hidden assumptions), ie behaviours that are functionally goal oriented (in some sense) but are not in and of themselves desired by us as the 'tool users'?
-
Seems to me if you are using intentionality as your marker of mental state you have to determine if the LLM - which does create a representation of things in the world - has a way to interpret them as a representations. Otherwise, you are left with asking whether the text of a book, which represents something in the world, is a mental state.
-
@UlrikeHahn @philosophy yes it's important *whether* it could do it. In the kind of view I present, it couldn't really do it, deceiving us would be more of a side effect of a malfunction. To put it differently, I take LLM to be very similar to maps: a map is *about* a city or country, but it's not a mental state. A map can deceive us but not intentionally (unless its maker has this purpose)
-
To follow up - I don’t see any evidence that LLMs interpret or understand what they write. There is plenty of evidence that they don’t- based solely on their output.
-
@qruyant so the thing about the map is that it is wholly static, and the thermostat has a wholly deterministic, simple response hat is locked. But LLMs *do not* and this is exactly where I think my points about flexibility matter. LLMs, in some sense, 'select actions'.
I think that's a pretty big discontinuity, no?
-
isn't the answer to that given with the capacity for action? The book is wholly inert and passive. LLMs 'do stuff' - and they now not only 'do text', ChatGPT4o can also segment images and highlight the "table", and draw a table for me.
what else is there?
-
Icastico, I'm not personally convinced that 'based solely on their output, there is plenty of evidence that they understand what they write' is a great argument. *People* also get things wrong and misunderstand all the time.
And with regard to 'interpretation', what more is there to interpretation than having a representation that supports flexible, cross context, cross modality response routines in the external world?
what is the extra sauce that you see?
-
An industrial robot does stuff as well. Does it have mental states? The thermostat example above applies as well - it changes/reacts/acts based on “perception” /input from the environment.
I don’t know the answer here, but the ambiguity around definitions of “mental states” makes it a tricky topic. If “aboutness” plus “action” is all you need, then maybe LLMs have mental states - but then so do lots of dynamic systems that we wouldn’t generally consider to be in this conversation.
-
My gut is that you need an epistemological component as part of the “mind” for there to be “interpretation