Do #LLMs have mental states?
-
Icastico, I'm not personally convinced that 'based solely on their output, there is plenty of evidence that they understand what they write' is a great argument. *People* also get things wrong and misunderstand all the time.
And with regard to 'interpretation', what more is there to interpretation than having a representation that supports flexible, cross context, cross modality response routines in the external world?
what is the extra sauce that you see?
-
An industrial robot does stuff as well. Does it have mental states? The thermostat example above applies as well - it changes/reacts/acts based on “perception” /input from the environment.
I don’t know the answer here, but the ambiguity around definitions of “mental states” makes it a tricky topic. If “aboutness” plus “action” is all you need, then maybe LLMs have mental states - but then so do lots of dynamic systems that we wouldn’t generally consider to be in this conversation.
-
My gut is that you need an epistemological component as part of the “mind” for there to be “interpretation
-
@icastico I think the big difference to the thermostat is that it doesn't have representations (at least on my understanding of thermostats and representations, but I could be wrong about both).
But I think available evidence suggests that LLMs *do*. Not only are those self-formed representations they are causally efficacious in generating the LLMs behaviour, including underwriting a flexible response repertoire that includes multiple modalities. That's nothing like a thermostat, no?
-
@icastico this is interesting! could you say more about what you mean by 'epistemological component'?
-
@twsh Thomas, could you elaborate a bit more on your thoughts to date here?
and if it's so hard, what does this mean going forward for phil. of language and phil. of mind?
is there some revision of concepts required? or more 'it is what it is', LLMs are new, 'in between' things?
-
@david_colquhoun just adding this for the avoidance of doubt: By saying this is a 'specific question about a technical definition' I am *not* trying to gate keep who can respond (I'm not myself a philosopher for a start!). Your thoughts are very much welcome, but I would like this thread to remain on point.
-
@UlrikeHahn yes, but is it what brings "aboutness"? If so, what's the relation between this feature and being about something else (in a more relevant sense than maps)?
-
@qruyant ok, progress in as much as you earlier didn't think my point about flexibility mattered at all
a different way of putting what we're grasping at (?) is to what extent 'intentionality' itself requires 'agency'. And the flexibility seems to me to matter for that, no?
-
Ulrike Hahnreplied to Ulrike Hahn last edited by [email protected]
@qruyant also, I would like to again point out that (on my limited understanding) the notion of 'intentionality' does not, as a matter of consensus, have a necessary, intrinsic connection with 'agency' or 'intending', so that this is pushing toward either a reformulation or a particular account of intentionality among many
-
@UlrikeHahn interesting. I work with measuring mental states like emotions and stress using sensors ('affective computing'). Apparently we are working from different textbooks, as it were.
-
@andrewspink
I think those, by definition, are conscious states in standard philosophical parlance? -
I am thinking along the lines of Searle - some sense that the LLM “knows” stuff. Representation isn’t equivalent to “knowing” in this framing. Following a complex computational algorithm isn’t the same as knowing.
-
Modern thermostats certainly have similar representation of temperature when compared to what an LLM would have. A sensor’s output would be converted to a digital representation of the ambient temperature and this would be fed to an algorithm for dynamic responses that can also include user preferences and time of day.
-
@icastico ok. So why *wouldn't* 'extracting an underlying model on the basis of a next token prediction task' that allows you to not just master that task with some degree of accuracy but generates representations with sufficient power to causally support drawing an image count as 'knowing'?
what is 'knowing' if no that?
(I'm not, for a second, saying your wrong, I'm just saying I can't currently understand your answer in a way that makes it feel like a sufficient answer to our question) -
re modern thermostat, I stand corrected! ;-).
There are still fundamental differences with respect to that representation. One it is wholly externally imposed (which the LLMs representation of table issues is not) and the uses a thermostat can make of it are fixed (whereas the whole point of foundation models is that this is not the case).
No external agent has (specifically) created the LLMs representation or set up how it is used.
-
This gets back to that term “interpret” - an LLM that makes a mistake doesn’t evaluate and correct based on examination of its own output - it responds to a new prompt indicating an error with the same “fill-in-the-blank” algorithm - perhaps informed by new information (if that was included in the new prompt). Since the LLM doesn’t know what the representation means- it can’t reinterpret its output without guidance from the user. An LLM wouldn’t re-examine its answer and go “oh wait - I forgot to carry the 1” because it doesn’t know what its answer means.
At least that is what I infer from the output I see from these systems.
-
Dimitri Coelho Molloreplied to Ulrike Hahn last edited by
@UlrikeHahn there is quite some dispute about what counts as the mark of the mental, if anything, and similarly for the cognitive. Often something like a belief-desire psychology is considered central for the mental, while it is not so for the cognitive.
I've argued that LLMs do have aboutness regardless of interpretation in this paper, referred to as intrinsic meaning, in case you might be interested: https://arxiv.org/abs/2304.01481
That just means that they have what it takes to form their own reps.
-
@icastico so your sense of "knowing" is more about 'meta-awareness' (in which case young children and non-human species presumably can't 'know') and not about a particular kind of lawful relationship with outcomes?
-
I don’t think I would take it that far. Kids certainly interpret and know before they know they know. If you know what I mean.