Do #LLMs have mental states?
-
@twsh Thomas, could you elaborate a bit more on your thoughts to date here?
and if it's so hard, what does this mean going forward for phil. of language and phil. of mind?
is there some revision of concepts required? or more 'it is what it is', LLMs are new, 'in between' things?
-
@david_colquhoun just adding this for the avoidance of doubt: By saying this is a 'specific question about a technical definition' I am *not* trying to gate keep who can respond (I'm not myself a philosopher for a start!). Your thoughts are very much welcome, but I would like this thread to remain on point.
-
@UlrikeHahn yes, but is it what brings "aboutness"? If so, what's the relation between this feature and being about something else (in a more relevant sense than maps)?
-
@qruyant ok, progress in as much as you earlier didn't think my point about flexibility mattered at all
a different way of putting what we're grasping at (?) is to what extent 'intentionality' itself requires 'agency'. And the flexibility seems to me to matter for that, no?
-
Ulrike Hahnreplied to Ulrike Hahn last edited by [email protected]
@qruyant also, I would like to again point out that (on my limited understanding) the notion of 'intentionality' does not, as a matter of consensus, have a necessary, intrinsic connection with 'agency' or 'intending', so that this is pushing toward either a reformulation or a particular account of intentionality among many
-
@UlrikeHahn interesting. I work with measuring mental states like emotions and stress using sensors ('affective computing'). Apparently we are working from different textbooks, as it were.
-
@andrewspink
I think those, by definition, are conscious states in standard philosophical parlance? -
I am thinking along the lines of Searle - some sense that the LLM “knows” stuff. Representation isn’t equivalent to “knowing” in this framing. Following a complex computational algorithm isn’t the same as knowing.
-
Modern thermostats certainly have similar representation of temperature when compared to what an LLM would have. A sensor’s output would be converted to a digital representation of the ambient temperature and this would be fed to an algorithm for dynamic responses that can also include user preferences and time of day.
-
@icastico ok. So why *wouldn't* 'extracting an underlying model on the basis of a next token prediction task' that allows you to not just master that task with some degree of accuracy but generates representations with sufficient power to causally support drawing an image count as 'knowing'?
what is 'knowing' if no that?
(I'm not, for a second, saying your wrong, I'm just saying I can't currently understand your answer in a way that makes it feel like a sufficient answer to our question) -
re modern thermostat, I stand corrected! ;-).
There are still fundamental differences with respect to that representation. One it is wholly externally imposed (which the LLMs representation of table issues is not) and the uses a thermostat can make of it are fixed (whereas the whole point of foundation models is that this is not the case).
No external agent has (specifically) created the LLMs representation or set up how it is used.
-
This gets back to that term “interpret” - an LLM that makes a mistake doesn’t evaluate and correct based on examination of its own output - it responds to a new prompt indicating an error with the same “fill-in-the-blank” algorithm - perhaps informed by new information (if that was included in the new prompt). Since the LLM doesn’t know what the representation means- it can’t reinterpret its output without guidance from the user. An LLM wouldn’t re-examine its answer and go “oh wait - I forgot to carry the 1” because it doesn’t know what its answer means.
At least that is what I infer from the output I see from these systems.
-
Dimitri Coelho Molloreplied to Ulrike Hahn last edited by
@UlrikeHahn there is quite some dispute about what counts as the mark of the mental, if anything, and similarly for the cognitive. Often something like a belief-desire psychology is considered central for the mental, while it is not so for the cognitive.
I've argued that LLMs do have aboutness regardless of interpretation in this paper, referred to as intrinsic meaning, in case you might be interested: https://arxiv.org/abs/2304.01481
That just means that they have what it takes to form their own reps.
-
@icastico so your sense of "knowing" is more about 'meta-awareness' (in which case young children and non-human species presumably can't 'know') and not about a particular kind of lawful relationship with outcomes?
-
I don’t think I would take it that far. Kids certainly interpret and know before they know they know. If you know what I mean.
-
Ulrike Hahnreplied to Dimitri Coelho Mollo last edited by
"Often something like a belief-desire psychology is considered central for the mental" and there is a slender peg on which to hang things if ever there was
will check out your paper in a moment
-
@icastico how, though - on your version of 'knowing'?
-
Even young children evaluate the result of their intentional acts to see if they were successful and learn from that experience. Their experiences change the algorithm they use the next time based (at least in part) on internal adjustments - not exclusively on external prompts.
-
@UlrikeHahn @philosophy I think when something is capable of representing something it can be perceived of as having a "mental state" I guess, but in the same way that a slime mold might approach food by the shortest possible path
-
Dimitri Coelho Molloreplied to Ulrike Hahn last edited by
@UlrikeHahn yep, this whole debate is rather complex (or muddled), and I think somewhat misguided. We probably should draw finer distinctions between different kinds of cognitive system, rather than care much about marks of this or that vague category.