Do #LLMs have mental states?
-
Dimitri Coelho Molloreplied to Ulrike Hahn last edited by
@UlrikeHahn @pbloem The thing is that it is rather unclear that the barebones 'intentionality' of Dretske is the 'intentionality' that some have defended is the mark of the mental. A very liberal way to go (that some have taken) is to claim so, but then all sorts of things would count as having mental states, including bacteria, etc. This has a cost in terms of the usefulness of the concept. And would make the claim that LLMs have mental states rather unsurprising, with the bar set so low.
-
Dimitri Coelho Molloreplied to Ulrike Hahn last edited by
@UlrikeHahn @pbloem on belief-desire psychology, there is a complicated matter regarding the relations between folk psychology and scientific psychology, the ways we understand ourselves and others in everyday life and the sort of targets cognitive science and neuroscience have, etc. On this, as usual, I recommend Dennett, 'True Believers', especially.
-
Ulrike Hahnreplied to Dimitri Coelho Mollo last edited by
-
Ulrike Hahnreplied to Dimitri Coelho Mollo last edited by
fair point, but at the same time, I had (mistakenly?) taken (part of) the point of the notion of intentionality to be a detaching from, in effect, just saying "whatever humans have" because then it's not really helpful for elucidating the 'mental'? So the question just becomes 'how much of what humans have is required', no?
Basically, the usefulness vanishes in both directions (bar too high, bar too low)?
-
Dimitri Coelho Molloreplied to Ulrike Hahn last edited by
@UlrikeHahn @pbloem I recommend also Three Kinds of Intentional Psychology, also by him. True Believers especially hinges on dispositionalism about beliefs, desires, etc., which I think is the right account, but of course is debatable.
He was very good at attacking things from unexpected angles, and sometimes dissolving (or trying to) problems, which is a way to solve them
-
Incorporating your word, your name, or your input is how the predictive text generation works. It responded to a specific prompt for a more concise response and repeated the terminology you defined back to you. That is one of the patterns it was trained on. That’s impressive. People do that.
But the fact that its explanation included “feelings” based reasoning- feelings which it doesn’t have - exposes what it’s doing; it is “mindlessly” completing patterns. It is mimicking responses from text produced by people who have communicative grounding - but it isn’t collaboratively working towards mutual understanding the way those people were.
Edit: also, the second blurge, rather than a concise “by design my default response is comprehensive” shows the lack of communicative grounding related to that novel word.
-
Dimitri Coelho Molloreplied to Ulrike Hahn last edited by
@UlrikeHahn @pbloem arguably non-human animals also have belief-like and desire-like states, agency, etc., so this tends not be a human-only view.
In any case, it is indeed far from easy to figure out how best to conceptualise and theorise about this whole mess. We would be out of a job if it weren't, though, so all is not bad
FWIW, I'm sceptical there are going to be neat bars to clear either way, and some things will go down to choices we make depending on what turns out to be useful
-
@icastico Icastico, I feel like we're talking at cross purposes here. You're trying to explain to the kinds of things that it isn't doing, and that llama 3 at least, likely can't do. I'm on board with those. I don't have beliefs in magical LLM abilities that differ from yours. I'm pointing out that it's incorporating a wholly novel word in such a way that it can respond to it over multiple rounds of exchange. It has assimilated (by whatever means) what 'blurge' refers to in some minimal sense.
-
Did it? If so, it seems it would have avoided the second blurge. I don’t see evidence that it interpreted the meaning of “blurge” - hence my skepticism.
-
@icastico you can say "Incorporating your word...or your input is how the predictive text generation works" but given that no one has ever uttered the word 'blurge' the only context in which one could 'predict' it is the context of this very communicative exchange. So how is that not (some form of) 'communicative grounding'?
-
@icastico it llama 2 can't change its initial response format but it `responded' to my
question in two placesFor "Could you give me an answer that is less of a blurge?" it gave me a shorter answer and for
>>> I guess my qustion is why you blurge like this in the first place?
it gave me a semantically and contextually appropriate 'response', no?
-
Ulrike Hahnreplied to Dimitri Coelho Mollo last edited by
"some things will go down to choices we make depending on what turns out to be useful"
and that's how it should be!
But I think this very helpful exchange overall today is also making me realise more that the real issue with LLMs is that we have a network of inter-related, mutually supporting, conceptual terms and LLMs push against every one of them (I'm sure you'd long got to this point), so useful answers need to take into account much, or all of this, network all at once
-
Ulrike Hahnreplied to Dimitri Coelho Mollo last edited by
@dcm @pbloem
I appreciate solving by dissolving, my point was more that his particular attempts haven't tended to work for me. I thought the point of his `intentional stance' was that beliefs, intentions and desires are convenient fictions, in which case I have low expectations for the current context other than an answer that says 'call them mental states if useful' which I can already get to just by words being words ;-). But I will read Three Kinds of Intentional Psych! thanks again! -
A blurge follows. Apologies.
We, of course, only have objective access to the words produced (just like with people) - but I think there is evidence that the LLM’s responses are not communicatively grounded.
Yes. It gave you a shorter answer based on your explicit instructions and it was able to incorporate a new vocabulary word that you defined for it. That is literally what LLM’s are designed to do. It mapped an association between “blurge” and a semantic network of terms for “too much info”. It is an open question whether that counts as “interpretation” in the sense I was getting at. It was able to represent the meaning of your text.
For the “why” question - it’s a mixed bag. It started with a semantically and contextual appropriate response - again doing what it is designed to do - then it started confabulating and talking about its feelings and motivation. In a human-to-human interaction, those assertions would be appropriate- but in the context of the LLM talking to a human, it seems contextually inappropriate in two ways. The first is that it ignores your stated preferences to avoid blurges and in the second it attributed feelings and motivation to itself which it is unlikely to have.
One of the keys to communicative grounding is the “quality”/veracity assumption (from Grice) - I assume you are telling me what you believe to be true. If you are flouting that assumption I look for a reason for why you are telling me an untruth and interpret based on that (e.g., you are saving face, telling a joke, trying to deceive). When the LLM attributes human emotions to itself - it flouts the veracity assumption- but why? If it was communicatively grounded, it should be apparent and I should be able to interpret its “actual” meaning (knowing that we both know it isn’t emotional). Maybe it is trying to make me feel comfortable with its alien nature. Maybe it is telling a joke. Whatever interpretation I come up with, it requires that I attribute motivation to the LLM and that I attribute to it the capacity for interpretation of its utterances.
But the other option is to assume it is simply and mindlessly filling in patterns. That seems more likely. And that would not be communicatively grounded.
-
@UlrikeHahn I can try.
My first thought is I think the same as the one you posted. If you think about mental states the way that I was introduced to in philosophy of mind classes, it's very natural to think that LLMs/AIs of some sort could have mental states.
I think that we always revise and refine concepts, but I'm not sure that we need to do much in this case.
-
Thomas Hodgsonreplied to Thomas Hodgson last edited by
@UlrikeHahn What interests me the most right now as a philosopher of language is what we should think about these questions:
1. Do the sentences produced by AIs have meanings? (Characters, contents, whatever.)
2. Can AIs say things?
3. Can AIs mean things?
In particular, I am interested in the arguments people use for negative answers.
I think there is a parallel to the point about mental states: be careful not to rule out human language use.
-
Thomas Hodgsonreplied to Thomas Hodgson last edited by
@UlrikeHahn I did try to articulate some thoughts in a short talk recently, but that's all I have written about it.
-
Dimitri Coelho Molloreplied to Ulrike Hahn last edited by
@UlrikeHahn @pbloem I think it is more than that (and he is not really a fictionalist about those). The point connects to discussions we had here, relating to patterns, higher-level explanatory constructs, generalisations and predictions, etc.
There may be no thing in the brain that we look at it and say 'oh, look, that's the belief that p', but that does not mean that the claim the person believes that p is false or a fictionalisation.
-
@twsh great slides! your questions are extremely congenial to how I have been thinking about things and to my interest/motivation in starting this thread.
-
Ulrike Hahnreplied to Dimitri Coelho Mollo last edited by
@dcm @pbloem
read it now (bar a few Google book pages missing) and it's really great. Happy to publicly eat humble pie on that! And I agree that this connects to our #JuarreroBook discussions, and it's no coincidence that the link is agency and meaning.