Do #LLMs have mental states?
-
@icastico hmm, I feel like the 'communication' route has the same way of dissolving into open issues as the one we are currently on, e.g., how is this example exchange I just ran with Ollama not 'communicatively grounded' in some way?
-
Dimitri Coelho Molloreplied to Ulrike Hahn last edited by
@UlrikeHahn @pbloem As Peter said, the notion of intentionality is tied to things like beliefs and desires, which come with the whole package of agency, etc. In an early paper, I distinguish mental states from cognitive states, and intentionality from aboutness for this reason.
So I think the definition in the OP is baking quite a bit more than it might seem. (Worthwhile to mention the odd fact that philosophers of mind and phil.s of cognitive science are categories that don't overlap much. -
It seems a great example of how the LLM is not collaborating actively - it is simply filling in the blanks using vacuous patterns. That second blurge about why it blurges shows how understanding of the conversation is particularly lacking. Do you think the LLM “fears” anything? Is it really telling you something about itself to guide you to mutual understanding?
-
@icastico no, I don't think it 'fears' anything, and I also don't think it's "over eager" and, yes, the language it uses makes me slightly nauseous (like AI art also does). But it is still managing to incorporate a novel 'word' and concept into the exchange - that's my point.
We can always point to some further thing down the road, and say, well it doesn't do *that* yet.
But why is an ability to incorporate a novel 'word' into an exchange not evidence of 'communicative grounding'?
-
Ulrike Hahnreplied to Dimitri Coelho Mollo last edited by
I can see that one *could* bake in more depending on how one wants to understand intentionality, but say I want to follow a Dretske-style informational account. Is the issue not then done and dusted?
I guess it's not clear to me (as a non-philosopher) that that whole package is necessarily baked in even though it may be on particular accounts.
while try and check out your other paper too!
-
-
Dimitri Coelho Molloreplied to Ulrike Hahn last edited by
@UlrikeHahn @pbloem The thing is that it is rather unclear that the barebones 'intentionality' of Dretske is the 'intentionality' that some have defended is the mark of the mental. A very liberal way to go (that some have taken) is to claim so, but then all sorts of things would count as having mental states, including bacteria, etc. This has a cost in terms of the usefulness of the concept. And would make the claim that LLMs have mental states rather unsurprising, with the bar set so low.
-
Dimitri Coelho Molloreplied to Ulrike Hahn last edited by
@UlrikeHahn @pbloem on belief-desire psychology, there is a complicated matter regarding the relations between folk psychology and scientific psychology, the ways we understand ourselves and others in everyday life and the sort of targets cognitive science and neuroscience have, etc. On this, as usual, I recommend Dennett, 'True Believers', especially.
-
Ulrike Hahnreplied to Dimitri Coelho Mollo last edited by
-
Ulrike Hahnreplied to Dimitri Coelho Mollo last edited by
fair point, but at the same time, I had (mistakenly?) taken (part of) the point of the notion of intentionality to be a detaching from, in effect, just saying "whatever humans have" because then it's not really helpful for elucidating the 'mental'? So the question just becomes 'how much of what humans have is required', no?
Basically, the usefulness vanishes in both directions (bar too high, bar too low)?
-
Dimitri Coelho Molloreplied to Ulrike Hahn last edited by
@UlrikeHahn @pbloem I recommend also Three Kinds of Intentional Psychology, also by him. True Believers especially hinges on dispositionalism about beliefs, desires, etc., which I think is the right account, but of course is debatable.
He was very good at attacking things from unexpected angles, and sometimes dissolving (or trying to) problems, which is a way to solve them
-
Incorporating your word, your name, or your input is how the predictive text generation works. It responded to a specific prompt for a more concise response and repeated the terminology you defined back to you. That is one of the patterns it was trained on. That’s impressive. People do that.
But the fact that its explanation included “feelings” based reasoning- feelings which it doesn’t have - exposes what it’s doing; it is “mindlessly” completing patterns. It is mimicking responses from text produced by people who have communicative grounding - but it isn’t collaboratively working towards mutual understanding the way those people were.
Edit: also, the second blurge, rather than a concise “by design my default response is comprehensive” shows the lack of communicative grounding related to that novel word.
-
Dimitri Coelho Molloreplied to Ulrike Hahn last edited by
@UlrikeHahn @pbloem arguably non-human animals also have belief-like and desire-like states, agency, etc., so this tends not be a human-only view.
In any case, it is indeed far from easy to figure out how best to conceptualise and theorise about this whole mess. We would be out of a job if it weren't, though, so all is not bad
FWIW, I'm sceptical there are going to be neat bars to clear either way, and some things will go down to choices we make depending on what turns out to be useful
-
@icastico Icastico, I feel like we're talking at cross purposes here. You're trying to explain to the kinds of things that it isn't doing, and that llama 3 at least, likely can't do. I'm on board with those. I don't have beliefs in magical LLM abilities that differ from yours. I'm pointing out that it's incorporating a wholly novel word in such a way that it can respond to it over multiple rounds of exchange. It has assimilated (by whatever means) what 'blurge' refers to in some minimal sense.
-
Did it? If so, it seems it would have avoided the second blurge. I don’t see evidence that it interpreted the meaning of “blurge” - hence my skepticism.
-
@icastico you can say "Incorporating your word...or your input is how the predictive text generation works" but given that no one has ever uttered the word 'blurge' the only context in which one could 'predict' it is the context of this very communicative exchange. So how is that not (some form of) 'communicative grounding'?
-
@icastico it llama 2 can't change its initial response format but it `responded' to my
question in two placesFor "Could you give me an answer that is less of a blurge?" it gave me a shorter answer and for
>>> I guess my qustion is why you blurge like this in the first place?
it gave me a semantically and contextually appropriate 'response', no?
-
Ulrike Hahnreplied to Dimitri Coelho Mollo last edited by
"some things will go down to choices we make depending on what turns out to be useful"
and that's how it should be!
But I think this very helpful exchange overall today is also making me realise more that the real issue with LLMs is that we have a network of inter-related, mutually supporting, conceptual terms and LLMs push against every one of them (I'm sure you'd long got to this point), so useful answers need to take into account much, or all of this, network all at once
-
Ulrike Hahnreplied to Dimitri Coelho Mollo last edited by
@dcm @pbloem
I appreciate solving by dissolving, my point was more that his particular attempts haven't tended to work for me. I thought the point of his `intentional stance' was that beliefs, intentions and desires are convenient fictions, in which case I have low expectations for the current context other than an answer that says 'call them mental states if useful' which I can already get to just by words being words ;-). But I will read Three Kinds of Intentional Psych! thanks again! -
A blurge follows. Apologies.
We, of course, only have objective access to the words produced (just like with people) - but I think there is evidence that the LLM’s responses are not communicatively grounded.
Yes. It gave you a shorter answer based on your explicit instructions and it was able to incorporate a new vocabulary word that you defined for it. That is literally what LLM’s are designed to do. It mapped an association between “blurge” and a semantic network of terms for “too much info”. It is an open question whether that counts as “interpretation” in the sense I was getting at. It was able to represent the meaning of your text.
For the “why” question - it’s a mixed bag. It started with a semantically and contextual appropriate response - again doing what it is designed to do - then it started confabulating and talking about its feelings and motivation. In a human-to-human interaction, those assertions would be appropriate- but in the context of the LLM talking to a human, it seems contextually inappropriate in two ways. The first is that it ignores your stated preferences to avoid blurges and in the second it attributed feelings and motivation to itself which it is unlikely to have.
One of the keys to communicative grounding is the “quality”/veracity assumption (from Grice) - I assume you are telling me what you believe to be true. If you are flouting that assumption I look for a reason for why you are telling me an untruth and interpret based on that (e.g., you are saving face, telling a joke, trying to deceive). When the LLM attributes human emotions to itself - it flouts the veracity assumption- but why? If it was communicatively grounded, it should be apparent and I should be able to interpret its “actual” meaning (knowing that we both know it isn’t emotional). Maybe it is trying to make me feel comfortable with its alien nature. Maybe it is telling a joke. Whatever interpretation I come up with, it requires that I attribute motivation to the LLM and that I attribute to it the capacity for interpretation of its utterances.
But the other option is to assume it is simply and mindlessly filling in patterns. That seems more likely. And that would not be communicatively grounded.