I'm seeing new rounds of posts on whether #AI communicates or truly uses language.
-
for a longer version of that thought (all comments welcome), see
Do LLMs REALLY reason, understand, think, summarise...?
One way in which discussions of AI capabilities are unsatisfying is that they often descend into what feels like argument about words or ...
UlrikeHahn (write.as)
-
the point that #LLMs don’t “truly” communicate is the point of the original Bender et al. stochastic parrots paper, and the problem with it remains that all of our other computation-based interactions and computational tool use *also* doesn’t involve “communication”. For that reason, the presence or absence of “true communication” predicts nothing about the utility of different computational systems we have built so far.
That makes it hard to see why this is a crucial determinant now
-
or to put this differently, it feels like a continual (re-)discovery of the fact that (on the standard notion of computation) computation is based on syntactic, not semantic properties. Its value simply derives from the fact that syntactic relationships and syntactic operations track semantic ones in useful, meaningful, ways.
It's nothing special about LLMs, it's what computation (to date) has been.
The confusion arises simply because LLMs seem, more than other comp. systems to "speak"
-
the thing I personally find frustrating is that not only does the point about 'true communication' not go far in practice for those reasons (imo), but also that current AI systems actually seem really interesting with respect to whether they might, in fact, go slightly beyond classical, purely syntactic notions of computation for two reasons: multi-modality and embedding in an overall system drawing on reinforcement learning with human feedback
-
I would love it if there was more discussion around the question of whether multi-modality and RLHF do, in fact, constitute some (minimal) form of embodied or `situationally grounded' meaning and, relatedly, mean #LLMs represent a notion of computation that goes (a bit) beyond merely syntactic operations.
I'd love that not as a way of discussing LLMs' practical value, but as a theoretical question of fundamental interest for a number of disciplines
-
@UlrikeHahn
I think RLHF is quite interesting but I really can't see it as a form of situational grounding (which I've empirically studied in multimodal human interaction)RLHF takes response variants, gathers human preferences for them on a limited number of dimensions (e.g. humannes, helpfulness, harmlessness), and derives a statistical profile from those preferences that is used as a nozzle to modify the flow of regurgitated text output. It is more like sycophancy as a service
-
@dingemansemark yes, I can see that from one perspective RLHF is just more (purely syntactic) ML. What I'm just a little bit less sure about is whether the unit or system as a whole gives an embedding context.
It's so difficult in thinking about these issues not to just slide back into a Chinese room-like point. Or to put it differently, I think the means by which the overall system comes to embed the constraints (ie as a statistical profile) seems different from the relationship itself 1/2
-
@dingemansemark 2/2 and I wonder to what extent it is the relationship itself that matters and how exactly it affects what is represented
-
@dingemansemark sorry, that's all really ill-formed and muddled, but that's precisely why I'd love the debate.
Also, I'm not sure the two points don't matter specifically in combination (multi-modality *and RLHF). Maybe it would help to start from the other end. What *else* is there shaping human communication than some kind of grounding and interactional constraints?
-
@UlrikeHahn but what NLP calls multimodality (a heap of text:image labels) is also incomparable to the rich, embodied situational grounding seen in human-human interaction — I see not a difference in degree but in kind
the RLHF nozzle obscures the fundamental differences by making LLMs behavior superficially more human-like, which is why I find it interesting (and why users find LLMs compelling)
-
@dingemansemark thanks Mark! I think my problem here is the same as it always is with embodied views of cognition. I'm totally sympathetic, but ultimately I feel they never fully deliver the goods and end up feeling question begging to me.
So, yes, human-human interaction is richer, and that's somewhere you can draw the line, but *why there*?
Why not view the LLM case as proto-communication? WHY does 'richness' as a notion with a continuum cross a *qualitative* threshold for you? 1/2
-
@dingemansemark 2/2 or to put it differently, do you think we communicate with dogs who learn words? do they communicate with us? Does Kanzi exhibit linguistic communication?
what is your view on those, and how are they the same or different to LLMs?
-
@UlrikeHahn
yes, we communicate with dogs and other animals.i don't care much for policing the boundaries of language (see https://www.jbe-platform.com/content/journals/10.1075/avt.00095.ras )
LLMs are just a fundamentally different kind of entity: not evolved, not precarious, not autopoietic, not self-sustaining, not self-organizing — all things we share with dogs and many other beings we coexist with
some elements of what they do may seem similar, how they do it is not, as argued here:
https://direct.mit.edu/opmi/article/doi/10.1162/opmi_a_00160/124234/The-Limitations-of-Large-Language-Models-for -
@dingemansemark will read, but just to clarify, nothing about our exchange is (to me) about 'policing the boundaries of language. It's about the underlying intuitions.
see the link to my blog post earlier in the thread which is precisely about why drawing boundaries for the sake of it is not a useful endeavour.
We're already on the same page with that.
-
@dingemansemark and yes, I agree that "LLMs are just a fundamentally different kind of entity: not evolved, not precarious, not autopoietic, not self-sustaining, not self-organizing", but what I am (still) missing is the explicit connection of that to 'communication' or embodied meaning.
-
@UlrikeHahn how could an entity like that ever 'mean' or 'communicate' anything except in the eye of the (autopoietic, etc) beholder? to me it feels like a category mistake on the order of saying a funhouse mirror is capable of insulting you
-
@dingemansemark because standard accounts of 'meaning' haven't actually expicitly posited an additional clause: "and btw this definition applies only to auto-poetic, evolutionarily formed agents"
it doesn't mean they shouldn't, it's just that I don't think they have
see also this unfolding thread:
Ulrike Hahn (@[email protected])
Do #LLMs have mental states? on standard definitions in philosophy a state or event is a "mental state" if and only if it is a conscious state or an intentional state I assume LLMs aren't conscious but how is a currently active representation that mediates ChatGPT4o drawing a table, identifying a table in an image, or answering a query about tables *not* an intentional state in the sense of this definition, i.e. something that has 'intentionality' in the sense of 'aboutness'? @[email protected]
FediScience.org (fediscience.org)
-
@UlrikeHahn you may be right about 'standard' accounts (which?) though I think any account that includes notions like speaker's meaning, intention, commitment (=most of them since at least Malinowski and Firth a century ago) are pretty clear on the intersubjective nature of meaning (see Bender & Koller or @davidschlangen on this)
I still don't see more than a funhouse mirror that enables us to see, at best, ourselves in a new light
-
@dingemansemark @davidschlangen
Mark, I think the issue is that historically those accounts haven't really needed to specify what counts as "a speaker" in the first place.
We can now sneak in "only autopoetic systems that are a product of evolution can qualify as speakers" but we haven't done that in the past (because we didn't have to), and it remains (to me) question begging to just assert it as a condition now.
and I don't think the appeal to inter-subjectively negotiated meaning 1/2
-
@dingemansemark @davidschlangen 2/2 is sufficient to establish that. Yes, there is an important part of meaning that is negotiating (and creating) meaning in context. But there are also parts that aren't and that are parasitic on language as a conventional system.
Just ruling those aspects of meaning out doesn't work for me, see there-
"Stochastic parrot" is a misleading metaphor for LLMs
Metaphors are hugely important both to how we think about things and how we structure debate, as a long research tradition within cogniti...
UlrikeHahn (write.as)