@UlrikeHahn @dcm @twsh @qruyant @davidschlangen @BayesForDays @pbloem @floatygoodstuff
you certainly got what you asked for (productive debate) — and I've been enjoying reading some of the other subthreads!
@UlrikeHahn @dcm @twsh @qruyant @davidschlangen @BayesForDays @pbloem @floatygoodstuff
you certainly got what you asked for (productive debate) — and I've been enjoying reading some of the other subthreads!
@UlrikeHahn (I mean on one level I know why: cognitive science has for a long time been able to mostly ignore those material conditions & processes of situated action, choosing to privilege computations, representations, mental states, beliefs. Here too Suchman is an excellent starting point: her 1987 critique of the then-standard Schank & Abelson 'cognitive' account of planning is a model for the kind of move that we need, imo, if we want to think clearly about what LLMs are and what they do)
@UlrikeHahn
but why then, as a cognitive scientist, ignore all the material contributions & human labour involved in making it seem so? I can't summarise Suchman in a toot but trust me she's worth reading
it seems to me a serious overreach to call whatever digital operations an LLM carries out when prompted (provided with material, text or otherwise) an 'action in the external environment' (why "the"? "external" how?). That is my ground for saying I see an eagerness to ascribe agency
@UlrikeHahn
the heavy lifting done by members of a community of practice is in large part reciprocal; in the funhouse mirror of LLMs, it's really all you
how does the response to the prompt come to be "an action in the external environment"? which environment, precisely?
I don't understand the impulse to ascribe agency to an LLM and what seems to be a certain eagerness to obscure the processes of human sense-making and facilitation at work (see Lucy Suchman's Human/Machine Reconfigurations)
@UlrikeHahn
i think it is more precise to say they *don't* operate with meaning (conventional or otherwise), they just deal in tokens we happen to be able to assign meaning to
we are doing the heavy lifting here
as long as our prompt plus its own output fit the context size, it'll happily parrot our neologisms back at us (and yes I use that metaphor advisedly: I think it is apt in this context)
FWIW I think RLHF is the star here in packaging it all in the most obliging way possible
@UlrikeHahn you may be right about 'standard' accounts (which?) though I think any account that includes notions like speaker's meaning, intention, commitment (=most of them since at least Malinowski and Firth a century ago) are pretty clear on the intersubjective nature of meaning (see Bender & Koller or @davidschlangen on this)
I still don't see more than a funhouse mirror that enables us to see, at best, ourselves in a new light
@UlrikeHahn how could an entity like that ever 'mean' or 'communicate' anything except in the eye of the (autopoietic, etc) beholder? to me it feels like a category mistake on the order of saying a funhouse mirror is capable of insulting you
@UlrikeHahn
yes, we communicate with dogs and other animals.
i don't care much for policing the boundaries of language (see https://www.jbe-platform.com/content/journals/10.1075/avt.00095.ras )
LLMs are just a fundamentally different kind of entity: not evolved, not precarious, not autopoietic, not self-sustaining, not self-organizing — all things we share with dogs and many other beings we coexist with
some elements of what they do may seem similar, how they do it is not, as argued here:
https://direct.mit.edu/opmi/article/doi/10.1162/opmi_a_00160/124234/The-Limitations-of-Large-Language-Models-for
@UlrikeHahn but what NLP calls multimodality (a heap of text:image labels) is also incomparable to the rich, embodied situational grounding seen in human-human interaction — I see not a difference in degree but in kind
the RLHF nozzle obscures the fundamental differences by making LLMs behavior superficially more human-like, which is why I find it interesting (and why users find LLMs compelling)
@UlrikeHahn
I think RLHF is quite interesting but I really can't see it as a form of situational grounding (which I've empirically studied in multimodal human interaction)
RLHF takes response variants, gathers human preferences for them on a limited number of dimensions (e.g. humannes, helpfulness, harmlessness), and derives a statistical profile from those preferences that is used as a nozzle to modify the flow of regurgitated text output. It is more like sycophancy as a service