We had our first talk in the seminar series ‘The Cognitive Science of Generative AI’
-
@MathieuP @locha noise as in “mistakes the number 5 for the number 8” in a learned problem that had 5£ in the description of Option A, and, by wonderful coincidence, that confusion happens to also pick out the more risky option in the test problem, and then that same wonderful coincidence is replicated 80% of the time, in just the right places over the entire set of 60 test items?
-
@UlrikeHahn @locha Hence my point: are we sure that the "non-identical gambles" are not somewhere in the training data (imagine a 1980s book with a large table of experiments and results, with lots of variations)?
-
@MathieuP @locha I don’t know what probability you assign to ‘sure’, but aren’t we effectively looking at the probability that the lead researcher Eric Schulz who has a PhD in experimental psychology and a masters in statistics and, incidentally, wrote his phd on generalisation, understands less about experimental methods than you do?
-
@UlrikeHahn @locha "Sure" as in "I have complete knowledge about what is in my training dataset". I do not dispute the author's understanding on their fields of expertise.
It has however been my experience that people who should definitely know better may have a somewhat hazy grasp on how a LLM work (or for that matter, how many models behave when trained with huge, imperfectly controlled training datasets). -
@MathieuP @locha
I guess my priors are different: my experience would suggest that researchers testing 35 different LLMs on 7 cog psych tasks (see link below), and then building a cognitive model by fine-tuning a language model on behavioural data from 60,000 participants with 10,000,000 choices led by the above mentioned researcher who directs Human Centered AI at Helmholtz have more than "a somewhat hazy grasp on how a LLM works"
CogBench: a large language model walks into a psychology lab
Abstract page for arXiv paper 2402.18225: CogBench: a large language model walks into a psychology lab
arXiv.org (arxiv.org)
1/2
-
-
adding one more post here, because I can see from some of the comments that this thread has created confusion. The work they describe involves both running models on behavioural tasks from the psych literature as is, and, more recently, models fine tuned on behavioural data. My 'thoughts' 1 and 2 apply in different ways and to different extent to those two different parts of their work.
-
@UlrikeHahn Looking forward to look at this… I’ve seen the « genAI is a competent reasoner » hypothesis rally a lot of very smart people. But it doesn’t seem to explain the failures all that well for me. I think it’s very much a debate in progress, but the « genAI works because of humongous data », also supported by a lot of very smart people (Alexei Efros’ work is what tipped the scale for me), seems to explain the failures better to me.
-
@locha 2/2 or is the position trying say that models ‘are just statistical devices that latch on to regularities that can be learned bottom up given sufficient volumes of data’? If that’s the position, then there is a way in which it is obviously true, but -to me as a cognitive scientist- it also begs the question of exactly how, and in what ways, that differs from (human) ‘competent reasoning’
-
Ulrike Hahnreplied to Louis Chartrand last edited by [email protected]
@locha I guess my own position is that “gen AI works because of humongous data” isn’t really a position without further info. Is it trying to imply that the model does a bunch of glorified table look up? Then I think there is ample evidence that they don’t (including from some of the failures, very much the same way that failures are indicative of underlying representations and processes in human reasoners)? 1/2
-
@UlrikeHahn "humongus data" is really more of a family of positions, but I think I'd take the "lookup table" take as an approximation. As in, given the prompt that's given, it makes a collage of what it takes to be the best fit (both in terms of what seems to fit there, given past examples, and what should pleases the asker, thanks to RLHF). I'm not sure what would constitute a failure for this, as it's clearly not a lookup table, but I haven't heard of a really precise account of this. 1/2
-
@locha an example of a failure that shows it’s not just a look up table: a hallucinated citation - it was never in the training set, by definition.
-
@UlrikeHahn Thanks! I feel that the idea of « collage » kinda works here. It stitches together parts of quotes it has seen elsewhere. So there is a lookup-like behaviour, but as the LLM keeps asking itself « what would work best next? » it might latch on to something else.
-
@UlrikeHahn Maybe as an aside, I figure this process to be a bit like what RG Millikan envisioned in her 1984 book. There are phrastic patterns and people stitch them together to make sentences in ways that are as likely as possible to be pragmatically felicitous. Which means copying instances where it worked.
-
@locha yes, but the key bit is that it stitches them together in plausible ways (people go writing to authors for those references). You have to have learned a good amount of structure to be able to do that (while of course still fail to understand fundamentally something about references).
Analysing error patterns this way is exactly how the language development literature, for example, seeks to understand what representations have been acquired.
-
-
@locha again, I think if you look at the actual cognitive science/cognitive linguistics literature you will find many extremely well worked out accounts on what you are calling “phrastic patterns that people stitch together”
