We had our first talk in the seminar series ‘The Cognitive Science of Generative AI’
-
@UlrikeHahn Looking forward to look at this… I’ve seen the « genAI is a competent reasoner » hypothesis rally a lot of very smart people. But it doesn’t seem to explain the failures all that well for me. I think it’s very much a debate in progress, but the « genAI works because of humongous data », also supported by a lot of very smart people (Alexei Efros’ work is what tipped the scale for me), seems to explain the failures better to me.
-
@locha 2/2 or is the position trying say that models ‘are just statistical devices that latch on to regularities that can be learned bottom up given sufficient volumes of data’? If that’s the position, then there is a way in which it is obviously true, but -to me as a cognitive scientist- it also begs the question of exactly how, and in what ways, that differs from (human) ‘competent reasoning’
-
Ulrike Hahnreplied to Louis Chartrand last edited by [email protected]
@locha I guess my own position is that “gen AI works because of humongous data” isn’t really a position without further info. Is it trying to imply that the model does a bunch of glorified table look up? Then I think there is ample evidence that they don’t (including from some of the failures, very much the same way that failures are indicative of underlying representations and processes in human reasoners)? 1/2
-
@UlrikeHahn "humongus data" is really more of a family of positions, but I think I'd take the "lookup table" take as an approximation. As in, given the prompt that's given, it makes a collage of what it takes to be the best fit (both in terms of what seems to fit there, given past examples, and what should pleases the asker, thanks to RLHF). I'm not sure what would constitute a failure for this, as it's clearly not a lookup table, but I haven't heard of a really precise account of this. 1/2
-
@locha an example of a failure that shows it’s not just a look up table: a hallucinated citation - it was never in the training set, by definition.
-
@UlrikeHahn Thanks! I feel that the idea of « collage » kinda works here. It stitches together parts of quotes it has seen elsewhere. So there is a lookup-like behaviour, but as the LLM keeps asking itself « what would work best next? » it might latch on to something else.
-
@UlrikeHahn Maybe as an aside, I figure this process to be a bit like what RG Millikan envisioned in her 1984 book. There are phrastic patterns and people stitch them together to make sentences in ways that are as likely as possible to be pragmatically felicitous. Which means copying instances where it worked.
-
@locha yes, but the key bit is that it stitches them together in plausible ways (people go writing to authors for those references). You have to have learned a good amount of structure to be able to do that (while of course still fail to understand fundamentally something about references).
Analysing error patterns this way is exactly how the language development literature, for example, seeks to understand what representations have been acquired.
-
-
@locha again, I think if you look at the actual cognitive science/cognitive linguistics literature you will find many extremely well worked out accounts on what you are calling “phrastic patterns that people stitch together”
