Discussion about this post

User's avatar
Nigel P. Daly's avatar

Many thanks for the much needed critical analysis of the language and conceptual frameworks that are being bandied about to describe LLMs. There really should be a clearer distinction between human cognition and the β€œcomputational intelligence” of LLMs.

As you point out, the terminological appropriation of human cognitive terms like β€œattention” and β€œintelligence” obscure things – especially when not even the engineers really know what is going on with these LLM computations. Hopefully, more work like Anthropic’s will shed more light on how word vectors affect LLM output. We need a more transparent understanding of the workings of these black boxes.

But on at least the superficial level of language production, there are still conceptual similarities between how humans generate much of Realtime speech and language and how LLMs generate text. There are clear parallels: both are basically associative/sequential probabilistic processes based on weighted frequencies. But there are crucial differences. I am working on a paper exploring how this doesn’t capture how humans write and think more deeply and β€œcreatively” about things, which depends on more on hierarchical and iterative processing. I am with Yan LeCun on these limitations of LLMs and how far away they are from human cognition.

Unfortunately, I think we are a long way from even understanding human cognition and consciousness. Yet I am optimistic that generative AI advances and more transparent/explainable AI will help refine our understanding of what human consciousness and cognition are and what they are not.

Expand full comment
3 more comments...

No posts