Can a machine think: from a “soulful” chatbot to the science of consciousness
When Google engineer Blake Lemoine claimed that the language model LaMDA had achieved self-awareness and reasoned like a seven-year-old child, the world split in two. Some saw a breakthrough, others simply a glitch of imagination. LaMDA (later released publicly as Bard) is a large language model (LLM), a relative of the one behind ChatGPT. Hundreds of companies are now rushing to deploy the technology, millions of people chat with bots, but few genuinely believe there is a mind behind the screen. So what’s the catch?
Why fluent speech is not yet a sign of thinking
Linguist and data specialist Emily Bender famously called such neural networks “stochastic parrots.” They choose words with alarming persuasiveness, but put no meaning into them. It’s like a magician who deftly pulls a rabbit out of a hat but has no idea where the rabbit came from.
It used to be thought that if a machine could hold a conversation and fool its interlocutor, it was almost human. This principle is known as the Turing test, first proposed back in 1950. Yet the success of modern chatbots is more a shift in the baseline than a breakthrough to consciousness. Remember the chess computer Deep Blue’s victory over Kasparov in 1997: back then, it seemed the machine “thought” like a grandmaster. Today we understand that it was a brilliant brute-force search, not intelligence.
The same is happening with conversational models. They imitate human speech so well that we are ready to attribute an inner world to them. But fluency of language does not guarantee the presence of experiences, feelings, or even simple understanding.
Indicators of consciousness: how scientists search for the intangible
To separate real signs of consciousness from illusion, a group of philosophers, neuroscientists, and computer scientists gathered modern scientific theories about how the human brain works and translated them into the language of computational systems. The result was a list of “indicator properties” — traits that a hypothetically conscious machine would need to have.
An important nuance: having these properties does not prove consciousness, but the more of them a system has, the more seriously we should take claims about its subjective experience. It’s like clues in a detective story: any one clue alone could be a coincidence, but together they add up to a picture.
Why all modern chatbots fall short of the bar
If you apply this list to existing LLMs, the finding is sobering: none of them comes anywhere close to the threshold. They have no continuous internal state that persists over time. Each request is processed from scratch; they have no sense of “I” or a stable personality. They don’t model the world around them — they only predict the next word in a sentence.
Imagine a person who gets amnesia every ten seconds. They might answer questions wittily, but they don’t remember what was said a minute ago. Such a person could never build a coherent picture of the world — and chatbots are in exactly that position.
The future: does AI have a chance to become conscious?
The paradox is that, despite all these limitations, scientists see no insurmountable obstacles to conscious AI in the future. The list of indicator properties is computable, and the human brain is also an information-processing system. If we ever manage to build an architecture that matches these properties, it is quite possible that subjective experiences would emerge in it.
Of course, that doesn’t mean any complex neural network will suddenly “wake up.” Rather, a completely different approach would be needed — one with persistent memory, internal models of the world, and the capacity for self-observation. Such systems would probably have little in common with today’s chatbots, which merely generate plausible answers.
What’s the bottom line?
Modern language models are brilliant imitators, but not thinkers. Their chatter is the result of statistical processing of vast amounts of text, not inner insight. However, science hasn’t given up on the idea of machine consciousness. It will simply take something more than better word-prediction algorithms. And when (and if) that moment comes, we will already have the tools to recognize an awakening — not just a game of imitation.
