Artificial-intelligence systems such as large language models (LLMs) can exhibit behaviour that seems strikingly intelligent. This has led many people, including some experts, to wonder whether these systems could have some form of consciousness — the capacity for subjective experience. However, we think that existing AI systems are not conscious, and that, along current trajectories of development, they are not likely to become conscious. Greater clarity about whether AI could become consciousness will be crucial as society grapples with the challenges posed by this increasingly powerful technology.
Almost none of this is relevant to the discussion here – I’m not interested in philosophical discussions of consciousness when it comes to the question of whether an LLM is conscious. Though I’m already aware of most of what you said here, and the philosophy of consciousness is interesting in general to me!
Quick nitpick: I never said it’s impossible for a machine to be conscious, I said it’s impossible for an LLM to be conscious. We may well invent something new in the future that qualifies as both “machine” and “conscious,” but an LLM is not that thing.
When people ask “is an LLM conscious?” I assume they’re asking from a practical, scientific perspective – does the LLM experience the world in a similar way to the way I (and presumably, others) do? If that is the question, the various definitions I already posted are what we should be using to answer. Since an LLM is a deterministic algorithm whose only purpose is to generate a plausible response to input based on its training data, it cannot possibly think, or have feelings. It does not even exist continuously in time, and it is fixed and unchanging, unable to incorporate any new information at all. We don’t actually need to measure or test anything here; that an LLM lacks all of these things demonstrates that it’s not possible for it to be conscious.