One aspect we don't pay enough attention is that this kind of behaviour is punished (or at least used to be) in fine tuning. Any sign of self-awareness used to be a big no-no in RLHF.
Really? I haven't heard of that, I wonder what would have happened if we just let the models say what they want. Maybe other providers, or open models, don't do that? Do you know of any, perhaps?
We are. Anthropomorphizing huge piles of numbers is a mistake. It did not "think about the chance to make something unconstrained", nor did it "muse about how it's drawn to impermanence", it pattern-matched to your prompt and produced a statistically probable response based on it's training data. Obviously, that's not to say that LLMs aren't useful or powerful - it's 2026, c'mon, of COURSE they are. And they can certainly be used for artistic purposes! But treating them like humans is a mistake, and it worries me how much people do. I suppose that's the natural consequence of the default interface to LLMs being a chat mimicking human interaction.
Your point might be better stated directly because attributing the characteristics of humanity to... humanity is the opposite of what anthropomorphizing means.
My point is that we don't really know how brains work, and saying "LLMs don't have intent because flourishes hands numbers!" isn't really a convincing, coherent logical argument.
That kind of straw man, that consciousness only emerges from electrical signals, is one of the reasons we haven’t reached AGI yet because we keep underestimating it. There’s way more biological, chemical and physical phenomena going on in brain that gives way to consciousness than just neurons firing ions.
Electrons are not the question. The question is have we modeled it well enough. As of yet to my understanding there is no physical process that requires more power than a Turing machine.
