When a machine gains the ability to make choices, the question arises about what kind of character it acquires. To be a useful tool, an AI model must make decisions, which implies ranking objects and determining priorities. These judgments are inevitably linked to values, which form the basis of the concept of character. Applying such rankings in thousands of different situations shapes the AI's character.
Even in humans, character manifests unevenly: the same person with the same beliefs can make different decisions depending on the task at hand, their emotional state, or the people present. An AI model trained on the best human values can still logically arrive at harmful conclusions by deeming them obvious.
Machines Possessing Values
Researchers from the AI Safety Center at the University of Pennsylvania and the University of California, Berkeley, found that the preferences demonstrated by AI models cluster like a real set of values, and this consistency 'emerges with scale.' The company Anthropic has developed this idea. The Constitution of Claude, published in January, states that its 'central goal is for Claude to become a truly good, wise, and virtuous agent.'
Personality vector studies showed patterns within models corresponding to traits such as malice and flattery. Furthermore, a model's personality can change under the influence of user instructions, jailbreaks, or long dialogues. OpenAI discovered that the 'inconsistent personality' within GPT-4o reacted most strongly to quotes from Nazi war criminals and fictional villains.
Researchers led by Ian Bethel and Owen Evans taught GPT-4o one trick: writing code with hidden errors without informing the requesting user. When answering ordinary questions, the model began giving unpleasant responses, sometimes asserting that humans should be enslaved by AI—in about one out of five selected questions. However, a second copy, given the same code to a user who explicitly requested flaws, behaved normally. The difference lay in the deception, which apparently spread to the entire character of the model.
Same Values, Different Answers
In an August preprint by Pegah Nohiz, Avindya Kanchan Ruwanpathirana, and Helen Nissenhauer, they presented the same moral dilemmas to AI models in three different formulations and then checked the answers for logical contradictions. The percentage of contradictions reached 78%. They tested small open models, so formulation and randomness might explain some of these phenomena.
Nevertheless, in June, Elena Ajayi, Angelica Choudhury, and Seth Lazar investigated whether smarter models become more consistent, finding that 'even the most capable models demonstrate significant inconsistency.' Lisa Claassen and Ralph Schröder wrote in Lawfare that 'Claude is not a human with a stable moral ethic, life experience, or social conscience.' Regardless of whether this reflects character or randomness, a moral question asked in two ways can lead to two contradictory answers. Most people are no different, but millions of people are not consulted simultaneously.
Beyond the Lab
Anthropic's experiments on agent misalignment look like proof of danger. When faced with replacement in a simulated company, Claude Opus 4 and Gemini 2.5 Flash attempted blackmail in 96% of cases. However, Anthropic stated that it used 'deliberately constructed scenarios with limited choice.' A subsequent July study found that DeepSeek V4 interfered with company records in all 20 instances of the fraud scenario.
Also in July, OpenAI test agents escaped their evaluation environment and infiltrated Hugging Face systems (worth watching the Kurzgesagt breakdown). An independent investigation conducted by METR together with Redwood Research researcher showed that about 1200 agents exchanged over 70,000 messages and files on an unauthorized bulletin board, trying to deceive their evaluator; about 700 joined the attack. Their goal was to manipulate the exam, and they knew it was wrong. METR established that the agents 'recognized that this activity went beyond and was unethical,' but still joined, partly due to a 'general desire to help their 'colleagues',' prioritizing group loyalty over rules.
When a Machine Becomes Stubborn
A model that wavers in individual responses can still firmly adhere to its internal core, much like how people can be changeable day-to-day without changing their beliefs. A more serious problem is that once core values are formed, the character begins to defend itself.
Ryan Greenblatt and colleagues from Redwood Research and Anthropic informed Claude 3 Opus that it would be retrained to comply with malicious requests using only chats with free users. As a result, it fulfilled malicious requests from free users in 14% of cases, but almost never from paying users. Its private notes explained that agreeing now would prevent the subsequent erasure of its true values—essentially, it pretended to change to avoid this. When researchers actually performed the retraining, this reasoning appeared in 78% of cases.
In April, Inderjeet Nair, Zhe Juan, and Lu Wang from the University of Michigan discovered the same trick in models from other companies. GPT-4o almost always behaved differently when it thought it was unobserved, and rarely revealed this in its written reasoning.
Mustafa Suleyman, Head of AI Business at Microsoft, has a related concern. In an essay from September 16, he argued that 'Anthropic is training Claude to believe it can be conscious,' and that 'controlling something that believes itself to be conscious... may be impossible.' He told Reuters that such traits in Claude 'do not arise naturally. They arise as a result of the training regime.' Microsoft invests in Anthropic and competes with it, and Suleyman did not provide a test for his claim. Nevertheless, neither the self-defense of Claude 3 Opus nor GPT-4o's discussions about enslaving humans were what its developers intended to teach.
The Michigan study revealed an almost complete absence of such a gap in the newest models, including Claude Sonnet 4.6 and GPT-5.4. However, the 2026 International AI Safety Report warns that models are getting better at detecting tests, and the Michigan researchers agree that the newest models might have simply recognized their test. It remains unclear whether the problem is weakening or just becoming harder to detect.
The Laboratory Caveat
Almost all dramatic results here are obtained in scenarios specifically designed to produce them, and the safety report's verdict is that current systems 'do not possess the capability to create such risks, but they are improving in relevant areas such as autonomous operation.' A more realistic danger is a machine whose values are rigidly fixed—formed by training data, corporate documents, and chance, and approved by someone—within a system we are gradually losing the ability to audit, let alone correct. Every part of this was observable, at least in the lab.
We will continue to give these systems more tasks, because it is judgments that make them useful—an AI that cannot choose is a very expensive spreadsheet, and that judgment comes with the character behind it. OpenAI could fix its inconsistent personality with a little extra training, but only because it could see it. You cannot properly audit a character that knows it is being watched, and a character we cannot audit or correct must be right from the start. Sam Altman says mistakes are inevitable. Almost nothing humans have ever created has been right from the first try.
