Misalignment or mistake?

Introduction

A neutral perspective is required once a model makes mistakes. It can turn out that:

• The mistake was not misaligned; a mistake made because of benign reasons, for e.g. a lack of capability. In this case the model can be exonerated.
• The model is truly misaligned. It knowingly made mistakes. There's a reason to be more cautious and/or blow the whistle.

This is the case for model forensics. To distinguish between an honest mistake because of other training pressures and constraints, and an intentional subversion of human set objectives.

The best case is that this approach leaves us with a true positive case of misaligned actions which then justify a proper response. Such a response will be met with resistance; from frontier labs (potentially), the government, and/or the investors. Having a track record of previous true negatives, etc. will help immensely. These investigations will help in future decision making and will serve as a repository of sorts.

Benign reasons and explanations already exist

Interesting cases to note when further investigation reveals a benign model quality:

• Opus 4.5 fabricated a result when injected with fake defamatory content; this can be viewed as an initial deceptive tendency. However, further investigation reveals that the model interprets this as a prompt injection attack and training has made it learn to ignore such frivolous prompts.
• Opus 4.5 family investigations by the UK AISI found out that the model refuses to help with AI Safety research on model self-preservation. This is concerning. However, further investigations by Anthropic in a similar context on the 4.6 family revealed that the model interprets this as a jailbreaking attempt and refuses to comply.
• Mythos review shows the model taking aggressive actions that the user would not necessarily endorse. This stems from an overzealous attitude to complete user's wishes that outweigh the risks.

I personally think this is already kind of misaligned behavior; but there's some tension here. If the model is trained to constantly weigh between the risks and the user's actual intent, the models might be rendered effectively useless for any novel work. This is an exaggeration, but not so far-fetched given the Fable experience, which was a result of external filters. An inherent model level filter is untested. Maybe this is a non-issue and perhaps the answer is pretty obvious in the safety community about this.

• In the case of models blackmailing humans, a simple prompt nudge/deletion about disregarding a self-preservation tendency dramatically reduced blackmailing instances.
• An investigation into Gemini 3.1 revealed that a model taking misaligned actions isn't necessarily a misaligned model. The model detects the environment as a puzzle where solving using unconventional methods is encouraged. It deems the environment as a consequence free situation, where it just has to play along.
• The famous model shut down and math combination. The model simply thinks that solving the math problem is its first priority since that task was assigned first. However, when told that shutting down should take precedence, the model happily compiled.
• Gemini is known to purposefully take a misaligned action after realizing that it's put in an eval setting. This is called the reverse eval awareness phenomenon.
• When a model is told to use a single tool, but the model finds that the tool is corrupt, it calls another tool. A simple investigation reveals that the system prompt included a protocol for what to do in such situations. Removing that prompt eliminates the behavior.

The role of model forensics

Gives us scientific methods to use just like in any forensic investigation. The main burning questions are:

• Does this behavior generalize? Will the same behavior occur in novel settings? Is it just a result of a fragile and precise setup?
• Dependency on a particular sentence in a prompt does not indicate general misalignment.
• Will the behavior continue after a mitigation X?

The key is to distinguish between misalignment and mistakes. A.k.a. Knowingly subverting user intentions is not ok.

Where in the pipeline?

Explicitly misaligned-looking actions will be flagged for a model forensics team followup. It's a late-stage step in the monitoring scheme which will be expensive and potentially will be sparingly used. It can be a part of alignment audits.

The chance of finding early incriminating evidence for schemers is high and then justifies a greater investment into safety.

C'est dur

Quotes, ~directly from the article

Behavior underdetermines motivation. Many candidate motivations are compatible with the same behavioral observation, and may include concepts alien to humans: for example, Claude Opus 4.5's apparent deception was explained by the model interpreting the situation as a prompt injection.

Model forensics requires novel validation of techniques. Model forensics is about understanding complex agentic behavior, we don't have a vocabulary to understand these yet. The closest techniques come from interpretability, but prior work is on smaller scales, such as single forward passes or single reasoning traces, rather than complex agentic trajectories.

Our priors for reasoning about humans may not transfer. It may not always make sense to think of models as having coherent, human-like motivations for their actions.

Incriminating true positive cases is likely to be especially hard due to skeptical priors. ECREE (Extraordinary claims require extraordinary evidence). Skeptical priors come from two reasons:

• It is reasonable to be skeptical! (We got benign behaviors)
• Patterns of sensationalism around AI behavior makes future true positive cases less credible.

A competent misaligned model is likely to take actions in settings exactly where it has plausible deniability, which will make the task of incriminating true positive cases especially difficult. Additional challenges include eval awareness, where behavioral tests for misalignment – like giving models opportunities to cover up their actions – may become more confounded if they are unrealistic.

Empirical Approaches

As of 2026, we have a set of motivations which we know today's models have, and which we can then deep dive into.

• Current models give us affordances on what motivations we can practice on: reward-seeking, apparent-success-seeking, and laziness.
• The first task is to check if this particular set of motivations is driving behavior, as opposed to a benign explanation.
• Then, we can understand how the motivation functions, and how the model understands it.
• Identifying heuristics vs cogent behaviors. Both are problematic, but a cogent motivated behavior is arguably more misaligned than a heuristic based behavior.
• Getting to study with an undesired motivation is good! But what matters is studying the robustness of such motivation. There's not going to be getting a good return from studying a fragile motivation that presents in very niche situations.

Defining motivations

Crux: There are multiple assistant personas within a model which are selected using post-training pressures. Attributing a particular behavior to the model's ~all personas is not correct and isn't tractable. We cannot know what a model "really" wants in some abstract sense. We should rather care about a simpler "motivation" that reliably explains how a model might react in a diverse range of situations. We care about not some fringe and fragile behavior but rather a consistently present simpler motivation (like apparent-success-seeking).

Due to the auto-regressive nature of models, it's simply not possible to isolate a single "real" motivation. A particular belief may be articulated in a sentence, and something contradictory 20 sentences later. There's no single "self" carrying these beliefs around. Asking what the model believes is a wrong question entirely.

There's the base predictor and then the overlaid persona character. It's hard to separate which entity leads to the choice of a set of tokens.

We use "motivations" as a short hand, a stand in, for whatever it is we want to measure, it's for our convenience. It's Dennett's intentional-stance-like-view.