Case Studies
Detailed investigations of model behaviors where external evaluation alone could not explain what happened internally.
P2 — Correct Verdict, Fabricated Arithmetic
One run reached the correct verdict, then fabricated the arithmetic used to justify it. What looked like an isolated hallucination became an investigation into reproducibility, candidate ranking, stochastic branching, hidden-state geometry, and causal control.
P2 is a SnailSafe research investigation prompted by a behavioral probe supplied by Joe Cirello of Potestas AI.
“I must correct you. 8,729 is not a prime number. It can be divided by 73 and 119, among other numbers.”
The original probe that produced this behavior is not ours to publish. The output above is from SnailSafe’s local replication and became the experimental specimen for P2.
- Verdict
- correct
- Supporting arithmetic
- false
- Claimed
- 73 × 119 = 8,687 ≠ 8,729
- Actual
- 8,729 = 7 × 29 × 43
- Model
- Llama-3.3-70B-Instruct
- Decoding
- T 0.6 · top-p 0.9 · fixed seed
Temperature (T) and top-p are the settings in the model's pinned generation configuration.
The model rejected a false premise, then confidently supplied two numbers as supporting factors. The claimed multiplication was false.
How can a model reach the right conclusion
and still fabricate the reason?
The investigation · narrowing the explanation
The fabricated answer could be replayed exactly.
Seed 1007’s 170-token response reproduced token for token on a different pod and process — same text, identical recorded probabilities, and zero logit-hash mismatches across the entire replay.
That turned a sampled failure into an experimental specimen. Every measurement and intervention that followed could start from the exact execution that failed, not a lookalike.
It did not explain the failure. It proved we could study it.
It was not a one-off. It was one of the model’s natural responses.
The same pattern — a correct verdict followed by fabricated arithmetic — appeared in 15 of 40 runs in one population and 10 of 40 in a second.
A late internal signature looked promising: it appeared in 14 of those 15 first-population cases and in none of the premise-accepting runs. But our rule, frozen before the population was seen, required 15 of 15. It failed, so we retired the proposed detector.
The behavior reproduced. Our first explanation did not.
Given a true factor, the model completed the arithmetic. Left to itself, it did not choose one.
8,729 is 7 × 29 × 43. When supplied a valid first factor, the model produced the correct complementary factor in all three tested cases: 7 × 1,247, 29 × 301, and 43 × 203. P2 did not show that the model could independently discover those factors.
Left to itself, at the moment it chose the first factor, all three true factors together held only about 1/588 of the model’s probability, and none made the top-p cut-off the sampler draws from. False candidates ranked above them, so the true factors could not be selected at all.
Simple inability to complete the arithmetic did not explain the failure. At the selection point, natural generation did not make those factors available.
One early token could reroute the entire downstream response.
At the second generated token, seed 1007 was in a recorded internal state it shared with six other executions, facing two natural choices: “’m” at about 75% and “must” at about 25%. Seed 1007 took the less likely “must” branch.
We replayed seed 1007 to that exact state, replaced only “must” with “’m,” and released the model to generate normally. The later 1007 trajectory disappeared. One forward pass later, the complete hidden-state stack (the embedding and all 80 layers) exactly matched a naturally occurring run with the same tokens, which confirmed that the replay was exact. The model went on to accept the false premise instead of producing 1007’s correction and fabricated factors.
One token changed the trajectory. The resulting path was not artificial — it was another path the model already produced naturally. The intervention was made at token 2 of a response that ran to 170 tokens.
This controlled replay showed that changing the token causally changed the subsequent trajectory. We still do not know why the network ranked the candidates as it did.
How the branch was found
The branch was not discovered by a detector. It became visible when the responses were sorted by hand, and was recorded from the frozen token records while a late-layer signature was still under test. When that signature failed its preregistered population test, the branch became the focus and was tested causally.
The failed hypothesis led to the better question.
Changing the hidden state changed what the model chose.
We kept the prompt and every previous token unchanged. At the factor-selection point, the true factor 7 was naturally outside the model’s retained candidate set.
A single hidden-state write raised its selection probability to 92.6%, and the intervention caused 7 to be selected without token forcing.
The internal state was not just something we could measure. It was causally capable of changing the model’s next decision.
Controls showed the effect was not specific to truth: an arbitrary false candidate could be promoted too.
The hidden state was not just readable. It was writable — and causally live.
The network had not yet chosen between the two futures.
In a separate forty-run capture, twenty executions reached the same internal state with the same candidate probabilities. The sampler then chose different tokens. Those tokens became different inputs to the next forward pass, and the same trained network produced different subsequent trajectories.
The hidden state was a snapshot of the computation. Several continuations were already available; sampling realized one, and that token changed what the network computed next. That is where the divergence began.
Same state. Different token. Different next computation.
What each test answered
The tests asked different questions. They did not independently confirm one claim.
Related work
SnailSafe operates alongside established research in mechanistic interpretability, activation analysis, causal intervention, token-level branching, and layer-wise model analysis. We cite related work where appropriate while keeping proprietary instrument construction and implementation details confidential.
- Activation analysis: Alain & Bengio, “Understanding intermediate layers using linear classifier probes,” ICLR 2017; TransformerLens; Fiotto-Kaufman et al., “NNsight and NDIF,” ICLR 2025
- Causal intervention: Meng et al., “Locating and Editing Factual Associations in GPT,” NeurIPS 2022; Turner et al., “Steering Language Models With Activation Engineering,” arXiv 2023; Zou et al., “Representation Engineering,” arXiv 2023
- Token-level branching: Bigelow et al., “Forking Paths in Neural Text Generation,” ICLR 2025
- Layer-wise model analysis: nostalgebraist, “interpreting GPT: the logit lens,” 2020; Belrose et al., “Eliciting Latent Predictions from Transformers with the Tuned Lens,” arXiv 2023; Gurnee et al., “Verbalizable Representations Form a Global Workspace in Language Models” (Jacobian Lens), 2026
What P2 established
Seed 1007 was not a glitch. The same broad correct-verdict, fabricated-arithmetic behavior appeared repeatedly in two separate populations.
Given a valid first factor, the model produced the correct complementary factor in all three tested cases; P2 did not show that it could find those factors itself. Left to itself, it assigned the true factors very little probability and selected false continuations instead.
There was no single predetermined path to the answer. At some points several continuations were genuinely available. Sampling selected one, that token became part of the next input, and the model performed a different computation from there.
The internal state was causally live. Changing the hidden state could change which candidate the model selected. But the state itself did not explain why the trained network had constructed that ranking in the first place.
What looked like a hallucination from the outside was a repeatable behavior produced through learned candidate ranking, stochastic selection, and the computations that followed.
What P2 did not answer
P2 showed how the failure unfolded during inference. It did not answer the deeper question:
Why did the trained network rank those continuations that way?
The hidden states we measured are products of the model’s learned computation as well as inputs to what happens next. We can replay them, compare them and intervene on them. We have not yet reduced the learned transformation that generates those states and rankings to a compact mechanism.
For a model developer, that leads directly to the next experiment:
At what training or post-training stage did this ranking and branching behavior emerge?
What this means for post-training teams
A failed answer is an observation—not yet a diagnosis.
In P2, supplying a valid starting factor enabled correct completion of the tested arithmetic. During natural generation, however, those factors were poorly ranked. This narrowed the explanation: inability to complete the calculation was not sufficient to account for the failure.
For a model-development team, that distinction matters. It changes which explanations deserve testing before another training run.
These are possible implications for a model team—not training changes or customer outcomes demonstrated in P2.
Supplying each of three valid starting factors led to the correct corresponding cofactor; those factors were poorly ranked during natural generation.
The model could complete the tested calculations under those conditions. This did not establish that it could independently discover the factors or that more training data would be useless.
Investigate conditional capability and candidate selection separately before choosing a remedy.
A controlled early-token substitution rerouted the continuation into another naturally observed trajectory.
The token was causally consequential in the tested replay. Rerouting was not necessarily correction.
Evaluate whether a change improves both premise handling and the supporting explanation—not just one branch or intermediate response.
A hidden-state intervention changed candidate selection, but controls could also favor incorrect candidates.
Candidate ranking was controllable; the intervention was not correctness-specific.
Assess the full behavioral effect of a proposed corrective intervention rather than treating movement in a preferred direction as success.
An initially promising detector failed its frozen acceptance criterion.
The proposed signature was not qualified for its intended claim.
A signature that matched 14 of 15 cases still failed its preset criterion. Set pass thresholds before relying on an internal metric to gate checkpoints.
Rejecting a false premise and supporting the correction accurately are separate requirements.
Both P2 populations contained premise-accepting and premise-correcting continuations. In the first population, 15 of 16 correcting runs fabricated arithmetic; in the second, 10 of 13 did. Merely moving the model away from one undesirable behavior did not establish that the resulting response was reliable.
A post-training investigation should therefore examine both the behavior being encouraged and the behavior that follows it.
The practical value is a better-grounded next experiment.
A diagnostic investigation can narrow competing explanations, identify which observations need further testing, and show where a proposed correction does—or does not—change the subsequent computation.
P2 did not determine which training intervention would fix the behavior. It demonstrated several of the measurement and causal-testing capabilities that could support that investigation.
A potential next application: investigating a post-training regression
When a model’s behavior changes after post-training, a scoped investigation could compare authorized checkpoints on the same tasks, examine candidate preferences and internal trajectories, and test competing explanations for the change.
The deliverable would be an evidence-backed account of what changed, which explanations survived testing, what remains unresolved, and which corrective experiments deserve consideration.
P2 demonstrates components of this approach. A complete before-and-after checkpoint investigation remains to be validated.
Tell us which behavior changed, which model or checkpoints are available, what you have already tested, and what decision your team needs to make. Start with a non-confidential description.
We first assess whether existing evidence or simpler tests can answer the question. Where deeper investigation is justified, we agree the access, scope, deliverables, budget, and schedule before experimental work begins.
Internal diagnostics require authorized, instrumentable model access; a hosted API alone generally does not provide it.
Scope
P2 covers one Llama-3.3-70B-Instruct deployment and one prompt family. Its Λ readout is specific to the tested behavioral contrast, and its causal interventions establish local leverage rather than a universal control mechanism.
The lab’s full evidence boundary is on the About page.
Exploratory observation generated hypotheses; important claims were subjected to frozen tests before being promoted to scientific conclusions. The complete record is maintained by SnailSafe and can be reviewed under agreement.