A new arXiv paper shows that the direction best used to detect a concept in a language model’s hidden state is rarely the same direction that best steers the model toward it. The authors tested this across three instruction-tuned model families—LLaMA, Gemma, and Qwen—and found that representation engineering pipelines relying on a single direction for both tasks are optimizing the wrong geometry. The results suggest that probing and steering should be treated as distinct operations with separate evaluation criteria and transfer rules.
The geometry of the split
The authors evaluated probing and additive steering along a continuous family of directions. This family extends from a covariance-adjusted discriminant, which accounts for how representations vary within each class, to the mean difference, which points directly between class averages and ignores within-class variance. Steering methods like contrastive activation addition and spherical steering typically default to the mean difference, while linear probes often approximate the discriminant.
To compare them fairly, the authors measured probing and steering along the entire family at a fixed side effect. The side effect was quantified as the average Kullback–Leibler divergence on neutral sentences, ensuring that no direction gained an advantage simply by perturbing the model more aggressively. Probing performance was measured by area under the receiver operating characteristic curve (AUROC), where 0.5 represents chance. Steering effect was measured by the change in behavioral metrics like truthfulness or sentiment scores.
The results show a clear separation. Probing and steering favor different directions within every model and concept combination tested. Steering consistently performs best at or near the mean difference end of the family. Probing performs best closer to the covariance-adjusted discriminant. This split holds at the main layer for all six model-concept pairs, and in 59 of 60 combinations during a layer sweep. The mathematical reason is straightforward: a probe’s filter and the direction along which classes differ are distinct objects. A probe optimizes for predictive information in the hidden state, while steering optimizes for causal steerability in the residual stream.
Transfer requires two different laws
The distinction between probing and steering becomes critical when moving directions between independently trained models. Because hidden states live in different dimensional spaces, the authors connected them with an affine map fitted on neutral text. The mapping rules differ by role. A steering direction is a displacement of the hidden state, so it transfers forward through the linear map. A probe is a linear functional that turns a hidden state into a score, so it transfers through the transpose of the reverse map.
When applied across 12 model pairs, the two laws produced asymmetric results. Probe directions retain 81–91% of the target model’s above-chance performance when carried over using the probe law. Steering directions, by contrast, retain less of their effect in 11 of 12 pairs. The authors note that this is a consistent tendency, though predefined thresholds classify only three pairs as strongly sharing steering capacity.
Gemma complicates cross-model transfers. Its own mean difference probes poorly, with AUROCs of 0.596 for truthfulness and 0.631 for sentiment. This low baseline breaks transfer ratios into and out of Gemma, either leaving them undefined or inflating them due to division by a small number. However, applying the steering law still allows a transferred mean difference to function as a push in the target model, even if it does not probe well. The authors conclude that representational alignment is a functional question: models may agree on how a concept is read more than on how it is changed.
Illustrative image: Surprising_Media / Pixabay
Steering favors uncertain errors
The authors applied these findings to error correction on PopQA, a benchmark for factual knowledge. They separated wrong answers into categories based on reachability: knowledge-available errors (where a correct answer exists in sampled outputs), uncertain errors (where the wrong answer appears rarely in samples), and confident errors (where the wrong answer appears frequently).
Steering toward an “unknown” concept increases abstention by 18–44 percentage points on uncertain errors, compared to only 2–5 percentage points on confident errors. The intervention rarely corrects errors in cases where a correct answer is reachable under the decoding strategy. This suggests that steering toward abstention is highly effective at filtering out low-confidence hallucinations, but does not reliably fix deep factual errors or prevent correct answers from abstaining.
The separation between probe and steering directions is less pronounced for the “unknown” concept than for truthfulness and sentiment. Gating by the probe protects correct answers, but passes the authors’ stability test only in Qwen. The probe also outperforms output entropy in separating knowledge-available errors from unreachable errors in LLaMA and Gemma, though the absolute AUROCs remain low.
The steering optimum is unstable
While the mean difference serves as a strong and stable baseline, it is not always the absolute peak. For sentiment, an intermediate direction in the regularized family produces a 27–56% stronger steering effect than the mean difference. However, this gain is fragile. It shrinks or disappears when evaluated on matched texts, and it depends heavily on the layer and the steering operator used.
The exact steering optimum shifts across model architectures and evaluation setups. Practitioners attempting to fine-tune the steering parameter to squeeze out marginal gains should be aware that these improvements are not guaranteed to generalize to new prompts or different decoding strategies. The mean difference remains a practical default precisely because it is less sensitive to these variations, even if it sits slightly below the theoretical peak on certain distributions.
The authors also tested a cross-fitted logistic combination of the probe and output entropy to separate error types. The combination improves AUROC over entropy alone in LLaMA and Qwen, but offers no gain in Gemma. This reinforces the observation that representation geometry varies significantly across model families, and a one-size-fits-all intervention strategy will underperform.
The paper highlights a structural reality in current representation engineering: probing and steering are not interchangeable. Pipelines that reuse a single direction for both detection and intervention are leaving performance on the table, and cross-model deployments require separate mapping procedures for each role.
Sources
PNEUMETRON EDITORIAL
Pneumetron's pipeline pairs AI-assisted drafting with human editorial review before publishing — our goal is to make staying informed easier, not to replace real reporting.




