The Pneumetron
NEWSTOOLSINFRASTRUCTURE
Pneumetron.Probing and steering directions diverge in modern language models
Share
Skip to article content
  1. Home
  2. ›
  3. News
  4. ›
  5. AI Research
  6. ›
  7. Probing and steering directions diverge in modern language models
AI Research·November 22, 2026

Probing and steering directions diverge in modern language models

BY PNEUMETRON|5 MIN READ · 1,002 WORDS5 MIN READ
Tools
Share

In This Article

  • The geometry of the split
  • Transfer requires two different laws
  • Steering favors uncertain errors
  • The steering optimum is unstable
  • Sources
Illustrative photo: switch, it, server, server cabinet
Illustrative image: blickpixel / Pixabay

An arXiv paper shows the best direction to detect a concept in a model’s hidden state is rarely the best one to steer it. Transfer laws also differ, and steering toward abstention works best on uncertain errors.

Key Takeaways

  • 01Probing and steering optimize for different directions within a model, requiring separate evaluation.
  • 02Probe directions retain 81–91% of target performance when transferred, while steering directions drop sharply.
  • 03Steering toward unknown boosts abstention on uncertain errors by 18–44 percentage points versus 2–5 on confident errors.
  • 04Intermediate steering directions can outperform mean difference by 27–56%, but gains are unstable across layers and texts.

A new arXiv paper shows that the direction best used to detect a concept in a language model’s hidden state is rarely the same direction that best steers the model toward it. The authors tested this across three instruction-tuned model families—LLaMA, Gemma, and Qwen—and found that representation engineering pipelines relying on a single direction for both tasks are optimizing the wrong geometry. The results suggest that probing and steering should be treated as distinct operations with separate evaluation criteria and transfer rules.

The geometry of the split

The authors evaluated probing and additive steering along a continuous family of directions. This family extends from a covariance-adjusted discriminant, which accounts for how representations vary within each class, to the mean difference, which points directly between class averages and ignores within-class variance. Steering methods like contrastive activation addition and spherical steering typically default to the mean difference, while linear probes often approximate the discriminant.

To compare them fairly, the authors measured probing and steering along the entire family at a fixed side effect. The side effect was quantified as the average Kullback–Leibler divergence on neutral sentences, ensuring that no direction gained an advantage simply by perturbing the model more aggressively. Probing performance was measured by area under the receiver operating characteristic curve (AUROC), where 0.5 represents chance. Steering effect was measured by the change in behavioral metrics like truthfulness or sentiment scores.

The results show a clear separation. Probing and steering favor different directions within every model and concept combination tested. Steering consistently performs best at or near the mean difference end of the family. Probing performs best closer to the covariance-adjusted discriminant. This split holds at the main layer for all six model-concept pairs, and in 59 of 60 combinations during a layer sweep. The mathematical reason is straightforward: a probe’s filter and the direction along which classes differ are distinct objects. A probe optimizes for predictive information in the hidden state, while steering optimizes for causal steerability in the residual stream.

Transfer requires two different laws

The distinction between probing and steering becomes critical when moving directions between independently trained models. Because hidden states live in different dimensional spaces, the authors connected them with an affine map fitted on neutral text. The mapping rules differ by role. A steering direction is a displacement of the hidden state, so it transfers forward through the linear map. A probe is a linear functional that turns a hidden state into a score, so it transfers through the transpose of the reverse map.

When applied across 12 model pairs, the two laws produced asymmetric results. Probe directions retain 81–91% of the target model’s above-chance performance when carried over using the probe law. Steering directions, by contrast, retain less of their effect in 11 of 12 pairs. The authors note that this is a consistent tendency, though predefined thresholds classify only three pairs as strongly sharing steering capacity.

Gemma complicates cross-model transfers. Its own mean difference probes poorly, with AUROCs of 0.596 for truthfulness and 0.631 for sentiment. This low baseline breaks transfer ratios into and out of Gemma, either leaving them undefined or inflating them due to division by a small number. However, applying the steering law still allows a transferred mean difference to function as a push in the target model, even if it does not probe well. The authors conclude that representational alignment is a functional question: models may agree on how a concept is read more than on how it is changed.

Illustrative photo: circuit, electronic, phone, samsung Illustrative image: Surprising_Media / Pixabay

Steering favors uncertain errors

The authors applied these findings to error correction on PopQA, a benchmark for factual knowledge. They separated wrong answers into categories based on reachability: knowledge-available errors (where a correct answer exists in sampled outputs), uncertain errors (where the wrong answer appears rarely in samples), and confident errors (where the wrong answer appears frequently).

Steering toward an “unknown” concept increases abstention by 18–44 percentage points on uncertain errors, compared to only 2–5 percentage points on confident errors. The intervention rarely corrects errors in cases where a correct answer is reachable under the decoding strategy. This suggests that steering toward abstention is highly effective at filtering out low-confidence hallucinations, but does not reliably fix deep factual errors or prevent correct answers from abstaining.

The separation between probe and steering directions is less pronounced for the “unknown” concept than for truthfulness and sentiment. Gating by the probe protects correct answers, but passes the authors’ stability test only in Qwen. The probe also outperforms output entropy in separating knowledge-available errors from unreachable errors in LLaMA and Gemma, though the absolute AUROCs remain low.

The steering optimum is unstable

While the mean difference serves as a strong and stable baseline, it is not always the absolute peak. For sentiment, an intermediate direction in the regularized family produces a 27–56% stronger steering effect than the mean difference. However, this gain is fragile. It shrinks or disappears when evaluated on matched texts, and it depends heavily on the layer and the steering operator used.

The exact steering optimum shifts across model architectures and evaluation setups. Practitioners attempting to fine-tune the steering parameter to squeeze out marginal gains should be aware that these improvements are not guaranteed to generalize to new prompts or different decoding strategies. The mean difference remains a practical default precisely because it is less sensitive to these variations, even if it sits slightly below the theoretical peak on certain distributions.

The authors also tested a cross-fitted logistic combination of the probe and output entropy to separate error types. The combination improves AUROC over entropy alone in LLaMA and Qwen, but offers no gain in Gemma. This reinforces the observation that representation geometry varies significantly across model families, and a one-size-fits-all intervention strategy will underperform.

The paper highlights a structural reality in current representation engineering: probing and steering are not interchangeable. Pipelines that reuse a single direction for both detection and intervention are leaving performance on the table, and cross-model deployments require separate mapping procedures for each role.

Sources

  • arXiv: Do Probing and Steering Favor the Same Directions Within and Across Language Models?
#representation engineering#llm steering#probing#cross-model transfer#popqa
PE
PUBLISHED BY

PNEUMETRON EDITORIAL

Pneumetron's pipeline pairs AI-assisted drafting with human editorial review before publishing — our goal is to make staying informed easier, not to replace real reporting.

Source Material:arXiv ↗
Share this article
Share
Stay Informed

Never miss a signal.

Subscribe to the Pneumetron Intelligence Digest.

← Previous
S1CogBench Benchmarks System One Decision Models on Cognitive Experiments

More from AI Research

View All →
AI ResearchJust now
Illustrative photo: cyberspace, data, wire, electronic

S1CogBench Benchmarks System One Decision Models on Cognitive Experiments

A new benchmark reveals that fast, structured "System One" decision models often fail to adapt to cognitive task demands like humans, and high autonomous accuracy does not guarantee human-like prediction.

BY PNEUMETRON5 MIN READ
Read more
AI Research12h ago
Illustrative photo: laptop, macbook, codes, coding

Lang-SVG introduces hierarchical trees for semantic image vectorization

Researchers have published Lang-SVG, a method that reconstructs raster images as hierarchical, semantically labeled SVG trees rather than flat layers. The approach uses open-vocabulary segmentation and MLLMs to organize vector primitives into parent-child structures, improving both structural alignment and rendering fidelity under strict control-point budgets.

BY PNEUMETRON5 MIN READ
Read more
AI Research12h ago
Illustrative photo: server, cloud, development, business

AtomicChat Releases Uncensored LoRA for Inkling-2-Mini

AtomicChat published a rank-1 LoRA adapter that strips refusal behavior from the 26.4B MoE Inkling-2-Mini model. The edit drops harmful refusal rates to zero while costing 1.5 points on MMLU and spiking first-token KL on previously refused prompts.

BY PNEUMETRON4 MIN READ
Read more
AI Research12h ago
Illustrative photo: network, server, system, infrastructure

Open-source guard scans iOS and Android projects before app store submission

A new open-source playbook and shell-based guard automates pre-submission checks for mobile apps, integrating with AI coding agents to block submissions against 117 known rejection patterns and 95 regulatory deadlines.

BY PNEUMETRON4 MIN READ
Read more

Comments

In This Article

  • The geometry of the split
  • Transfer requires two different laws
  • Steering favors uncertain errors
  • The steering optimum is unstable
  • Sources

Most Read

01
AI Research·3d ago
Tencent releases LoopForge to orchestrate coding agents in self-hosted pipelines
02
AI Research·5d ago
Four New AI Papers Advance Reasoning, TTS, and Decoding Efficiency
03
AI Research·5d ago
Recent AI Research Reveals Gaps in Coding Agents and Advances Scientific Modeling
04
AI Research·5d ago
Anthropic Upgrades Fable and Mythos as Frontier AI Expands Open-Source and Robotics
05
AI Research·3d ago
Game2Player Scales Visual Agent Training with Co-Developed Game Environments
Daily Digest

Get top AI & tech signals delivered to your inbox every morning.

Subscribe →
Follow Signals
X / TWITTERLINKEDININSTAGRAMYOUTUBETELEGRAM

THE PNEUMETRON

The publication of modern technology, AI, and infrastructure.

AI-assisted, human-reviewed.

NEWS CATEGORIES

  • AI RESEARCH
  • TECHNOLOGY
  • WORLD
  • BUSINESS
  • POLITICS
  • SCIENCE
  • HEALTH
  • SPORTS
  • ENTERTAINMENT

AI & RESEARCH

  • AI RESEARCH
  • INFRASTRUCTURE
  • TOOLS
  • DEEP DIVES
  • TECHNICAL DISPATCHES

RESOURCES

  • ABOUT US
  • CONTACT
  • CAREERS
  • ADVERTISE
Pneumetron.
© 2026 Pneumetron. AI-assisted, human-reviewed.
All rights reserved.
ABOUTTOOLSPRIVACYTERMSCONTACT