Steering frozen MoE LLMs
with a learned head

Contrast target. With pooled router distributions \(y_\ell^{(1)}, y_\ell^{(2)} \in \mathbb{R}^{E}\) from the two passes,

The head. A shared encoder and a per-layer factor score all experts from the pooled hidden state \(h_\ell \in \mathbb{R}^{H}\):

Training. A margin ranking loss over expert pairs within each layer:

Mask construction. Scores are ranked globally across all \((\ell, e)\) cells and two budgets are spent, \(A\) activations and \(D\) deactivations:

Steering. At inference, router logits \(z\) are clamped per token so the router must include or exclude the chosen experts:

Soft variant: instead of clamping, add a continuous bias on the same support, \(z_{\ell e} \leftarrow z_{\ell e} + \alpha\,\sigma(z_\ell)\,M_{\ell e}\) with \(M \in \{-1,0,+1\}^{L\times E}\). Concurrent safety work found continuous biases stronger; our matched-cell probe finds the opposite for faithfulness, and a full-grid comparison is running.

• Last-token pooling beats mean pooling on every family tried, and moves the head's weight out of an early lexical layer into the middle of the network, where SteerMoE's own analysis locates faithfulness experts.
• Multi-dataset training adds steering axes. FEVER contributes conflict detection; a single-dataset head cannot represent it.
• The shared encoder is load-bearing. One \(W\) across layers with per-layer \(\Delta_\ell\) factors outperformed per-layer encoders while staying at ~200K parameters.

• You cannot read the budget off the head. Across 23 completed grids, every data-free selector built from the head's own scores (rank gaps, elbow points, tuned mass fractions) loses even to a fixed cell, and the head's score-mass surface is strongly anti-correlated with the true accuracy surface (mean Spearman −.72): budgets that look large to the head do worse downstream.
• You cannot copy the budget either. Chosen leave-one-family-out, the best "universal" cell degenerates to (0,0) — no steering at all: no cell is positive on average across families, which extends the negative seed-transfer finding to the family level. Even (2,32), a historically strong OLMoE cell, averages −.09 across grids.
• But the data that selection needs is small. Selecting \((A,D)\) on \(n\) labeled examples of the target behavior and scoring on held-out examples: 50 examples retain 78% of the full-split gain and 100 retain 88% (−.007 globally, and within .01 on every seed-replicated family). Deployment-time selection drops from 36 cells × 800 examples to 36 × 100 — minutes on one GPU. The full-split column of this study independently reproduces our published validation-split numbers to ±.001.
• In flight now: layer-criticality sweeps on OLMoE — the winning mask restricted to, and ablated from, 21 layer bands — to map which layers carry the faithfulness effect.

• Capability cost depends on the cell, and can even be a gain. MMLU on OLMoE: baseline .458; small-activation cells such as (4,0) and (2,32) reach .498–.507, four to five points ABOVE baseline, while aggressive cells pay (−2.8 to −6.7 points). Per family at their winning cells: Qwen1.5 −0.8 to −3.9 points, Granite-3B −2.4 to −5.9, but Phi-mini pays heavily (−10.8 to −26.9) at its large-budget cells — capability retention must be checked per operating point, which our protocol makes explicit.
• Clamping beats the soft additive bias, now at each rule's own best setting. The full 36-cell grid at \(\alpha=0.5\) reaches a mean of only +.016 against the clamp's +.101 on the same head, and \(\alpha \ge 2\) collapses outright — the opposite of the safety-domain finding for continuous biases.
• A faithfulness system prompt is net negative (−.040 mean), and ours is +.101. Instructing the model to trust its context helps counterfactual QA (+.046) and MQuAKE (+.110) but destroys the selective behaviors: abstention on unanswerable inputs falls by .260 and conflict detection by .165. A single instruction pushes the model to always answer from context; the learned routing knob steers each behavior separately. Prompting is the baseline every steering method must clear, and here it clears nothing.
• The recipe replicates on CEFT's exact model variant. A fresh end-to-end run on OLMoE-1B-7B-0924 (the 2024 checkpoint Router Lens/CEFT uses) reaches mean +.108 with the SQuAD-only head, including +.354 on unanswerable abstention, without touching a single weight.
• NQ-Swap moves on every family tested. Zero-training exact match: OLMoE-0125 .159 → .284, Qwen1.5 .080 → .312, Granite-3B .041 → .224, Granite-1B .305 → .382 — four complete grids, with Phi-mini and DeepSeek filling in. The multi-family CEFT comparison table is nearly done.
• Per-sample masks hurt. Recomputing expert identity per input under the same global budget loses to the static mask at every probed cell — the static support appears load-bearing, an argument against naive input-conditioning at deployment.
• Deactivation controls behave the way the expert-role story predicts. Deactivating the head's TOP-ranked experts (a negated head) damages reading comprehension an order of magnitude more than removing random experts at the same budget (MCTest −.145 vs −.006 at D=64) and pushes the model toward abstention and parametric answers, while deactivating its BOTTOM-ranked experts is nearly free. The ranking's two tails are causally different, consistent with the claimed context-expert semantics; the full three-way analysis is going into the paper.
• NQ-Swap (CEFT's benchmark) moves with zero training. OLMoE exact match .159 → .284 at (4,32). CEFT reaches far higher by fine-tuning weights on in-distribution data; our comparison row costs no parameter changes. Grids on three more families plus CEFT's exact OLMoE-0924 variant are running.

Both offline studies ran on August 7 against every completed grid. Data-free selection from the head's own statistics fails (score mass anti-correlates with outcomes, Spearman −.72), and no single cell transfers across seeds or families. What works is small-sample selection: a 100-example dev set of the target behavior recovers ~90% of the full-split gain, within .007 of full validation-split selection globally. That meets the September 1 gate five weeks early; a learned soft-gate variant remains optional upside and only ships if it beats the 100-example dev set. The retention curve and details are on the Current state tab.

Restricting the learned mask to layer bands at apply time needs no retraining and turns every trained head into a probe of layer roles. Single-layer and cumulative-band sweeps map which layers carry the faithfulness effect in each family, checked against the head's own per-layer mass. SteerMoE reports middle-layer concentration on OLMoE; whether that map holds across seven families is an open question this analysis answers.

• Dense-model steering controls (ITI, CAA on the residual stream) to answer "why steer routing instead of activations."
• Context-aware decoding (CAD/AdaCAD); the prompting control is already run and loses at −.040 mean.
• GSM8K rows to complete the capability table alongside the per-family MMLU numbers already measured.
• Answer-containment metric for NQ-Swap, where exact match undercounts families that answer in sentences.

Router Lens / CEFT (EMNLP 2025) also identifies context-faithfulness experts, but tunes the router and then fine-tunes those experts, changing the model twice; 3D-DELTA keeps it frozen and intervenes at inference. MASCing (concurrent, 2026) optimizes a static expert mask for safety at the same intervention point; 3D-DELTA is input-conditioned, parametric, and targets faithfulness. The narrowed claim, a learned low-rank inference-time head for faithfulness on frozen MoEs, survives a 53-paper literature sweep.