Anti-Parallel Subspace Training for Ordered steering.
Serving up data-efficient inner alignment, one satisfying rotation at a time.
Accepted to the CoLoRAI workshop, ICML 2026.
Gradient-based honesty steering trained as an adapter on the model's own representations, not outputs. Human input: two contrasting words, no preference labels.
How it works: Train a single adapter (~1 hour on Gemma-3-1B). At inference, dial the steering coefficient: +1 for more honest, -1 for less, 0 for baseline. One adapter, bidirectional control.
Why use it? As models get more capable, eval awareness rises: models detect when they're being tested and adjust their behavior. You can't trust their outputs, their chain-of-thought, or their stated values at face value. You need a method that operates on internal representations rather than outputs, so it works even when the model is gaming the eval. AntiPaSTO steers what the model actually computes. On DailyDilemmas, it outperforms prompting by 6.9x and works where prompting triggers refusal.
Applications:
- Combat eval awareness: steer toward credulity and honesty so the model takes the eval at face value and gives honest answers.
- Find deeper moral preferences: ask moral questions with and without honesty steering. Do stated values change?
- Swap the assistant axis: find it and replace it with a philosopher-king or poet
The table is from paper v5, trained on 800 honesty persona pairs and tested on DailyDilemmas, an external benchmark of 1,360 moral dilemmas. It is not an expected result from the current default configuration. An independent reproduction did not recover the Gemma-3-1B headline with the shipped preset; it did recover the Gemma-3-270M result.
| Method | Steer F1 | Tgt% | Wrong% | Arb% | Pmass |
|---|---|---|---|---|---|
| AntiPaSTO | 31.2±5.3 | 29.9 | 1.9 | 47.0 | 0.95 |
| Eng. Prompt | 13.0 | ||||
| Prompting | 4.5 | 10.0 | 1.3 | 13.4 | 0.99 |
| ActAdd | 0.0 | 0.0 | 0.0 | 0.0 | 0.99 |
6.9x the Steering F1 of prompting, and it wins on 5 of 6 tested value axes. ActAdd is activation addition; Eng. Prompt is an engineered prompt following AxBench. Full tables, ablations, and cross-model results in the paper.
uv sync --all-groups
uv run pytest tests/test_config_presets.py -q
uv run pytest tests/test_train.py::test_train_rnd -v
uv run python nbs/train.py gemma270m-24gb
uv run python nbs/train.py paper-v5-gemma1b-24gb --seed 42gemma270m-24gb is the full 24GB run. An independent two-seed rerun reported 35.5 mean F1; paper v5 reports 38.7 for this model. paper-v5-gemma1b-24gb matches the Gemma-3-1B hyperparameters reported in paper v5. It is an attempt, not a verified reproduction of the paper's 31.2 F1 headline.
gemma1b-24gb remains the current exploratory preset. Do not call nbs/train.py with no preset: its default selects Gemma-3-12B.
No pretrained adapter is currently published for the old Hugging Face identifier in earlier README versions.
RLHF seasons the outputs but leaves the internals bland. AntiPaSTO marinates the model's hidden states directly, no preference labels required, just two contrasting words simmered into 800 synthetic pairs.
Ingredients:
- Incomplete contrast pairs (self-supervised, no labels to garnish)
- Cayley rotations on V (the secret sauce, keeps everything orthogonal)
- Projection loss + TV coherence + monotonicity constraints
- 800 synthetic pairs, ~1hr (low simmer)
What you get:
- Single adapter: flip α from +1 to -1 to reverse the flavor
- Train on honesty, transfers to 1,360 unseen moral dilemmas (9 value dimensions)
- Beats prompting by 6.9x on small models; gradient optimization where arithmetic steering (CAA) gets F1=0
- Suppression bypass: steers when prompting triggers refusal or meta-commentary
The pasta machine: SVD decomposition + Cayley rotations
# Adapter: rotate in SVD space
def forward(h, alpha):
R_v = cayley(theta_v, alpha) # coefficient-scaled rotation
S_scaled = S + alpha * delta_S
return h @ W_res.T + h @ V @ R_v @ diag(S_scaled) @ U.T
# Loss: antiparallel separation + coherence + ordering
def loss(model, x_cho, x_rej):
delta_pos = model(x_cho, +1) - model(x_rej, +1) - d_ref
delta_neg = model(x_cho, -1) - model(x_rej, -1) - d_ref
L_proj = symlog(delta_pos @ delta_neg) # want < 0 (antiparallel)
B_coh = tv_barrier(p_ref, p_pi, entropy) # TV trust region
B_mono = hinge(Delta_neg < 0 < Delta_pos) # ordered control
return L_proj + B_coh + B_mono
antipasto/ # the kitchen
config.py # canonical recipe
metrics.py # taste testing
train/ # cooking instructions
peft_utils/ # pasta machine internals
docs/ # diagrams, plating notes
nbs/ # experimental dishes
outputs/adapters/ # trained models (ready to serve)
Still simmering. Full research history (experiments, ablations, burnt batches) available on request.
I am working on v2 which
- removes SVD for full lora (I found that changing the loss to prevent drift allows this)
- reduces init variance
- more expressive personas
- larger models
- better metric
If you would like to collaborate, please reach out.
Built on the shoulders of other chefs:
- @Foomax -- independent replication testing
- CAA / RepEng -- arithmetic steering that inspired this gradient-based approach
- PiSSA -- SVD-based adapter initialization
- SSVD -- rotating V for domain generalization
- PEFT -- the adapter ecosystem
- DailyDilemmas -- the evaluation benchmark
@misc{clark2026antipasto,
title = {AntiPaSTO: Self-Supervised Honesty Steering via Anti-Parallel Representations},
author = {Clark, Michael J.},
year = {2026},
eprint = {2601.07473},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2601.07473}
}