Blog

Steering about 50 of Qwen3-8B’s MLP neurons gives most of a persona vector’s effect, and steering 100 gives nearly all of it; the top directions of a randomly rotated basis barely work.

A negative result: approximations of the Fisher information don’t yield better clusters of parameter-decomposition components than coactivation does. Playing around with Silico from Goodfire.

A reproduction of Sarfati et al.’s “The Shape of Beliefs”

How LLMs encode in-context beliefs as curved manifolds, and how manifold-aware steering changes them with fewer side effects than linear steering.

Decomposed concept-injection introspection (Gemma-3 12B, Qwen-2.5 32B) into separable components: representation (what the model encodes about an injection) and report (the prompt-dependent late-layer circuitry that surfaces it), explaining apparent conflicts across prior protocols.