Transformer interpretability sandbox
Watch a prediction form.
Project each hidden state into vocabulary space and follow competing next tokens through the transformer. Compare a direct logit lens with a calibrated tuned lens, layer by layer.
Deterministic teaching simulator. Trajectories are illustrative, not measurements from a named model.
INPUT AT THE READOUT POSITION
The capital of France is
Layer-by-layer token probability
Each colored path is one candidate next token. Hover or move the layer slider to inspect the decoded distribution.
Layer 0
Interpretation
TokenDecodable is not causal. A token appearing in a lens readout does not prove the model uses that representation to produce its answer. Intervention tests are a separate step.
A lens reads a trajectory. It does not replay the computation.
The residual stream is updated iteratively. A vocabulary projection can make that changing state legible, but every lens bakes in assumptions about how intermediate features align with the output vocabulary.
The direct readout is simple.
Normalize the intermediate residual state, multiply by the model's unembedding matrix, then apply softmax.
hl → norm → WU → softmax
Prediction sharpens with depth.
Early layers often carry broad lexical or syntactic signals. Later layers increasingly align with the final next-token distribution.
Logit lens
Uses the final unembedding directly. It is cheap and transparent, but early-layer readouts can be systematically distorted.
Tuned lens
Learns one affine translator per layer before unembedding, improving predictive calibration while adding probe parameters.
Causal test
Patch, ablate, or steer the representation and measure the final output. Readout and causal effect answer different questions.
Read the lens without overclaiming.
Lock the prompt and model. Inspect the whole trajectory. Compare lenses. Measure uncertainty. Form a hypothesis. Test it with an intervention.
Find a transition
Look for a stable rank change, entropy collapse, or temporary intermediate token across adjacent layers.
Compare the probes
If only one lens shows the pattern, probe bias is a plausible explanation. The disagreement is evidence, not noise to hide.
Intervene separately
Use activation patching or ablation to ask whether the surfaced direction changes the final answer under controlled edits.
Primary research and methods.
These sources define the residual-stream view, tuned calibration, encoder-decoder extension, and gradient-space counterpart.
Belrose et al., 2023Eliciting Latent Predictions from Transformers with the Tuned LensOpen paperElhage et al., 2021A Mathematical Framework for Transformer CircuitsOpen researchLangedijk et al., 2023DecoderLens: Layerwise Interpretation of Encoder-Decoder TransformersOpen paperKatz et al., 2024Backward Lens: Projecting Language Model Gradients into Vocabulary SpaceOpen paper
Turn a picture into an interpretability protocol.
Use the checklist to preserve enough context for another researcher to reproduce the readout. Then explore how Superpowers can build inspectable tools and cache repeatable computer-use workflows.
0 of 4 ready