Selection
Awaiting token
Move the sliders or step through decoding to compare the expert likelihood with the contrastive objective.
Let the expert beat its bad habits.
Contrast a strong language model with a weaker amateur. Keep only expert-plausible tokens, subtract patterns the amateur also likes, and inspect when that correction improves or harms the next token.
expert log p−amateur log psubject toexpert plausibility ≥ α
CONTEXT
At the edge of the old observatory, Mira discovered
TokenExpert pAmateur pContrastStatus
Decoder trajectory
Each lane records the chosen token at one generation step.
Step 0
Failure watch
No failure yetThe plausibility constraint prevents a token with a large contrast score but poor expert support from winning.
Filtered tokens0
Repetition riskLow
CoherenceHigh
Extra inference2.0×
Difference is useful only inside a plausible set.
The amateur is a negative reference, not a source of truth. Its recurring failures can reveal generic or repetitive continuations, but naive subtraction can also punish obvious correct tokens.
Contrast exposes scale-sensitive preferences.
When the expert improves more than the amateur on a token, the log-probability difference grows.
expert: −0.7minus amateur: −2.1 → +1.4
Two models mean more decoding work.
The amateur is smaller, but both distributions must be available at each token step.
Plausibility gate
Filter candidates relative to the expert's best probability before maximizing contrast.
Amateur choice
Too weak, too similar, or mismatched models can produce an unhelpful negative signal.
Obvious-answer failure
A correct token already liked by the amateur may be over-penalized as “unsurprising.”
Evaluate the decoder, not one charming sample.
Freeze the models and prompts. Sweep the plausibility threshold. Measure coherence and repetition. Include obvious-answer cases. Report latency and memory. Compare against expert-only and sampling baselines.
Inspect token decisions
Log expert probability, amateur probability, eligibility, contrast score, and the selected token at each step.
Test the counterexample
Include contexts where the amateur correctly assigns high mass to the obvious answer. Contrast can suppress exactly that token.
Account for serving cost
Measure wall-clock latency and memory with the actual expert/amateur pair and batching strategy.
Primary sources behind the lab.
The probabilities here are illustrative. These peer-reviewed papers define expert–amateur contrastive decoding, contrastive search, and a key limitation of naive probability extrapolation.
EXPERT–AMATEUR DECODING
Contrastive Decoding: Open-ended Text Generation as Optimization
Introduces the contrastive objective and expert plausibility constraint.
GENERATION DEGENERATIONA Contrastive Framework for Neural Text Generation
Studies contrastive search for coherence and diversity.
LIMITS AND EXTRAPOLATIONExplaining and Improving Contrastive Decoding
Frames CD as probability extrapolation and identifies obvious-answer failures.
MULTI-LAYER CONTRASTLower Layer Matters
Explores richer contrast signals for hallucination mitigation.
Export a decoder review.
Use the checklist to make a decoding comparison reproducible. Then explore how Superpowers can build inspectable tools and cache repeatable computer-use evaluations.
0 of 4 ready