Iterative pruning searches gradually.
Repeatedly train, remove a fraction of low-magnitude weights, rewind the survivors, and retrain. One-shot pruning often destroys useful structure at high sparsity.
Prune the network.
Keep the winning ticket.
A sparse mask is not enough. Test whether a subnetwork can retrain from the right initialization and match the dense model within the same optimization budget.
Accuracy across sparsity
Dense baseline, selected mask, and random-mask controlROUND 0
Round 0
Active weights100%
Ticket accuracy94.0%
Random control94.0%
Compute spent1.0×
The mask and initialization travel together.
The lottery ticket hypothesis proposes that dense, randomly initialized networks contain sparse subnetworks that can train in isolation when reset to suitable early weights. Pruning after training merely finds a compact model; rewinding tests the stronger trainability claim.
Random masks are the necessary control.
Matching parameter count does not prove a winning ticket. The selected connectivity must outperform arbitrary sparse subnetworks.
Rewind choices matter.
Large networks often need a short warmup before reset rather than the exact initial weights.
Budget parity matters.
A sparse model receiving much more optimization is not a fair efficiency comparison.
Sparsity is structured.
Unstructured weight removal may not translate to hardware speedups without compatible kernels.
Test trainability, not just compression.
A credible winning ticket matches the dense model under a controlled training recipe, beats random masks at equal sparsity, survives repeated seeds, and reports both parameter savings and realized compute.
Train the isolated subnetwork from its recorded rewind checkpoint.
Keep epochs, schedules, augmentation, and evaluation consistent.
Compare random masks, random reinitialization, one-shot pruning, and dense training.
Primary sources behind the lab.
The curves here are explanatory rather than measured. These papers establish winning tickets, stabilization by rewinding, scaling behavior, and critical analyses.
FOUNDATIONAL
The Lottery Ticket Hypothesis
Introduces trainable sparse subnetworks at initialization.
REWINDINGStabilizing the Lottery Ticket Hypothesis
Shows early weight rewinding helps deeper networks.
SCALINGDeconstructing Lottery Tickets
Studies pruning, initialization, and optimization controls.
GENERALIZATIONLinear Mode Connectivity and the Lottery Ticket Hypothesis
Connects stability and trainable sparse subnetworks.
Export the ticket audit.
Capture the mask strategy, sparsity path, rewind point, control gap, and total compute.