More capacity can hurt.
Then help again.

Move through the interpolation threshold and watch test risk rise, peak, and descend. Change label noise, regularization, sample count, and whether capacity or training time is on the horizontal axis.

Generalization risk curve

Classical U-shape plus the overparameterized regime
PEAK
Train error1%
Test risk38%
Peak location1.2×
Selected capacity1.4×

Interpolation changes the bias–variance story.

Classical intuition predicts a U-shaped risk curve. Modern overparameterized models can exhibit another descent after fitting the training set exactly, because optimization and architecture bias select structured interpolating solutions rather than arbitrary ones.

The peak is a fragile regime.

Near interpolation, the model has enough freedom to fit noise but too little redundancy to distribute that fit benignly.

Noise raises the peak.

Clean labels reduce the cost of interpolation; corrupted labels make the transition substantially sharper.

Regularization smooths it.

Explicit penalties or early stopping can suppress the peak and shift the effective threshold.

More data moves the boundary.

Capacity must be interpreted relative to constraints, not as a raw parameter count.

Epoch-wise descent also exists.

For a fixed model, longer training can traverse underfit, interpolation, peak risk, and later benign fitting.

Measure the whole curve, not one checkpoint.

Sweep through interpolationInclude models below, at, and well beyond zero training error.
Separate capacity from computeLarge models may receive different optimization dynamics and effective regularization.
Repeat noise and seedsThe peak is sensitive to label corruption, data geometry, and initialization.

Primary sources behind the lab.

These papers document model-wise and epoch-wise double descent, benign interpolation, and high-dimensional risk. The displayed curves are explanatory.

MODEL-WISE

Reconciling Modern Machine-Learning Practice and the Classical Bias–Variance Trade-Off

Formalizes the double-descent risk curve.

DEEP LEARNING

Deep Double Descent

Shows model-wise, sample-wise, and epoch-wise forms.

BENIGN FITTING

Benign Overfitting in Linear Regression

Studies when interpolation can still generalize.

HIGH DIMENSIONS

Surprises in High-Dimensional Ridgeless Least Squares Interpolation

Analyzes interpolation risk in high dimensions.

Export a risk sweep.

Explore Super · Build agent websites · Computer-use cache