Do not optimize one point.
Optimize its neighborhood.

Compare SGD with sharpness-aware minimization. Perturb weights toward the local worst case, then update against that elevated loss to favor solutions whose surroundings stay stable.

Local loss landscape

Center solution, adversarial perturbation, and update target
SAM STEP
Center loss0.12
Worst neighbor0.22
Sharpness0.10
Robustness score82%

Flatness is a neighborhood question.

Two parameter vectors can have similar training loss while reacting very differently to small perturbations. SAM approximates a min–max objective: choose weights that minimize the maximum loss inside a radius around them.

First climb, then descend.

Compute the gradient, perturb parameters in the locally worst direction, evaluate the gradient there, and use that second gradient for the update.

Radius sets the neighborhood.

A tiny radius resembles ordinary training; an excessive radius can optimize the wrong scale.

Cost roughly doubles.

The basic method needs two forward-backward evaluations per update.

Sharpness is parameterization-sensitive.

Raw curvature should not be treated as an architecture-independent causal explanation.

Generalization remains empirical.

Always measure held-out performance, calibration, corruption robustness, and compute.

Audit the neighborhood and the budget.

Match computeCompare equal updates and equal gradient evaluations separately.
Sweep radiusReport the generalization and optimization sensitivity to ρ.
Measure more than sharpnessInclude test error, calibration, robustness, and convergence speed.

Primary sources behind the lab.

These papers introduce SAM, adaptive variants, efficient approximations, and critiques of simplistic flatness narratives. Curves here are explanatory.

FOUNDATIONAL

Sharpness-Aware Minimization for Efficiently Improving Generalization

Introduces the neighborhood min–max objective.

ADAPTIVE

ASAM: Adaptive Sharpness-Aware Minimization

Addresses scale sensitivity in sharpness measures.

EFFICIENT

Fisher SAM

Studies geometry-aware perturbations and efficiency.

CAUTION

Sharp Minima Can Generalize For Deep Nets

Shows naive flatness measures depend on parameterization.

Export the optimizer audit.

Explore Super · Build agent websites · Computer-use cache