Small batches are noise-limited.
Adding examples reduces gradient variance and makes each optimizer step more reliable.
Batch efficiency curve
Update efficiency0%
Critical ratio0x
Step time0 ms
Suggested LR multiplier0x
One curve connects statistics and systems.
Critical batch marks the bend.
Near the gradient noise scale, statistical efficiency begins to saturate.
Large batches become systems bets.
More workers can shorten wall time only if communication and stragglers stay controlled.