Influence is a local, first-order estimate.
It avoids retraining once per example, but its accuracy depends on smoothness, curvature estimates, optimization state, and the size of the data perturbation.
Which training example
moved this prediction?
Estimate how upweighting or removing each training point changes a test prediction, then compare the influence approximation with expensive leave-one-out retraining.
Estimated vs measured influence
Positive supports the prediction; negative opposes itPOINT A
Rank correlation0.91
Selected error0.02
Prediction confidence78%
Retraining saved7×
Differentiate through training.
Classical influence functions approximate how model parameters and test loss respond when one training point receives infinitesimally more weight. The inverse Hessian translates a training gradient into a parameter change.
Damping stabilizes inversion.
Adding λI controls poorly conditioned curvature but changes the attribution.
Removal is not infinitesimal.
Deleting groups or highly influential points can violate the approximation.
Deep networks complicate theory.
Non-convexity, flat directions, and incomplete convergence weaken classical guarantees.
Validate top-ranked points.
Use actual retraining or data ablations before making high-stakes conclusions.
Treat attribution as a hypothesis.
Primary sources behind the lab.
These papers establish modern influence functions, scalable approximations, representer-point attribution, and evidence about fragility in deep learning.
FOUNDATIONAL
Understanding Black-box Predictions via Influence Functions
Brings classical influence analysis to modern ML.
SCALINGScaling Up Influence Functions
Develops practical influence estimates for large models.
REPRESENTER POINTSRepresenter Point Selection for Explaining Deep Neural Networks
Attributes predictions to weighted training examples.
FRAGILITYInfluence Functions in Deep Learning Are Fragile
Examines sensitivity to damping and non-convexity.