Build datasets, train three classifier families, and inspect the boundary they learn. The goal is not merely high training accuracy. It is a decision rule that still works on examples the model never saw.
Blue and red spheres are labeled examples. The tiled field is the classifier's prediction surface: color is the predicted class, while tile height shows confidence.
Training data pairs inputs with labels. A classifier searches for parameters or a comparison rule that maps inputs to labels. The data does not uniquely determine that rule: infinitely many boundaries can memorize a finite sample. The model family and regularization decide which solutions are preferred.
Tall tiles indicate stronger confidence according to the current model. A tall wrong tile is not a contradiction. It is evidence that confidence reflects the model's internal score, not guaranteed truth.
The boundary is too simple for the pattern. Training and test accuracy are both poor. More data alone may not repair a model family that cannot represent the geometry.
The model captures stable structure. Training and test scores are both strong, and their difference remains modest across new samples.
The model follows sample-specific noise. Training accuracy rises while test accuracy stalls or falls, producing a widening generalization gap.
Learns one weighted plane. Its strong bias makes it data-efficient and interpretable, but it cannot represent XOR or concentric rings without transformed features.
One global boundaryStores examples and predicts from local votes. Small k can follow intricate shapes and isolated noise; larger k smooths the field but can erase minority structure.
Local memoryMeasures distance from class centers through bell-shaped kernels. Kernel width controls whether influence is narrow and fragmented or broad and smooth.
Curved similarityEvaluation is a design discipline. A held-out score is meaningful only when the split resembles the deployment population and no test information leaked into training choices.
Training examples update model parameters. Validation examples choose hyperparameters such as regularization and neighborhood size. Test examples estimate final performance after choices are frozen.
Switch among blobs, XOR, and rings. A method that excels on one geometry may fail on another because its representation encodes a different assumption about separability and locality.
A large train-test gap suggests variance, leakage, or distribution mismatch. A small gap can still hide universal failure when both scores are low. Always interpret the gap beside the absolute scores and class-specific errors.
No. Complexity is relative to data volume, noise, and the true pattern. A flexible model can generalize when enough representative data and appropriate regularization constrain it.
Repeatedly checking a test set influences decisions and gradually turns it into training information. A final untouched set provides a less biased estimate of performance on future samples.
The tile height visualizes raw model confidence, not a calibrated probability. Calibration requires comparing predicted probabilities with observed frequencies on held-out data.
Use precision, recall, false-positive costs, subgroup performance, calibration, robustness under shift, and uncertainty. The relevant metric depends on the consequence of each error.
Explore SuperPowers workflows that turn explicit goals into inspectable agent actions, from research and computer use to complete website generation.
Explore SuperPowersBuild websites with an agentComputer-use cache