One high-dimensional problem becomes several smaller clustering problems.
An eight-dimensional vector with four subspaces becomes four independent two-dimensional slices. Each slice selects one centroid from its own codebook.
Interactive embedding compression laboratory
Shrink vectors. Keep the neighbors.
Product Quantization splits a vector into subspaces, learns a small codebook in each one, and stores centroid IDs instead of every float. The result is compact, fast, and approximate.
ADC(q, code) = sum over subspaces of distance(query slice, selected centroid)
Subspaces
Neighbor distance
The database begins as eight-dimensional float vectors. Step forward to split, train, encode, and search.
Compression buys capacity with approximation.
Every result below is recomputed from the visible teaching vectors. The vector-only compression estimate excludes codebooks, IDs, and index metadata.
Vector compression0x32 B to 0 B
Reconstruction MSE0.000Lower preserves vectors better
Recall at 40%ADC overlap with exact top four
ADC table0Query-centroid distances
Exact L2 neighbors
Product-quantized ADC neighbors
A code is a tuple of local choices.
PQ is not scalar rounding. Each sub-vector is replaced by the ID of its nearest learned centroid, and those IDs jointly approximate the full vector.
The query can stay exact.
Asymmetric Distance Computation compares each exact query slice with every centroid once, then scores database codes by table lookup. It avoids quantizing the query.
More subspaces spend more code bits.
Increasing the number of subspaces usually reduces reconstruction error because each centroid models fewer dimensions, but every extra centroid ID lengthens the code.
Larger codebooks improve fit.
More centroids can reduce distortion, provided the training set is large and representative enough to learn them reliably.
Recall, not MSE alone, decides search quality.
A small reconstruction error can still swap close neighbors. Tune the codec against retrieval metrics on held-out queries.
The search path is a stack of approximations.
Scroll through the exact state transitions. The browser lab uses deterministic k-means so every slider change can be replayed and inspected.
Partition each vector into equal subspaces.
The embedding dimension must be divisible by the number of subquantizers. Dimension order matters because correlated features can make one split much harder to quantize than another.
Learn one centroid table per subspace.
K-means minimizes squared reconstruction error within each slice. Production codebooks need representative training vectors; a tiny or shifted sample creates brittle assignments.
Replace every slice with a centroid ID.
The concatenated IDs are the PQ code. Reconstructing a vector means concatenating the selected centroids, but ADC can search without materializing that reconstruction.
Precompute query-to-centroid distances and sum lookups.
For each database code, read one value from each subspace table and add them. The ranking is fast and compact, but only approximate, so measure neighbor recall.
Four decisions shape the operating point.
Expand each slice to see what a production experiment should hold constant while testing the compression-recall frontier.
Compression ratio is not a retrieval metric.
A smaller code is only useful when the nearest neighbors needed by the downstream task remain discoverable at an acceptable latency.
The codebook is part of the model.
Changing the embedding distribution without retraining or validating the quantizer can move vectors across centroid boundaries and degrade recall.
ADC keeps one side precise.
Leaving the query uncompressed removes one source of error and lets a single lookup table score many compact database codes.
A practical PQ evaluation checklist.
Benchmark the same vectors and queries with exact search, then vary one codec choice at a time.
- Report vector bytes, codebook overhead, index metadata, latency, and recall together.
- Train codebooks on a representative split and evaluate on held-out queries.
- Normalize vectors before L2 search when cosine similarity is the intended metric.
- Revalidate after changing the embedding model, feature order, or corpus mix.
Primary research and implementation references.
The original PQ formulation, optimized space decomposition, fast ADC work, and Faiss index documentation ground the mechanics shown in the lab.
Product Quantization for Nearest Neighbor SearchJegou, Douze, and Schmid, IEEE TPAMI 2011
Optimized Product Quantization for Approximate Nearest Neighbor SearchGe, He, Ke, and Sun, CVPR 2013
Quicker ADC: Unlocking the Hidden Potential of Product Quantization with SIMDAndre et al., IEEE TPAMI 2020
Faiss Indexes and Product Quantization CodesOfficial Faiss implementation documentation