Why can opposite designs both work?
Some binary quantization systems randomly rotate vectors while others preserve the original axes, and retrieval quality varies widely across representations at the same bit width. The paper studies these observations through the statistics of contrastive embeddings.
Separating covariance from coordinate heterogeneity
Under a Gaussian model, the full covariance structure affects ranking fidelity; marginal variances alone miss accumulated cross-coordinate signal. Unequal coordinate variances determine how much a magnitude bit can add and whether random rotation helps or removes useful structure.
Rotation equalizes variance and can support isotropic distance correction, but it can also erase heterogeneity used by another code. The paper derives approximate fidelity expressions and empirical scaling relationships across models and dimensions.
Turning representation structure into a design signal
Experiments across nine embedding families and eighteen datasets examine covariance, magnitude bits, and rotation. The results offer statistical guidance for choosing an encoding and preprocessing strategy under the stated Gaussian and approximation conditions.
READ AND CITE
Continue with the original paper.
The complete derivations, experimental setup, and results are available in the public manuscript.