Embedding quantization utilities mirroring sentence_transformers.util.quantize_embeddings.
Supports binary, unsigned binary bit-packing (:ubinary), and scalar :int8 quantization.
Summary
Functions
Computes the Hamming distance between two ubinary or binary packed tensors.
Quantizes embeddings to a lower precision.
Functions
Computes the Hamming distance between two ubinary or binary packed tensors.
Quantizes embeddings to a lower precision.
Precisions
:binary- values > 0 are packed 8-to-a-byte into signeds8(-128 offset, matchingsentence_transformers.util.quantize_embeddings).:ubinary- values > 0 are packed 8-to-a-byte intou8(reduces memory by 32x).:int8- linear scalar quantization mapped to[-128, 127].
Options (for :int8)
:ranges- optional{min_tensor, max_tensor}tuple specifying dimension ranges.:calibration_embeddings- optional tensor to compute quantization ranges from.