Sting 0.1.0 targets SentenceTransformers 6.1.0 inference behavior for a supported subset of modules and utilities. It is an Elixir API over Bumblebee, Axon, and Nx, not a replacement for the full Python package. Python is a test dependency only.
What parity means
Generated-checkpoint tests save tiny Python models, load those same weights in Sting, feed the same inputs to both implementations, and compare outputs within explicit floating-point tolerances. Other tests compare utility outputs directly. The suite pins the SentenceTransformers version and reports Python, Torch, Elixir, OTP, Nx backend, and Elixir dependency versions.
Passing these cases establishes agreement for the tested paths. It does not establish compatibility with every checkpoint, architecture, input shape, dtype, backend, or future SentenceTransformers version. Release validation uses Nx.BinaryBackend; accelerated backends require their own validation.
Coverage
| Feature | Evidence | Scope |
|---|---|---|
| Dense Transformer pipelines | Python-generated checkpoints | Pooling, Dense, Normalize, LayerNorm, inference-only Dropout |
| Pooling | Python comparisons | CLS, max, mean, mean-sqrt-length, weighted mean, last-token; padding and fully masked cases |
| Dense activations | Python comparisons | Identity, Tanh, ReLU, GELU, SiLU, Sigmoid, Mish, LeakyReLU |
| Prompts and sequence limits | Python-generated checkpoints and local regressions | Saved/default/named/custom prompts, prompt exclusion, truncation, compiled padding |
| Dense layer fusion | Python comparisons | Consecutive bias-free linear layers; refusal of biased or activated layers |
| Sparse encoders | Python comparisons | MLMTransformer + SpladePooling and SparseStaticEmbedding; dot similarity and search |
| Multi-vector encoders | Generated Python MultiVectorEncoder | Token projection, Normalize, one mask, final hierarchical pooling, query/document prompts and limits |
| Late-interaction scoring | Python comparisons and local regressions | MaxSim/MeanMaxSim, variable lengths, masked padding, fully masked documents |
| Similarity | Python utility comparisons | Cosine, dot, negative Euclidean/Manhattan distance, pairwise variants, pairwise angle |
| Quantization | Python comparisons | Binary/unsigned binary and int8 for tested shapes and calibration; Hamming results checked against Python bit operations |
| Search and mining | Python utility comparisons | Chunked top-k, paraphrase mining, communities, embedding-level hard negatives with tested margins/ranges |
| StaticEmbedding / WeightedLayerPooling | Local fixture tests | Implemented, without Python parity coverage |
| Matryoshka truncation | Local behavior tests | Slicing and optional renormalization, without Python parity coverage |
| Causal-model pooling fallback | Local configuration tests | Last-token selection for recognized causal architectures; not end-to-end parity |
| Cross encoding | Bumblebee delegation | No SentenceTransformers comparison coverage |
Evidence lives in
dense and utility comparisons,
sparse comparisons,
multi-vector comparisons,
and mining comparisons.
The source checkout also contains local regressions in test/sting_test.exs,
test/sting/dense_edge_test.exs, and test/sting/utilities_test.exs.
API and behavior differences
Encoding
- Dense and sparse servings return
%{embedding: tensor}for one string and a list of those maps for a list of strings. Stack their:embeddingvalues withNx.stack/1before passing them to matrix-based retrieval utilities. - Multi-vector encoding returns a
{tokens, dimensions}tensor for one string or a list of token tensors for multiple strings. MaxSim returns a{query_count, document_count}score matrix. - Sparse encodings occupy dense Nx vocabulary tensors. Zeros do not imply sparse storage or sparse-kernel acceleration.
- Base-model support follows Bumblebee. Unsupported modules are errors; arbitrary Python modules, training, fine-tuning, multimodal inputs, and ONNX/OpenVINO exports are outside this release.
- HierarchicalTokenPooling must be the final module and uses host-side clustering. Multiple MultiVectorMask modules are rejected. Scoring-mask removal of every token during encoding raises an error; explicitly masked scoring tensors are supported and tested separately.
Prompts and padding
Use Sting.load_tokenizer/2 and Sting.text_embedding/3 rather than bypassing them
with Bumblebee helpers when saved prompts or limits matter. Query helpers prefer
the "query" prompt; corpus helpers try "document", "passage", then "corpus".
Both otherwise use the saved default. An explicit prompt option takes precedence;
prompt: false disables the saved prefix.
Saved maximum sequence length controls truncation. Compilation buckets only add
padding, except that their largest size also caps truncation. :max_seq_length
is an explicit serving override. Compilation and dynamic tokenization preserve
special-token processing, including EOS-only padding tokenizers.
Retrieval and quantization
- Mining operates on already-computed embeddings and returns IDs/scores. It does not implement Python Dataset output, text/model orchestration, FAISS search, cross-encoder rescoring, or multiprocess mining.
- Hard-negative mining expects one positive vector per query and excludes every vector-equal corpus entry by default. Text identity is not available at this boundary. Filtering applies to the ranked candidate window.
- BinaryBackend search ties prefer lower corpus IDs. Other Nx backends and Python/Torch can choose different members at an equal-score top-k cutoff.
- Chunk sizes bound score blocks, not total corpus storage or returned results. Community extraction and hierarchical clustering still perform host work.
- Binary quantization pads each last dimension to a multiple of eight. Python comparisons include a 12-dimensional input that requires padding. Int8 supports explicit per-dimension ranges or calibration embeddings; reusing calibration is the caller's responsibility.
Reproduce the comparisons
From the repository root:
mix deps.get
python3 -m venv .venv
.venv/bin/pip install sentence-transformers==6.1.0 torch numpy datasets
STING_PYTHON="$PWD/.venv/bin/python" mix test --exclude network
To include public Hugging Face checkpoints:
STING_PYTHON="$PWD/.venv/bin/python" mix test
An incompatible Python environment fails with setup instructions. uv is also
supported when STING_PYTHON is unset. The Python parity CI job installs the
pinned SentenceTransformers version and runs local checkpoint comparisons.
Network tests are a separate release check because they depend on external
model availability. Keep the printed version information with parity results;
Torch and other Python dependencies are reported rather than fully locked.