Sting

SentenceTransformers inference in Elixir, built on Bumblebee, Axon, and Nx.

Load supported SentenceTransformers checkpoints, encode text, and search or mine the resulting embeddings. Python comparison tests target SentenceTransformers 6.1.0 for the supported inference paths. Sting implements a subset of that library; see the parity guide for coverage and differences.

Installation

Requires Elixir 1.17 or later on a compatible Erlang/OTP runtime. The package checks cover Elixir 1.17 / OTP 26 and Elixir 1.19 / OTP 28. The initial release uses Bumblebee 0.8, Axon 0.9, Nx 1.0, and Tokenizers 0.5. Python parity validation uses Elixir 1.19 / OTP 28 with Nx.BinaryBackend.

Add Sting to your application's dependencies:

defp deps do
  [
    {:sting, "~> 0.1.0"}
  ]
end

Python is needed only for comparison tests, not for inference. EXLA and Torchx are development dependencies in this repository; applications that need an accelerated backend must add and configure it themselves. Accelerator performance and numerical parity are not established by the BinaryBackend tests.

Encode text

repo = {:hf, "sentence-transformers/all-MiniLM-L6-v2"}

{:ok, model} = Sting.load_model(repo)
{:ok, tokenizer} = Sting.load_tokenizer(repo)
serving = Sting.text_embedding(model, tokenizer)

%{embedding: embedding} = Nx.Serving.run(serving, "Hello world")
Nx.shape(embedding)
#=> {384}

Use Sting's tokenizer and serving helpers to preserve saved prompts, the first module's tokenizer location, and sequence limits. A list of texts returns a list of %{embedding: tensor} maps. Local checkpoints use {:local, "/absolute/path/to/checkpoint"} instead of a Hugging Face repository.

Query and document search

The following continues with the loaded model and tokenizer:

queries =
  model
  |> Sting.encode_queries(tokenizer, ["What runs on the BEAM?"])
  |> Enum.map(& &1.embedding)
  |> Nx.stack()

documents =
  model
  |> Sting.encode_corpus(tokenizer, ["Elixir runs on the BEAM.", "Rust compiles to native code."])
  |> Enum.map(& &1.embedding)
  |> Nx.stack()

hits = Sting.semantic_search(queries, documents, top_k: 2)
# One result list per query; each result contains :corpus_id and :score.

Query encoding selects a saved "query" prompt when present. Corpus encoding tries "document", "passage", then "corpus". Both otherwise use the saved default. Explicit :prompt or :prompt_name options take precedence.

serving = Sting.text_embedding(model, tokenizer, prompt: "Represent this query: ")
serving = Sting.text_embedding(model, tokenizer, prompt: false)

For a checkpoint that defines a saved "query" prompt, select it with Sting.text_embedding(model, tokenizer, prompt_name: "query"). Named prompts must exist in the checkpoint. Pooling with include_prompt: false excludes prompt tokens.

Sequence limits and compilation

Saved max_seq_length limits are applied before padding. Override the limit explicitly with :max_seq_length. Compilation buckets select the smallest sufficient padded shape; the largest bucket also caps the token limit.

serving =
  Sting.text_embedding(model, tokenizer,
    max_seq_length: 64,
    compile: [batch_size: 2, sequence_length: [16, 32, 64]]
  )

Nx.Serving.run(serving, ["Hello world", "A second sentence"])

Supported features

Architecture support comes from Bumblebee. Supported modules do not imply that every checkpoint containing them can be loaded. Training, fine-tuning, multimodal models, ONNX/OpenVINO, and arbitrary custom Python modules are outside the initial release.

Sparse and multi-vector encoders

For a supported sparse checkpoint, use Sting.Sparse.load_model/2, Sting.Sparse.load_tokenizer/2, and Sting.Sparse.encode/4. Sparse output is a dense Nx vocabulary vector containing zeros. Sting.Sparse.semantic_search/3 uses dot-product similarity.

For a supported multi-vector checkpoint, use Sting.MultiVector.load_model/2 and Sting.load_tokenizer/2, then Sting.MultiVector.encode_query/4 or Sting.MultiVector.encode_document/4. These return a token matrix per text. Sting.MultiVector.maxsim/2 and Sting.MultiVector.mean_maxsim/2 score those representations.

Hierarchical token pooling runs on the host and must be the final module. Multiple MultiVectorMask modules are rejected. See the parity guide for tested behavior and output shapes.

Mining and quantization

Mining consumes embeddings, rather than Python Dataset or model objects:

embeddings = Nx.tensor([[1.0, 0.0], [0.9, 0.1], [0.0, 1.0]])
pairs = Sting.paraphrase_mining(embeddings, top_k: 2, max_pairs: 3)
communities = Sting.community_detection(embeddings, threshold: 0.9, min_community_size: 2)
packed = Sting.quantize(embeddings, :ubinary)

Hard-negative mining requires one positive embedding per query and returns indices and scores. Search and mining expose chunk sizes; reduce them for large embeddings or memory-constrained hosts. Equal-score cutoff membership can differ from Python or between Nx backends.

Development and parity tests

mix deps.get
mix format --check-formatted
mix test --exclude python --exclude slow --exclude network
mix docs --warnings-as-errors

For Python parity:

python3 -m venv .venv
.venv/bin/pip install sentence-transformers==6.1.0 torch numpy datasets
STING_PYTHON="$PWD/.venv/bin/python" mix test --exclude network

The bridge also supports uv and checks the SentenceTransformers version before running comparisons. Network-tagged tests additionally fetch public Hugging Face checkpoints; run the full suite with STING_PYTHON="$PWD/.venv/bin/python" mix test. See the parity guide for the evidence behind each compatibility claim.

A local Bumblebee checkout is explicitly opt-in through BUMBLEBEE_PATH. Release builds use the published Hex dependency; unset this variable before building a package.

Release notes and license

See the changelog and release procedure.

Licensed under Apache 2.0. The license text is included in the package. Model weights retain their own licenses.