# `Sting` [🔗](https://github.com/dkuku/sting/blob/v0.1.0/lib/sting.ex#L1) SentenceTransformers pipelines in Elixir on top of Bumblebee, Axon, and Nx. Sting supports a subset of Python's `sentence-transformers` inference modules by parsing `modules.json` and downloading component files on demand (e.g. `1_Pooling/config.json`, `2_Dense/model.safetensors`). ## Quick start {:ok, model_info} = Sting.load_model({:hf, "sentence-transformers/all-MiniLM-L6-v2"}) {:ok, tokenizer} = Sting.load_tokenizer({:hf, "sentence-transformers/all-MiniLM-L6-v2"}) serving = Sting.text_embedding(model_info, tokenizer) Nx.Serving.run(serving, "Hello world") #=> %{embedding: #Nx.Tensor} `text_embedding/3` uses a saved default prompt and saved maximum sequence length when the caller does not override them. # `repository` ```elixir @type repository() :: Bumblebee.repository() ``` # `community_detection` Finds non-overlapping cosine-similarity communities in embeddings. # `cos_sim` Computes the cosine similarity matrix between two tensors using `Sting.Similarity.cos_sim/2`. # `cross_encoding` Builds a cross encoding serving using `Bumblebee.Text.cross_encoding/3`. # `dot_score` Computes the dot product similarity matrix between two tensors using `Sting.Similarity.dot_score/2`. # `encode_corpus` Encodes corpus/document text using the first available saved prompt named `"document"`, `"passage"`, or `"corpus"`. If none of those aliases are configured, it falls back to the saved default prompt (if any). Explicit `:prompt` or `:prompt_name` options take precedence. Returns the same `%{embedding: tensor}` result shape as `text_embedding/3`. # `encode_queries` Encodes query text using the saved `"query"` prompt when available. This is a convenience wrapper around `text_embedding/3`. If no `"query"` prompt is configured, it falls back to the saved default prompt (if any). Explicit `:prompt` or `:prompt_name` options take precedence. Returns the same `%{embedding: tensor}` result shape as `text_embedding/3`. # `euclidean_sim` Computes Euclidean similarity using `Sting.Similarity.euclidean_sim/2`. # `hamming_distance` Computes Hamming distance between two binary/ubinary tensors using `Sting.Quantization.hamming_distance/2`. # `infer_fallback_pooling_mode` Infers fallback pooling mode when `modules.json` is missing. Returns `"last"` for causal-LM architectures, and `"mean"` otherwise. # `load_config` Loads root prompt metadata and the first module's saved token limit independently. # `load_embedding_head` Loads only the SentenceTransformers embedding head and attaches it to an existing `model_info`. Supports piping directly from `Bumblebee.load_model/2`: {:ok, model_info} = Bumblebee.load_model(repo, architecture: :base) |> Sting.load_embedding_head(repo) # `load_head` Alias for `load_embedding_head/3`. # `load_model` ```elixir @spec load_model(repository(), keyword()) :: {:ok, map()} | {:error, String.t()} ``` Loads a SentenceTransformers model from a repository. If `modules.json` is present in the repository, Sting parses the module pipeline, loads the underlying base model, and stacks the declared modules (pooling, dense projections, normalization, etc.) on top. If `modules.json` is not present, it applies fallback pooling: `:last` for causal language models and `:mean` for other model families. ## Options * `:fuse_dense` - whether to pre-multiply consecutive linear projection layers without bias into a single projection layer. Defaults to `false` * `:type` - the numerical precision or Axon mixed precision policy * `:backend` - the Nx backend to allocate parameters on * `:safetensors_reader` - function used to read `.safetensors` files # `load_tokenizer` Loads the tokenizer for the first module in a SentenceTransformers pipeline, or the repository root when `modules.json` is absent. # `manhattan_sim` Computes Manhattan similarity using `Sting.Similarity.manhattan_sim/2`. # `mine_hard_negatives` Selects hard negatives for query/positive pairs from corpus embeddings. # `pairwise_angle_sim` Computes pairwise angle similarity using `Sting.Similarity.pairwise_angle_sim/2`. # `pairwise_cos_sim` Computes pairwise cosine similarity using `Sting.Similarity.pairwise_cos_sim/2`. # `pairwise_dot_score` Computes pairwise dot product using `Sting.Similarity.pairwise_dot_score/2`. # `pairwise_euclidean_sim` Computes pairwise Euclidean similarity using `Sting.Similarity.pairwise_euclidean_sim/2`. # `pairwise_manhattan_sim` Computes pairwise Manhattan similarity using `Sting.Similarity.pairwise_manhattan_sim/2`. # `paraphrase_mining` Finds the highest-scoring unique pairs of already-computed embeddings. # `quantize` Quantizes embeddings to `:binary`, `:ubinary`, or `:int8` using `Sting.Quantization.quantize/3`. # `semantic_search` Performs semantic search between query embeddings and corpus embeddings using `Sting.Search.semantic_search/3`. # `text_embedding` Builds a text embedding serving that returns `%{embedding: tensor}` for one input and a list of those maps for a list of inputs. Set `return_tokenized: true` to also return each input's tokenized `%{"input_ids" => tensor, "attention_mask" => tensor}`. This mirrors `Bumblebee.Text.text_embedding/3` and additionally supports SentenceTransformers models configured with `include_prompt: false`. In those cases the prompt prefix is tokenized separately and a `prompt_length` input is passed to the model so that prompt tokens are excluded from pooling. ## Options * `:prompt` - a custom prompt string to prepend to every input. It takes precedence over the saved default prompt. Pass `false` or `nil` to disable prompt use. * `:max_seq_length` - overrides the saved token limit. Compilation bucket sizes control padding; they do not increase this limit. Smaller buckets cap it at the largest available bucket. * `:prompt_name` - a saved prompt name from `config_sentence_transformers.json`. It cannot be combined with `:prompt`; pass `false` to disable prompt use. With neither option, the saved default prompt is used. * `:output_attribute` - model output key to embed. Defaults to `:pooled_state`. * `:output_pool` - optional `:cls_token_pooling`, `:mean_pooling`, or `:last_token_pooling` for rank-3 outputs. Defaults to `nil`. * `:embedding_processor` - optional `:l2_norm`. Defaults to `nil`. * `:return_tokenized` - includes tokenized inputs in each result map. Defaults to `false`. * `:compile`, `:defn_options`, `:preallocate_params` - serving execution options. A compile sequence length sets a padding bucket and caps the effective saved or requested maximum sequence length. # `truncate_embeddings` Truncates embeddings along the last dimension to `truncate_dim` (Matryoshka representation). ## Options * `:normalize` - whether to re-apply L2 normalization after truncation. Defaults to `false`. --- *Consult [api-reference.md](api-reference.md) for complete listing*