fix(deps): update dependency sentence-transformers to v6 - #374
Open
dreadnode-renovate-bot[bot] wants to merge 1 commit into
Open
fix(deps): update dependency sentence-transformers to v6#374dreadnode-renovate-bot[bot] wants to merge 1 commit into
dreadnode-renovate-bot[bot] wants to merge 1 commit into
Conversation
| datasource | package | from | to | | ---------- | --------------------- | ----- | ----- | | pypi | sentence-transformers | 5.1.2 | 6.0.0 |
Contributor
Author
|
dreadnode-renovate-bot
Bot
added
area/python
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR contains the following updates:
| Package | Change | Age | Confidence |
|
Generated Summary:
sentence-transformersfrom>=5.1.0,<6.0.0to>=6.0.0,<6.1.0.This summary was generated with ❤️ by rigging
| sentence-transformers |
| 
|
>=5.1.0,<6.0.0→>=6.0.0,<6.1.0|Release Notes
huggingface/sentence-transformers (sentence-transformers)
v6.0.0: - MultiVectorEncoder for ColBERT & late interaction models, transformers v5, float32 scoring, faster training & encodingCompare Source
This major release introduces Multi-Vector Embedding models, also known as late interaction or ColBERT-style models, as a fourth model type alongside
SentenceTransformer,CrossEncoder, andSparseEncoder. Going forward, you'll be able to use Sentence Transformers for training, inferencing, and interpreting Multi-Vector Embedding models.It also modernizes the dependency floors to
transformersv5, fixes a class of silent scoring bugs caused by half precision, and speeds up both training and encoding.Install this version with
MultiVectorEncoder: ColBERT-style late interaction models (#3794)
Sentence Transformers v6.0 introduces
MultiVectorEncoder, for ColBERT-style late interaction retrieval. Where a regular embedding model compresses a whole text into one vector, a multi-vector model keeps one vector per token and scores query against document with the MaxSim operator. That preserves token-level matching information that a single vector has to average away, which usually means stronger retrieval at the cost of a bigger index. It is also the state of the art for visual document retrieval, where a text query is matched against page images directly, with no OCR step in between.Any PyLate checkpoint and any Stanford-NLP ColBERT checkpoint loads straight into it, and colpali-engine models for visual document retrieval work too, through the same familiar API you already use for dense, sparse, and reranker models.
Mars wins, as it should, though notice how close the four scores are. That is normal for MaxSim: the scores often look similar, but the ranking is still exact. The blogpost explores this in more detail.
Note what you get back: a list of 2D tensors on the model device, one per input, each of shape
(num_tokens, embedding_dim). Unlike dense embeddings, you cannot stack these into one rectangular tensor, because every input has its own token count. Passconvert_to_numpy=Truefor a list of numpy arrays instead, which is what you want once a corpus outgrows device memory.Multi-vector models are also asymmetric: queries and documents go through different prefixes, different length caps, and different scoring masks. Unlike many dense models, where the two are interchangeable,
encode_queryandencode_documentare required to get correct embeddings.The MaxSim operator
Scoring uses MaxSim: for each query token, take its highest similarity against any document token, then sum those maxima across the query.
You can read the operator as a soft alignment: every query token points at the one document token that best explains it, and the score is how well the document explains the query overall. The alignment does not have to be lexical, since the token embeddings are contextualized. But when an exact match does matter to you (a product code, a surname, a function name), MaxSim has a token sitting right there to match it, where a single-vector model had to fold it into an average.
Because MaxSim sums over query tokens, its magnitude scales with the query token count, so scores are not comparable across models with different query recipes. If you want scores on a bounded scale, use
similarity_fn_name="meanmaxsim", which divides by the query token count and gives you an average cosine similarity in[-1, 1].Scoring builds a 4-dimensional intermediate of every query token against every document token, which is the largest tensor in the operation. Every scoring function takes a
chunk_elementsbudget that bounds it, defaulting to 100 million elements (roughly 400 MB in float32), so lower it if you run out of memory. Scores and gradients are bit-identical whatever you set it to.maxsimandmaxsim_pairwisealso take adevice, which scores one chunk at a time on that device and moves each result straight back, letting you score a corpus larger than your VRAM on the GPU. Both are reachable throughsimilarity, which forwards any extra keyword arguments to the scoring function:When training, pass the budget to the loss instead, with
similarity_fct=partial(colbert_scores, chunk_elements=1_000_000). It chunks the document axis, so it composes with the loss-levelscore_mini_batch_size, which chunks the query axis.Are they any good?
lightonai/LateOnandlightonai/DenseOnwere trained by LightOn on the same data with the same ModernBERT backbone and the same 149M parameters, differing only in whether they keep one vector per token or pool down to one per document. Running both over all 13 NanoBEIR datasets isolates what that choice buys:Late interaction wins on 9 of the 13 datasets and on the mean, by roughly one NDCG point. The four it loses (ArguAna, FiQA2018, SCIDOCS, and SciFact) are the shape of the tradeoff you should expect: a real gain in retrieval quality at the same model size, paid for in index footprint, rather than a universal win on every dataset. The same pair scores 57.22 against 56.20 on the full 15-dataset BEIR, a comparable gap, so the margin is not an artifact of the small benchmark.
That footprint is the real cost. One vector per token instead of one vector per document is a lot more vectors, only partly offset by the smaller dimension. Encoding 4,874 Natural Questions passages with
lightonai/LateOnproduced 608,414 token vectors, an average of 124.8 per passage:all-MiniLM-L6-v2gte-modernbert-baseLateOnThat is about 42x the storage of the MiniLM index. Token Pooling cuts the vector count before any of that, real late interaction indexes compress heavily (the same vectors take 88 MB as a fast-plaid PLAID index), and using a multi-vector model as a reranker over a dense first stage avoids building an index at all.
Every checkpoint format loads
Multi-vector checkpoints have been published in several formats over the years.
MultiVectorEncoderreads all of them, so loading looks the same whatever the model started life as:The recipe knobs that differ per checkpoint (marker prefixes for queries and documents, length caps, whether queries are padded out with
[MASK]tokens, and which tokens are skipped when scoring documents) all live in the module configs, soprint(model)shows you exactly what you loaded:Following the design principle of the rest of the library, this behavior lives in swappable modules rather than in the model class: a
Transformerproducing contextualized token embeddings, a token-levelDenseprojecting each of them down, aMultiVectorMaskdeciding which tokens count during scoring, and a token-levelNormalize.Supported models
These are the checkpoints we test against directly, ranked by retrieval quality. The
sentence-transformerstag on the Hub is the list that stays current, and for text retrieval in particular, any PyLate or Stanford-NLP ColBERT checkpoint loads whether or not it carries the tag yet. Where arevisionis listed, pass it until the pull request on that repository is merged.Text retrieval (29 models). NanoBEIR is the mean NDCG@10 over the 13 NanoBEIR datasets, a fast proxy for English text retrieval quality. A
-means the model was not evaluated on it, which is the case for the non-English models.trust_remote_code=Truetrust_remote_code=Truerevision="refs/pr/4"Visual document retrieval (22 models). These embed page images as documents and text as queries. NanoViDoRe is the equivalent proxy over the ViDoRe benchmark subsamples.
trust_remote_code=Truetrust_remote_code=Truetrust_remote_code=Truetrust_remote_code=TrueNote that NanoBEIR and NanoViDoRe are small benchmarks, so their scores are not a substitute for evaluating on your own data, which is always the right way to pick a model.
Visual, audio, and video document retrieval
Late interaction is the state of the art for visual document retrieval: matching a text query against page images, with charts, tables, and layout intact, and no OCR step. This is what the ColPali family of models does, and those checkpoints run through the same API. Image documents are passed as URLs, local paths, or PIL images:
The code is unchanged from the text case. Underneath, the processor handles the visual prompt and the image patches, and MaxSim scores query text tokens against document image patches. Page images are not the only non-text modality either: text, images, audio, and video are all accepted, and a checkpoint supports whichever of those its processor does, which
model.modalitiesreports.Because MaxSim is a sum of per-query-token maxima, a ranking decomposes exactly: every point of a document's score belongs to one query token and one document token. The new
sentence_transformers.multi_vector_encoder.interpretabilitymodule overlays that decomposition onto the page as the standard ColPali heatmap, either aggregated over the query or one map per query token.Token pooling
If the index footprint worries you, the most effective knob is to store fewer token vectors.
HierarchicalTokenPoolingimplements the token pooling technique from Clavié, Chaffin, and Adams: it clusters each document's token vectors with Ward linkage on cosine similarity and replaces each cluster with its mean, keeping roughly1 / pool_factorof the tokens.By default, pooling applies to documents only, since queries are short and are the side you cannot afford to distort. On the Natural Questions corpus above, the reduction tracks
pool_factorclosely:pool_factorThe original experiments measured the retrieval cost of this on BEIR and found very little of it: 100.6% of the unpooled performance on average at
pool_factor=2, and 99.0% atpool_factor=3. How much it costs on your data is corpus-specific, so measure it with an evaluator before you settle on a factor.Update Stats
Introducing
MultiVectorEncoderhas been one of the largest updates to Sentence Transformers, introducing all of the following:MultiVectorMask,BaseTokenPooling,HierarchicalTokenPooling, andLambdaTokenPoolingmaxsim,maxsim_pairwise,mean_maxsim,mean_maxsim_pairwise) plus 6 named ColBERT scorers and 5 XTR scoring entry pointstorch.compileResources
🚨 transformers v5, torch 2.2, and new dependency floors (#3794)
Sentence Transformers v6.0 requires
transformersv5. The v4.x compatibility branches have been removed, which is what allows the new modality handling, chat template support, and unpadding paths to be relied upon rather than feature-detected. The floors that moved:transformers>=4.41.0,<6.0.0>=5.0.0,<6.0.0huggingface-hub>=0.23.0>=1.3.0,<2.0.0torch>=1.11.0>=2.2numpy>=1.20.0>=1.24.0scikit-learn>=0.22.0>=1.1.0typing_extensions>=4.5.0>=4.10.0datasets(train)>=2.0.0>=2.16.0accelerate(train)>=0.20.3>=1.3.0optimum-intel[openvino]>=2.0.0requires-pythonis unchanged at>=3.10. Note that multi-GPU training with streaming (IterableDataset) datasets needsaccelerate>=1.13.0in practice.🚨 Higher-precision scoring (#3892, #3893, #3924, #3926)
Half precision ties too many scores together to rank with. Three separate places where that mattered are now computed in float32.
Reranker scores are the big one.
CrossEncoder.predict(andrank) now upcast the logits to float32 before applying the activation function. A sigmoid in bfloat16 saturates and collapses the top candidates onto a handful of tied values, which randomizes their order. Measured oncross-encoder/ettin-reranker-32m-v1in bfloat16 over three NanoBEIR datasets with 100 candidates per query:NanoMSMARCO NDCG@10 alone goes from 0.0965 to 0.7093. If you run a half precision reranker with the default sigmoid activation, its ranking was essentially randomized before this release. Models using
activation_fn=nn.Identity()(raw logits) were unaffected, as bf16 logits keep enough relative spacing.Similarity scores from
model.similarity/similarity_pairwiseand thecos_simfamily are now computed in float32 for float16 and bfloat16 embeddings. With 10,000 realistic cosine scores (mean 0.7, standard deviation 0.05), float32 keeps 9,983 distinct values where float16 keeps 593 and bfloat16 keeps just 93. bfloat16 can represent only 129 distinct values in the whole of[0.5, 1.0).MaxSim sums over query tokens, reaching magnitudes where the bfloat16 grid is 0.125 wide, so
maxsimandmaxsim_pairwiseaccumulate the per-token maxima in float32 and always return float32 scores. The 4-dimensional scoring intermediate stays in the input dtype, so this does not change peak memory.Note that
encode()output dtypes are unchanged. Only the scoring step is upcast. ForCrossEncoder.predict, the returned dtype changes only withconvert_to_tensor=Trueorconvert_to_numpy=False, as the default numpy output was already float32.Separately, the multi-vector bf16 benchmarks were re-measured under this float32 accumulation (#3924). Most of the previously reported bf16 quality drop came from the scoring accumulation rather than from the embeddings: plain bf16 now sits at 99.0% of fp32 retrieval quality (was 95.0%), and bf16 with FlashAttention-2 is indistinguishable from fp32 at 99.96% (was 97.9%).
🚨 Other breaking changes (#3794, #3927, #3935)
similarityandsimilarity_pairwiseare methods, not properties. Calls likemodel.similarity(embeddings1, embeddings2)work unchanged, but assigning a custom function tomodel.similarityis no longer supported: it now silently shadows the method where it previously raised anAttributeError. Setmodel.similarity_fn_name = "dot"instead, which updates both. Note also thatmodel.similarity.__name__is now"similarity"rather than the resolved function name, which affected lossget_config_dict()output and generated model cards. The newsentence_transformers.util.similarity_fct_name()resolves it properly and the losses use it.model.encode([{"role": "user", ...}, {"role": "assistant", ...}])produces one embedding, where v5.x read it as a batch of two inputs. Wrap each conversation in its own list to encode a batch:model.encode([[msg1], [msg2]]). This applies toSentenceTransformer,SparseEncoder, andMultiVectorEncoder.CrossEncoderis unaffected.trust_remote_code=True(#3935). Loading a model whosemodules.jsonreferences a class outsidesentence_transformersexecutes third-party code, and a local directory no longer implies trust. This closes the bypass reported in #3801 and completes the deprecation cycle announced in v5.6 and v5.7. Unmet, it raises aValueErrornaming the class and pointing at the repository or local path to inspect. Trainer checkpoint reloading (load_best_model_at_end,resume_from_checkpoint) keeps working for programmatically built models without the flag.quantize_embeddingsreturns a list of per-input matrices when given a list of 2D arrays, where it previously stacked them into one 3D array. Update callers that indexed the stacked array. An empty list now returns[]instead of raising, and a(0, dim)matrix returns a correctly shaped empty result.encode(pool=..., precision="int8")now quantizes once after merging the worker results, so the calibration ranges match single-process encoding. Quantized indexes built with v5.x multi-process encoding are not bit-compatible and should be regenerated. Peak memory is higher, because the full float32 matrix is materialized before quantization.CrossEncoder.rankreturns Python floats (#3927) as its"score"values, where it previously returnednumpy.float32scalars or 0-dimensional tensors. The results are directly JSON serializable, matchingsemantic_search.convert_to_numpyandconvert_to_tensoronrankare now deprecated no-ops: callpredictdirectly if you want an array or a tensor. Beyond the cleaner output, this avoids a device synchronization per comparison when sorting, which took 212ms for 1000 CUDA scalars against 0.089ms for Python floats.Normalizemoved tosentence_transformers.base.modules. Existing models load fine and silently, but a model saved by v6.0 with aNormalizemodule cannot be loaded by Sentence Transformers older than v6.0.SentenceTransformercheckpoint as aCrossEncoder(or any other such conversion) no longer picks up the source'sprompts,default_prompt_name,similarity_fn_name,truncate_dim, oractivation_fn, as those describe a model you are not loading. A reranker's default prompt being prepended to everyencodecall was the motivating case. Explicit keyword arguments still win. These conversions are now also logged at warning level, so they are visible at default verbosity.SimilarityFunction.possible_values()now includes"maxsim"and"meanmaxsim". Setting an unsupportedsimilarity_fn_nameonSentenceTransformerorSparseEncoderraises immediately rather than failing later, and a newSUPPORTED_SIMILARITY_FN_NAMESclass attribute documents what each model type accepts.Faster training and encoding (#3938, #3794)
Multi-column losses now run one forward pass over merged columns (#3938). A training batch arrives as one feature dict per column (anchor, positive, negative_1, and so on), and the classic pattern runs the model once per column. The
SentenceTransformerandSparseEncoderlosses now pad and concatenate the like-width candidate columns into a single batch, keeping the anchor on its own forward pass since a 12-token query padded into 256-token documents costs more than it saves:Loss trajectories match, up to dropout sampling. Losses fall back to per-column forward passes whenever the columns cannot be merged safely, for example with differing feature keys, disagreeing prompts or router tasks, or flattened Flash Attention inputs. The cached losses keep using GradCache, and
AdaptiveLayerLossopts out.Backend benchmarks were re-measured for all four model types, with new Flash Attention columns and rewritten recommendations. For
SentenceTransformer, float16 with Flash Attention and unpadding is now the fastest GPU configuration at 3.87x over float32, and ONNX on GPU is no longer recommended for short texts as float16 now beats it. ForCrossEncoder, Flash Attention is explicitly not recommended, as unpadding does not apply to classification heads. ForSparseEncoder, plain float16 remains the recommendation even though FA2 unpadding is now supported. See Speeding up Inference for the flowcharts.Models can declare their dependency versions (#3934)
Model authors can now record which package versions their checkpoint needs, and loading verifies them up front instead of failing in a confusing way later. Add a
requirementsmapping toconfig_sentence_transformers.json, using PEP 440 specifiers:{ "model_type": "SentenceTransformer", "requirements": { "transformers": ">=5.15", "peft": { "specifier": ">=0.18,<0.20", "reason": "Older versions ignore the key_mapping, which silently randomizes the adapter weights." } } }Loading that model in an environment that does not satisfy it raises an
ImportErrorlisting every unmet requirement at once, with the optionalreasonincluded and a ready-to-run install command:"python"and"pytorch"are understood as special names, prereleases are accepted so nightlies and.dev0builds do not trip the check, and anything unparsable warns and is skipped rather than blocking the load. It works for all four model types. See Declaring Version Requirements for details.Evaluator and loss correctness (#3794, #3944, #3937)
Pooling(include_prompt=False)no longer corrupts repeated forward passes (#3944). The pooling module used to write its prompt-excluded mask back intofeatures["attention_mask"], but that key is what the encoder attends over on the next forward pass, and it is where the prompt boundary is read from. Any loss that embeds the same feature dicts twice therefore got a different answer each time.AdaptiveLayerLossis the headline victim: on a model with a 3-token prompt, two consecutive calls with identical inputs returned 2.706679 and then 10.334954, where it is now stable at 1.777709.DenoisingAutoEncoderLosswas hit from another angle, handing its decoder an all-zero cross-attention mask. If you trainedAdaptiveLayerLosson aninclude_prompt=Falsemodel with prompts, your results will move. As a side effect,encode(output_value=None)now reports the full mask the encoder used, matching theinput_idsandtoken_embeddingsin the same dictionary.sentence_embeddingandoutput_value="token_embeddings"are bit-identical.TripletEvaluatorandSparseTripletEvaluatorembed anchors withencode_queryand positives and negatives withencode_document, instead ofencodefor all three. This is a no-op for models withoutquery/documentprompts, but asymmetric models will report different triplet accuracy than in v5.x, since their prompts, router routes, and per-task length caps are now applied. Both also now reject unknownsimilarity_fn_namesand unknownmarginkeys at construction, where a typo previously degraded silently to a missing metric or a zero margin.InformationRetrievalEvaluatorbreaks score ties by corpus id, making its metrics independent ofcorpus_chunk_size. Previouslytorch.topk(..., sorted=False)plus a heap comparison made tie retention depend on chunk boundaries and favor larger corpus ids. Metrics change only where exact ties exist, such as duplicate documents or quantized embeddings. Inherited by the sparse and NanoBEIR variants.DistillKLDivLossandSparseDistillKLDivLossgained per-side temperatures (student_temperature,teacher_temperature) following the DenseOn and LateOn recipes, plus validation that catches previously silent misuse: non-positive or non-finite temperatures, fewer than three columns (a softmax over one candidate is constant, so the loss and its gradient are identically zero), and teacher score shapes that do not match the candidate columns. A new one-time warning reports how many teacher scores underflowed to exactly zero and recommends ateacher_temperaturefloor.top_kmust be positive, integer embeddings are upcast rather than producing an integer score grid, and a query padding mask is inferred from all-zero rows when none is given, matching the document side. XTR also now computes itsZnormalizer as the paper's retrieval count rather than a positive-maxima proxy.MultipleNegativesRankingLossrejects a NaNscalerather than accepting it.Bug Fixes
NoDuplicatesBatchSamplersilently no-opping on media datasets in #3794: PIL images and torchcodec decoders stringify to a fresh object address on every access, so every row looked unique and no duplicates were ever detected. Large numpy arrays had the opposite problem, as their truncated string representation made distinct arrays collide. Values are now keyed by content. Batches change for datasets with image, audio, video, or array columns. Plain text and numeric datasets are byte-identical.xxhash4.0 compatibility in #3928:xxh64_intdigestno longer acceptsstr, which crashed training withBatchSamplers.NO_DUPLICATES_HASHED(orprecompute_hashes=True). Strings are now encoded before hashing. Digests are unchanged, so precomputed hashes stay valid.pixel_valuesand other base-model arguments being silently dropped for PEFT models in #3794:PeftModel.forwardhid the wrapped model's parameters, so the forward argument allowlist was built from the wrapper. It now unwraps first.taskandnum_images_per_sampleleaking into thetransformersforward pass as unexpected keyword arguments in #3794, via an explicit denylist of Sentence Transformers internal feature keys that yields to a model actually declaring them.trainer.evaluate()for VLM losses in #3794 by no longer gating media count tracking onself.training.dataloader_persistent_workers=Truethat cost is paid again every epoch and every evaluation, commonly making training slower thandataloader_num_workers=0. The examples now usedataloader_num_workers=2with persistent workers.sentence_transformers.util.resolve_idsresolves ID columns against lookup datasets, replacing PyLate'sKDProcessingfor knowledge distillation data.SpladePoolingpath that pools flattened sequences directly. Note that plain float16 remains the SparseEncoder recommendation, as unpadding reaches 2.3x to 2.4x against float16's 2.5x.Denseto load configs containing unknown keys in #3794, dropping them with a warning instead of raising, so newer or foreign saves remain loadable.Examples, Documentation, and Notebooks
Module.on_model_readyhook in #3794 for modules that need model-dependent state after construction, documented alongside the other module extension points.query_lengthanddocument_lengthtoTransformer, applying per-task tokenization caps inpreprocess, and surface both in the generated model cards.All Changes
chore] Increment dev version after release by @tomaarsen in #3914tests] Update slow pretrained tests for transformers v5.6+ by @tomaarsen in #3911v6] Add support for MultiVectorEncoder models by @tomaarsen in #3794v6] fix: compute similarity scores in float32 to avoid low-precision ties (HPS) by @KisuYang in #3892v6] fix: upcast cross-encoder logits to float32 before activation (HPS) by @KisuYang in #3893v6] Forward similarity kwargs to similarity functions, expose MaxSim device & chunking by @Samoed in #3905fix] Encode strings before hashing for xxhash 4.0 compatibility by @tomaarsen in #3928v6] refactor: return Python floats from CrossEncoder.rank by @tomaarsen in #3927v6] unifydocument_chunk_elements/pair_chunk_elementsinto a singlechunk_elementsby @tomaarsen in #3931v6] Expose chunk_elements on the ColBERT scorers by @tomaarsen in #3932feat] Allow models to declare required dependency versions, verified on load by @tomaarsen in #3934v6] Requiretrust_remote_code=Truefor custom module classes, dropping implicit local-directory trust by @tomaarsen in #3935v6] Fix XTR input validation, padding masks, and integer embedding support by @eSVeeF in #3937v6] Extend the merged column forward to the SentenceTransformer losses by @tomaarsen in #3938v6] Keep the prompt-excluded mask out of the feature dicts by [@&chore(deps): update dependency ruff to ^0.12.0 #82Configuration
📅 Schedule: (UTC)
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
This PR has been generated by Mend Renovate CLI.