SPARSEUP by Linkup: a sparse encoder with 149 million parameters
The article examines SPARSEUP’s architecture and results: 56.4 nDCG@10 on BEIR-13, as well as over 97% recall when paired with the Seismic index, at approximately 380 microseconds per query. The analysis covers techniques for preserving vector sparsity, comparisons with other models, and the terms of distribution under Apache 2.0.
What the model is and how to access it
SPARSEUP is an open sparse encoder from Linkup Research with 149 million parameters. Its backbone is ModernBERT, and it is licensed under Apache 2.0.
The weights are available on Hugging Face. You can load the model through Transformers or Sentence Transformers by setting trust_remote_code=True.
Linkup calls the model the strongest public vocabulary-based sparse encoder known to the company among models with fewer than 150 million parameters.
How sparse representations work
In SPARSEUP, each vector dimension corresponds to a vocabulary token. These vectors work well with inverted indexes and better match rare words.
The model calculates activations using the formula log(1 + ReLU(x - 15)). It then keeps the 12 strongest dimensions for each input token before max pooling.
This limits the number of active dimensions per token, not the overall vector size. Case folding merges Byte-level BPE variants, such as heat, Heat, Ġheat and ĠHeat, while retaining the highest weight.
After merging, the number of output dimensions drops from about 50,000 to 34,000. Queries and documents receive the prefixes [Q] and [D]; scoring is done using a dot product.
The maximum scoring length is 128 tokens for a query and 512 for a document. These limits matter when configuring input processing.
How the model was trained
SPARSEUP training started from the LateOn-unsupervised checkpoint. The team restored the original ModernBERT MLM head.
For fine-tuning, they used a mixture of LightOn data and applied contrastive training only. For each query, they selected seven hard negatives from a pool of 50 and also used in-batch negatives.
The team did not use cross-encoder distillation. Training fit on a single H100.
What the comparisons show
SPARSEUP's average score on BEIR-13 without MS MARCO is 56.4 nDCG@10. The model card reports the following results for sparse encoders:
| Model | BEIR-13, nDCG@10 |
|---|---|
| SPARSEUP | 56.4 |
| opensearch-neural-sparse-encoding-doc-v3-gte | 54.6 |
| opensearch-neural-sparse-encoding-v1 | 52.44 |
| ModernBERT-VT | 52.4 |
| splade-v3 | 51.7 |
| granite-embedding-30m-sparse | 50.6 |
LACONIC-1B scored 58.7, but belongs to a different size category: the model has 1 billion parameters.
In a controlled comparison using the same backbone and data, LateOn scored 58.9, DenseOn scored 57.9, and SPARSEUP scored 56.4. The paper's results report a gap of 1.52 points behind DenseOn and 2.5 behind LateOn.
SPARSEUP outperformed DenseOn on ArguAna and HotpotQA, and did better on Touché. The model trails on datasets with more semantic queries; this is especially noticeable on FiQA. DBPedia is also noted as a weak spot.
On decontaminated BEIR, the gap with DenseOn narrows to 0.17 points. For decontaminated NQ and MS MARCO, only 21 and 46 queries were used, respectively, so Linkup warns that the results are noisy.
Sparsity and selection criteria
On MS MARCO, SPARSEUP uses an average of 47 non-zero terms per query and 190 per document. For SPLADE-v3, the figures are 25 terms per query and 170 per document.
With the Seismic inverted index, the model achieves more than 97% recall relative to exact search at about 380 microseconds per query on a single thread.
SPARSEUP is worth considering if sparse representations and search through an inverted index are important. When evaluating it, you should also consider its results on the datasets that matter to you: the model is weaker on semantic queries, FiQA, and DBPedia.
Linkup believes that increasing the vector size could add 1–2 BEIR points. The team decided to keep the model sparse.
Frequently asked questions
Related materials
All materials
Moving to the Probability Simplex Reduces the Modality Gap in Contrastive Multimodal Models
The paper arXiv:2610.32890 proposes replacing spherical embeddings with a probabilistic simplex, adding lightweight softmax adapters and Total Variation similarity. Across five retrieval benchmarks, the cross-modal centroid ℓ2 distance decreases by 97–99% while retrieval quality is maintained or improved.
3 November 2026
How ActionRAG Reconsiders Evidence During Diffusion Across the Memory Graph of LLM Agents
A graph-memory architecture is proposed that carefully reconciles entities during indexing and reevaluates relationships at query time based on local structure. On MemoryAgentBench, the method achieved an average accuracy of 55.1, outperforming the strongest retrieval baseline by 5.4 percentage points while keeping synthesis costs low per query.
3 November 2026
Fifteen companies backed MDK Tether for mining automation
Equipment manufacturers, mining pools, and operators have taken on the development of integrations with Tether’s open suite to standardize equipment monitoring and management; the QVAC AI agent is already built on this foundation. Independent confirmation of reduced electricity costs or increased efficiency has not yet been published.
2 November 2026