RBS-Attention: Accelerating Long-Prompt Prefill by Searching for Underestimated Blocks
The article presents a method that supplements block selection based on average similarity with a search for candidates that may hide important tokens. At a 128K context length, the authors report up to 20.65× faster attention during prefill; on Qwen3-32B, the RULER score was 88.65, compared with 89.52 for dense attention.

Which Part of Inference Does the Method Accelerate
Prefill processes the prompt before generation begins. Dense self-attention runs over the entire input, so a longer context increases the computational load of this stage.
RBS-Attention sparsely selects blocks for prefill and can reduce computational costs. But a block centroid averages token relevance and can hide an important token among irrelevant ones. The authors call this failure mode mean dilution.

The method’s practical purpose is to accelerate prefill without relying solely on the average relevance of an entire block.
How Block Selection Works
The method requires no training and uses two complementary branches:
- Base branch estimates a block’s average relevance using its centroid.
- Recovery branch takes into account a key block’s maximum radius and its distribution, which depends on the prompt, layer, and head. This helps identify blocks whose relevance may be underestimated.
Each branch applies a threshold to its own selection separately. The method then combines the masks and controls the contribution of recovery blocks. Execution remains on regular block-sparse FlashAttention.
This design is suited to scenarios where centroid-based selection might miss individual relevant tokens, while block-sparse execution still needs to be preserved.
What Speed Results Are Reported
Measurements were taken on an H100 with a 128K context. For Qwen3-30B-A3B-Instruct-2507-FP8, the paper reports the following speedups:
| Stage | Speedup |
|---|---|
| Standalone prefill-attention | 20.65× |
| Prefill-attention in vLLM | 11.92× |
| End-to-end time-to-first-token | 5.97× |
These results are for a specific configuration. They should be considered separately from quality evaluations and not extrapolated to other models without corresponding measurements.
What Is Known About Quality and Evaluations
On the dense Qwen3-32B model, the overall RULER accuracy was 88.65. Dense attention scored 89.52.
Additional quality evaluations were conducted on LongBench-v2, InfiniteBench, and Video-MME. Supporting experiments measured retention of relevant data, compared selectors at the same sparsity level, and tested different block sizes, thresholds, and memory budgets.
When choosing a method, speed and quality should be evaluated separately: the reported RULER results do not replace testing on tasks that matter for a specific application.
Frequently asked questions
Related materials
All materials
Toollery: Choose tools for LLM agents from tens of thousands of options without fine-tuning
Toollery builds a search index from descriptions of skills and tools, then passes the model only a limited set of candidates for each query. In experiments with libraries containing tens of thousands of functions, the approach helps maintain selection quality at a fixed top-K, although the overall effect depends on the workload profile and the provider’s caching.
11 November 2026
Step-by-Step Evaluation of GUI Agents for Dashboard Analysis: The DashAct Benchmark
Introducing DashAct, a benchmark of 348 human-verified trajectories that breaks down GUI agents’ work with interactive dashboards into three stages and four visual conditions. The results reveal persistent issues with action prediction and visual target grounding, as well as agents’ reliance on contextual and spatial support.
11 November 2026
WILSON: combining magnifications and patient slides for pathomorphological analysis and report generation
The preprint introduces WILSON, a vision-language model that encodes images of whole slides and series of slides from a single patient at different magnifications, rather than just a single “best” slide. Without task-specific fine-tuning, it outperforms specialized and larger slide models on a range of metrics while requiring substantially less compute, and its diagnostic descriptions received higher ratings from pathologists.
11 November 2026