Briefly: where exactly it hurts
When a long-context model serves a request, the main costs go not to the weights but to the KV cache — the stored keys and values for all tokens already read. The longer the document or dialogue, the more tensors have to be held in memory and the more data has to be re-read at each generation step. Hence the typical dilemma: either truncate the context or pay for it with memory and inference speed.
Existing solutions usually choose one of two extremes. Either they compress the entire state wholesale, risking the loss of fresh information that is exactly what's needed for the next answer, or they keep everything at the original precision and run into the accelerator's capacity.
It is precisely into this fork that the work discussed below tries to intervene.
What the authors propose
The paper "Minima-KV: Retention-Preserving KV Cache Compression with Mixed-Format Paged Attention" was submitted to arXiv on August 24, 2026 under number 2608.23834, category cs.AI, 13 pages and 3 figures, DOI 10.48550/arXiv.2608.23834. The authors are Sergii Kozyrev and Davyd Maiboroda of Minima AI, Inc. A curious detail: in the PDF link block on the preprint page, the caption points to the first author "and two others," although only two are listed in the bibliographic description. A small thing, but it reminds us that preprint metadata is worth double-checking.
The essence of the Minima-KV proposal is a hierarchy of KV cache pages, where different pages are stored in different numeric formats. The key word in the title is retention-preserving: the state of fresh requests is not evicted anywhere.
Anchors in FP8, archive in TQ3
The logic of the split is simple. Pages that were recently used and marked as protected (anchor) remain in FP8. Older pages that did not make it into the anchors are converted to packed TQ3 — a denser packed format. At the same time, each page of a live request can still be addressed directly: nothing is thrown out of the address space or replaced with an approximate copy.
How partial results are stitched together
The most interesting part is the arithmetic. Separate kernels for each format compute their own partial attention states, which are then combined via a globally normalized online-softmax merge. In plain terms: instead of bringing all pages to a single type before computation, the system computes each group's contribution in its own format and correctly sums the normalized results.
Thanks to this, decoding proceeds directly over the heterogeneous cache, without the so-called dense shadow — a full copy of the cache in a single format, which would negate all the memory savings.
What the measurements show
Memory and compression
The measurements were taken on profiles tied to a specific configuration of the Qwen3.6-27B model on a single NVIDIA RTX PRO 6000 Blackwell accelerator with 96 GB of memory. The result was 18.3 KiB of attention KV per live token. That is 3.50x compression relative to BF16 and 1.75x relative to FP8.
The second figure is more important than it seems at first glance. The comparison with FP8 shows that the gain is achieved not only through the move from half precision to more economical formats, but specifically through the hybrid storage scheme.
Quality on long context
The materializing profile — the one where hybrid pages are actually expanded into tensors — matched its dense control on RULER needle-in-a-haystack tasks at 16K. That is, on the needle-in-a-haystack search, no difference was found.
Then the trade-offs begin. On the LongBench v2 set of 503 questions, the deltas turned out to be negative: -0.80 percentage points at 16K, -0.60 at 32K, and -0.40 at 64K. Note the shape of the curve — the losses do not grow linearly with context length but fluctuate slightly. The authors offer no explanation, but the fact itself is worth keeping in mind when interpreting: a spread of a fraction of a percent is easily confused with measurement noise.
Canary check
A separate test — a single-pair canary — compared two direct decodings with requests of 59,008 tokens each. The results: active KV compressed by 3.625x, throughput was 0.9821x relative to the control, all 16 full-attention layers were routed without fallback to dense mode, and no dense shadow was retained.
Throughput slightly below one is, in essence, an honest trade: about two percent of speed for a multiple reduction in occupied memory.
Conclusions and caveats
The authors' stated conclusion is cautious: the results show a practical path to compressing long-context state in mixed formats without evicting pages of live requests. This is exactly what previous approaches lacked — compression that does not sacrifice fresh data for the sake of old.
What is worth keeping in mind when reading:
- The test profiles are tied to the configuration. This concerns a specific model and a specific accelerator; the portability of the figures to other architectures is not shown.
- The canary check is narrow. One pair of requests, one direct-decoding scenario — this is an illustration of workability, not statistics on load.
- The drop on LongBench v2 is non-zero. Fractions of a percentage point are not much, but in tasks where every unit of accuracy matters, it is already grounds for your own measurement.
- There is no data on latency under production conditions. Throughput in the canary test, by the description, all goes through 16 layers without a reserve path — it would be interesting to see how routing behaves in the worst case, when anchor pages become more numerous than usual.
The practical value of the work is not in record compression figures but in something else: it shows that a heterogeneous cache can be served directly, without assembling a full copy of it for each step. If this approach carries over to other models, long-context scenarios gain yet another lever in addition to hardware scaling.
