Researchers at Xiaomi's LLM-Core have published the HySparse2 architecture in an arXiv paper (2609.26368). This technology is a hybrid sparse attention mechanism featuring a two-level Key-Value (KV) sharing mechanism and is designed to work with MiMo-V3 class agents that require processing long horizons.
The official branding of this development includes Xiaomi, MiMo, and HySparse2. This publication focuses on the research architecture and claimed performance metrics, distinguishing it from previous open-weight reviews of MiMo-V2.6.
HySparse2 divides the main part of the model into a YOCO-style self-decoder and a cross-decoder. At the outer level, the KV Bridging mechanism connects only the full attention layers, while the cross-decoder's KV caches are projected from the self-decoder's hidden states. At the inner level, enhanced KV reuse maintains HySparse-style sharing within each hybrid block, but it switches from block-level selection to token-level selection. Furthermore, instead of a separate sliding window branch, recent local windows are forcibly included in the sparse selections.
Thanks to these changes, the Prefill stage can complete after the self-decoder, bypassing the cross-decoder layers during cache construction. This is particularly useful in multi-step agents where input tokens largely consist of tool observations.
When using 80B-A3B MoE models trained on identical data and graphs, the paper reports that after minor fine-tuning, HySparse2 improves MRCR-v2 and RULER-v2 scores by 11.30 and 19.81 percentage points, respectively, while maintaining lower AgentPPL and LongPPL values within a context of 256 thousand tokens. When working with 1 million tokens, the analysis shows a 2.92× reduction in preprocessing FLOPs compared to HySparse and a 5.02× reduction compared to Hybrid SWA. It is also noted that the FP8 KV cache size decreases to approximately 2.69 GB compared to 6.72 GB and 12.09 GB for the specified base models.
Ablation studies show that token-level selection promotes stronger search with a fixed attention budget. When scaling up to 290B-A8B, the KV Bridging mechanism maintains comparable quality while enabling early exit during preprocessing. For specialists studying Chinese LLM system research, HySparse2 is an architectural signal behind the claims of MiMo-V3 efficiency, rather than a new open-weight release.
