---
title: "Subquadratic sparse attention finally shipped"
date: 2026-05-01
series: 8
summary: "Your next long-context model probably won't run quadratic attention."
voice: builder
tags: [sparse-attention, nsa, dsa, ssa, long-context, deepseek-v3.2-exp, subquadratic, lightning-indexer]
image: "/notes/subquadratic-sparse-attention/image.png"
imageAlt: "3-architecture comparison card"
linkedin: null
sources:
  - title: "NSA paper (DeepSeek, ACL 2025)"
    url: https://arxiv.org/abs/2502.11089
  - title: "DeepSeek-V3.2-Exp release"
    url: https://api-docs.deepseek.com/news/news250929
  - title: "DeepSeek-V3.2 paper"
    url: https://arxiv.org/abs/2512.02556
  - title: "SGLang DSA support"
    url: https://github.com/sgl-project/sglang
  - title: "Subquadratic launch coverage (independent)"
    url: https://thenewstack.io/subquadratic-12-million-context-window/
  - title: "Subquadratic skepticism coverage"
    url: https://venturebeat.com/technology/miami-startup-subquadratic-claims-1-000x-ai-efficiency-gain-with-subq-model-researchers-demand-independent-proof
dateApprox: true
---

Your next long-context model probably won't run quadratic attention.

For five years, subquadratic attention was a research artifact. In 2025 it shipped.

**Three architectures to know:**

**NSA — Native Sparse Attention** (DeepSeek, ACL 2025 best paper)
Three parallel paths per query: compressed coarse-grained tokens, selectively retained fine-grained tokens, and a sliding window. Hardware-aligned. Trainable end-to-end — not bolted on after pretraining.

**DSA — DeepSeek Sparse Attention** (DeepSeek-V3.2-Exp, 2025-09-29)
Lightning indexer + fine-grained token selection. The first frontier model where sparse attention is the production path, not an ablation. MMLU-Pro 85.0 (par with V3.1-Terminus dense). AIME 2025: 89.3 vs 88.4. API price cut >50%.

**SSA — Subquadratic Selective Attention** (Subquadratic, launched 2026-05-05)
Content-dependent selection. 1M tokens production, 12M tokens research. Claims 52× speedup at 1M tokens vs FlashAttention-2 on B200s. Independent benchmarks pending — discount accordingly.

**What it means for your stack today:**

- SGLang v0.5.9 ships TRT-LLM DSA kernels. Flag: `--nsa-prefill-backend trtllm`. 3–5× speedup on Blackwell.
- A 1M-token system prompt is now a budget question, not a research project.
- Long-context evals get harder, not easier. Sparse models pass needle-in-haystack and still miss multi-hop reasoning at depth — your eval harness needs a long-context reasoning section, not just retrieval.

**The trap:** *subquadratic* is a complexity class, not a quality guarantee. Mamba and RWKV were both subquadratic. Both ended up hybrid. Bet on architectures that ship par-quality benchmarks, not ones that ship throughput plots.