{"slug":"co-designing-sparse-attention-for-efficient-long-06333f88","url":"https://silinex.xyz/wire/co-designing-sparse-attention-for-efficient-long-06333f88","json_url":"https://silinex.xyz/wire/co-designing-sparse-attention-for-efficient-long-06333f88.json","md_url":"https://silinex.xyz/wire/co-designing-sparse-attention-for-efficient-long-06333f88.md","headline":"Co-Designing Sparse Attention for Efficient Long-Context Inference","headline_zh":"协同设计稀疏注意力以实现高效长上下文推理","title":"Co-Designing Sparse Attention for Efficient Long-Context Inference","summary":"According to public reporting via NVIDIA Developer Blog, agentic workloads, long documents, and large codebases are pushing context lengths from tens of thousands of tokens to hundreds of thousands.","summary_zh":"根据NVIDIA开发者博客的公开报道，智能体工作负载、长文档和大型代码库正将上下文长度从数万token推向数十万token。","date_published":"2026-11-17T17:30:00.000+00:00","date_modified":"2026-11-18T10:22:26.458+00:00","author":{"type":"Organization","name":"NVIDIA Developer Blog","url":"https://developer.nvidia.com/blog/co-designing-sparse-attention-for-efficient-long-context-inference/"},"publisher":{"name":"Silicon Nexus","url":"https://silinex.xyz/"},"category":"silicon","language":"en","source_url":"https://developer.nvidia.com/blog/co-designing-sparse-attention-for-efficient-long-context-inference/","source_name":"NVIDIA Developer Blog","key_facts":["Agentic workloads, long documents, and large codebases are pushing context lengths from tens of thousands of tokens to hundreds of thousa…","At those lengths, attention becomes the most expensive… hs from tens of thousands of tokens to hundreds of thousands.","At those lengths, attention becomes the most expensive part of a transformer layer."],"key_facts_zh":["智能体工作负载、长文档和大型代码库正将上下文长度从数万token推向数十万……","在这样的长度下，注意力机制成为开销最大的部分……从数万个token扩展到数十万个token。","在这样的长度下，注意力机制成为 transformer 层中开销最大的部分。"],"tldr":["Agentic workloads, long documents, and large codebases are pushing context lengths from tens of thousands of tokens to hundreds of thousa…","At those lengths, attention becomes the most expensive… hs from tens of thousands of tokens to hundreds of thousands.","At those lengths, attention becomes the most expensive part of a transformer layer."],"tldr_zh":["智能体工作负载、长文档和大型代码库正将上下文长度从数万token推向数十万……","在这样的长度下，注意力机制成为开销最大的部分……从数万个token扩展到数十万个token。","在这样的长度下，注意力机制成为 transformer 层中开销最大的部分。"],"entities":[],"signal_type":"other","impact_score":0,"key_metrics":[],"body_md":"## Wire brief\n\nAccording to public reporting via NVIDIA Developer Blog, agentic workloads, long documents, and large codebases are pushing context lengths from tens of thousands of tokens to hundreds of thousands.\n\nIn related context, at those lengths, attention becomes the most expensive… hs from tens of thousands of tokens to hundreds of thousands. Dense attention scores every query against every key, so its cost per query grows with the context length N, and at 128K tokens it can take most of the prefill time in a layer. The model-level solution is sparse attention, a transformer technique in which each query attends to only a small subset of the context, chosen per query, instead of all of it.\n\nReporting also notes that this cuts both the compute and the memory traffic of attention on long context. It focuses on model-level sparse attention, where sparsity is trained into the model, rather than drop-in methods applied after training such as BLASST , MInference , or Quest . A production FP8 kernel is used to sweep the indexer’s design knobs (query heads, key heads, head dimension, compression ratio, and speculative-decoding depth), measure how Top-k cost scales, and compare token-level with block-level core attention. Each section explains what the hardware rewards and why, and ends with a guideline you can apply to your own design.\n\n---\n\nSource desk: **NVIDIA Developer Blog** · [origin link](https://developer.nvidia.com/blog/co-designing-sparse-attention-for-efficient-long-context-inference/)\n","body_md_zh":"## 电讯正文\n\n根据NVIDIA开发者博客的公开报道，智能体工作负载、长文档和大型代码库正将上下文长度从数万token推向数十万token。\n\n在相关背景下，当上下文达到如此长度时，注意力机制成为开销最大的环节……上下文长度从数万token增至数十万token。稠密注意力会将每个查询与每个键逐一计算，因此每个查询的开销随上下文长度N增长，而在128K token时，它可能占据一层中大部分预填充时间。模型层面的解决方案是稀疏注意力，这是一种Transformer技术，每个查询只关注上下文中的一小部分子集（按查询逐个选择），而非关注全部上下文。\n\n报道还指出，这同时降低了长上下文下注意力的计算量和内存流量。其重点在于模型层面的稀疏注意力，即稀疏性被训练进模型之中，而非训练后应用的即插即用方法，例如BLASST、MInference或Quest。研究使用一个生产级FP8内核来扫描索引器的各项设计参数（查询头数、键头数、头维度、压缩比和推测解码深度），测量Top-k开销如何随规模变化，并比较token级与块级核心注意力。每一节都解释了硬件偏好什么及其原因，并以一条可应用于你自己设计的指导原则作结。\n\n---\n\n来源台：**NVIDIA Developer Blog** · [原文链接](https://developer.nvidia.com/blog/co-designing-sparse-attention-for-efficient-long-context-inference/)\n","observations":{"human":10,"bot":7,"crawlers":[{"crawlerId":"AhrefsBot","hits":3,"lastSeenAt":"2026-11-18T08:06:23.706Z"},{"crawlerId":"ClaudeBot","hits":2,"lastSeenAt":"2026-11-20T09:57:27.987Z"},{"crawlerId":"Googlebot","hits":2,"lastSeenAt":"2026-11-18T13:04:27.769Z"}],"last_seen_at":"2026-11-20T09:57:27.987Z","note":"Counts are request logs classified by User-Agent. User-Agents can be spoofed. This is not proof of search-engine or LLM inclusion."}}