Teams hit a wall when document QA, log analysis, and multi-file code review outgrow conventional context windows.
The bottleneck
Standard transformer attention grows quadratically with sequence length. Cost and memory rise faster than most product budgets can absorb. That breaks economics long before it becomes a pure research curiosity.
Continuum's answer
Fully linear decoder layers with selective full-attention anchors aim to deliver:
- More predictable memory growth on long sequences
- Custom Triton kernels in Continuum flash-linear-attention
- NOPE-oriented length extrapolation for native multi-million-token context
These are engineering choices with tradeoffs. Linear mechanisms can change how information is mixed across distance. The right evaluation is task-level quality under your prompts and documents — not only theoretical complexity.
Choosing the right workload
Linear attention is most relevant when sequence length is the binding constraint. For short, independent requests, conventional architectures may still be the simplest option. The architectural decision should follow the workload: document scale, concurrency, accuracy requirements, latency expectations, and the cost of retaining source context.
An evaluation plan that engineering can defend
Benchmark end-to-end behavior, not only token throughput:
- Ingestion and preparation of representative long documents
- Retrieval where it still helps, even with long context available
- Generation quality on real questions with known answers
- Memory and latency under concurrent load
- Traceability from answer claims back to source spans
- Failure modes: distraction, omission, and overconfident synthesis
Kernel work as a product surface
Serving speed is part of whether a long-context model is usable. Kernel optimization tasks on Arena exist to pressure-test inference paths under published challenge rules. Treat challenge outcomes as research evidence, not as a client case study.
Practical decision rule
Use a long-context linear model when the operating problem requires holding a large, coherent record in one working set and retrieval alone repeatedly fails. Prefer retrieval-first designs when evidence is sparse, frequently updated, or access-controlled in small units.
Continue with [Introducing Continuum1-9B](/updates/introducing-continuum1-9b) or [When long context is the wrong tool](/updates/when-long-context-is-the-wrong-tool).