Continuum1-9B is Innomium's long-context foundation model for reasoning, math, and document-scale workflows. It is published as an internal product so engineers can inspect architecture, kernels, and evaluation snapshots before planning an engagement.
Highlights
- ~8.6B parameters (BF16, sharded safetensors)
- 2,097,152-token native context with linear compute design
- Reported snapshots such as ~75% MMLU under Innomium evaluation protocols
Architecture
Hybrid Gated Linear Attention and Gated DeltaNet layers with No Positional Embeddings (NOPE) target stable extrapolation far beyond ordinary training windows. Custom Triton work lives in the Continuum flash-linear-attention package referenced from the model card.
What long context changes — and what it does not
Long context is valuable when a workflow needs evidence distributed across a large record: a technical archive, a codebase, a sequence of logs, or a multi-document investigation. It does not remove the need for retrieval, data controls, or an evaluation plan. It changes the range of material the system can reason over in one pass.
For practical adoption, teams should define:
- the document boundary and retention rules
- expected response time and hardware budget
- how answers are attributed back to source material
- safety and privacy constraints for sensitive documents
Reproducibility and responsible evaluation
Benchmark numbers are a snapshot of a defined evaluation protocol, not a guarantee for every use case. Production evaluation should always include the data, prompts, guardrails, and latency expectations of the deployment itself.
Load from the published Hugging Face repository following the model card, typically with remote code enabled only after you have reviewed the implementation and trust boundary.
When Continuum is the wrong default
If requests are short, independent, and well served by retrieval plus a standard model, Continuum may add complexity without payoff. See [When long context is the wrong tool](/updates/when-long-context-is-the-wrong-tool) and [Why linear attention for 2M context](/updates/why-linear-attention-2m-context).
Commercial use follows the license on the model card. Contact Innomium if you need adaptation, evaluation design, or a production integration program.