✉ The Friday AI Brief: the week's 5 best AI stories, tools & comparisons — in your inbox every Friday morning.

Cerebras logo

Cerebras Puts Disaggregated Inference Into Production — and Claims 5x More Tokens per Megawatt

Cerebras has put disaggregated inference into production: GPUs handle prompt processing while its wafer-scale systems generate the response, delivering five times more tokens per megawatt than a Cerebras-only setup at the same generation speed. Why it matters: power, not chips, is the binding constraint on AI capacity — a 5x gain in tokens per megawatt is a direct answer to it.

November 17, 2026

What happened

Cerebras announced that its disaggregated inference solution is now in production on the Cerebras Inference Cloud and available to infrastructure partners. Inference has two stages: prefill, which processes the prompt, and decode, which generates output tokens one at a time. Cerebras is splitting them across different hardware — accelerators such as AMD Instinct GPUs handle prefill, and Cerebras CS-3 systems handle decode, the stage where its on-chip SRAM and very high memory bandwidth are strongest.

The production result

In Cerebras’s measured production deployment, the combined setup delivered 5x the throughput per megawatt of a CS-3-only configuration, with no loss in token generation speed. The company says the measurement comes from a shared GPT-OSS-120B endpoint on its inference cloud on November 14, comparing throughput per megawatt before and after disaggregation was enabled. Partners can adopt the setup through a reference architecture with either Cerebras’s end-to-end disaggregated software stack or a decode-only SDK; early-access partners named by the company include Amazon Web Services, Fireworks AI, Gimlet Labs and General Compute. Cerebras says it plans to deploy disaggregated inference broadly across hundreds of megawatts in its own cloud and partner deployments by the end of 2027, alongside its recently announced CS-4 and a planned 700MW of data center capacity.

Why it matters

Agentic tools like coding assistants need fast, interactive responses, and serving them is increasingly limited by how much power a data center can get, not by how many chips it can buy. Cerebras’s answer is deliberately un-purist: instead of insisting its wafers do everything, it hands the prompt-heavy half of the job to GPUs — including AMD’s — and keeps the half it is best at. If the 5x figure holds up in partner deployments, the same megawatt serves five times the interactive AI traffic, which is the kind of efficiency gain that changes capacity planning, not just benchmark charts. The number is company-measured on one endpoint, so treat it as a strong claim awaiting outside confirmation — but the architecture itself is now running in production, not on a slide.

Frequently asked questions

What is disaggregated inference?

Splitting an AI model’s two inference stages — prefill (processing the prompt) and decode (generating the answer) — across different hardware optimized for each, instead of running both on the same machine.

How much faster is Cerebras’s production deployment?

Cerebras reports 5x the throughput per megawatt versus a CS-3-only configuration, with no loss in token generation speed, measured on a shared GPT-OSS-120B endpoint on November 14, 2026.

Who can use Cerebras disaggregated inference?

It is live on the Cerebras Inference Cloud, and infrastructure partners can deploy it in their own data centers. Early-access partners named by Cerebras include AWS, Fireworks AI, Gimlet Labs and General Compute.

Sources: Cerebras, Unite.AI

Leave a Comment

Your email address will not be published. Required fields are marked *

Get the 5 best AI tools every week

Top AI news, tools, and prompts — one short email. Free, unsubscribe anytime.

Run a newsletter of your own? Monetize and grow it with SparkLoop →

Scroll to Top