Amazon Web Services and AI chipmaker Cerebras Systems have announced a collaboration aimed at dramatically accelerating how AI models generate responses in the cloud. The partnership combines AWS’s Trainium processors with Cerebras’ wafer-scale AI systems in a new architecture designed specifically for inference, the stage when trained AI models generate outputs.
The technology will be deployed in AWS data centers and delivered through Amazon Bedrock, the company’s managed platform for generative AI. By pairing two specialized processors and linking them with AWS’s Elastic Fabric Adapter networking, the companies say they can significantly increase the speed and capacity of AI inference workloads.
“Inference is where AI delivers real value to customers, but speed remains a critical bottleneck for demanding workloads like real-time coding assistance and interactive applications,” said David Brown, VP of compute and machine learning services at AWS. He added that splitting inference tasks across different processors allows each system “to do what it’s best at,” producing performance that could exceed existing approaches.
The collaboration also carries strategic implications for AWS’s custom silicon strategy. Trainium processors were initially positioned as alternatives to GPUs for training large models. By applying them to inference workloads in combination with specialized chips from partners like Cerebras, AWS may broaden the role of its in-house silicon across the AI stack.
Disaggregated Inference
The collaboration centers on a technique known as disaggregated inference. Traditionally, AI accelerators perform every step of inference on the same chip. AWS and Cerebras are instead separating the process into two distinct stages, prompt processing and response generation, and assigning them to different hardware.
The first stage, often called prefill, analyzes the user’s prompt and prepares the model’s internal data structures. This step is computationally intensive but does not require large amounts of memory bandwidth. AWS’s Trainium processors, designed as custom AI chips for cloud workloads, are well suited to handle this portion of the pipeline.
The second stage, known as decode, generates the output tokens that form the model’s response. Because each token is produced sequentially and requires repeated access to model data stored in memory, decode places heavy demands on bandwidth. Cerebras’ CS-3 system, powered by its wafer-scale WSE-3 processor, is optimized for precisely this type of workload.
Cerebras’ chip architecture differs sharply from conventional processors. Rather than dividing a silicon wafer into many small chips, the company builds a single processor across the entire wafer. The resulting device includes roughly 900,000 cores and massive on-chip memory bandwidth. That design allows it to move data between logic and memory circuits far more quickly than many conventional AI accelerators.
Andrew Feldman, CEO of Cerebras, said the partnership expands access to the company’s technology. “Partnering with AWS to build a disaggregated inference solution will bring the fastest inference to a global customer base,” he said. “Every enterprise around the world will be able to benefit from blisteringly fast inference within their existing AWS environment.”
Optimizing Inference
The systems will run foundation models offered through Amazon Bedrock, including open-source LLMs as well as Amazon’s Nova models. Cerebras hardware will be installed directly in AWS data centers so customers can access the capability through the same cloud interfaces used for other AI services.
The initiative is part of a larger shift in the AI industry toward optimizing inference performance rather than focusing solely on training large models. As generative AI applications spread into enterprise workflows, from software development to automated customer support, response speed is central to user experience.
Some workloads, like AI coding agents, generate far more output tokens than typical chat interactions, creating additional pressure on infrastructure. In these cases, faster decode performance can significantly reduce latency and improve productivity for developers using AI tools.
The companies expect the combined Trainium and CS-3 infrastructure to become available through Amazon Bedrock in the coming months.
