The Inference Frontier: from 100 to 10,000 tokens per second — Sean Lie, Cerebras CTO
Cerebras co-founder and CTO Sean Lie joins swyx following Hot Chips to discuss the launch of the CS-4 wafer-scale system, preview the roadmap to 10,000 tokens per second on CS-5, and analyze OpenAI's new Jalapeño chip. Lie breaks down why 100–200 TPS is becoming the new batch mode, the physical limits of competing SRAM and DRAM architectures, the shift toward heterogeneous data centers, and the strategic semiconductor competition between the US and China.
- Silicon logic design is no longer the main constraint in AI accelerators; integration outside the die is where performance gains live. Read →
- Cerebras CTO Sean Lie argues that current standard inference speeds of 100 to 200 tokens per second will soon feel like offline batch processing. Read →
- Discrete SRAM chips hold only megabytes of memory per die, forcing systems to link thousands of separate chips simply to store weights for trillion-parameter frontier models. Read →
- Frontier data centers scaling from hundreds of megawatts to gigawatts must treat the entire facility as a single disaggregated computer rather than a uniform grid of identical GPUs. Read →
- OpenAI deploys Cerebras wafer-scale hardware for live operational emergencies where response speed directly affects service uptime. Read →
- Chinese research labs now build roughly 95% of the highest-quality open-weight models used by developers worldwide. Read →