The 100,000 Sandbox Problem — Akshat Bubna, Modal CTO
Akshat Bubna, CTO of Modal, discusses the company's evolution from a developer experience platform to one optimized for agent experience and specialized AI workloads. He details Modal's "super cloud" strategy, including elastic sandboxing, advanced autoscaling for inference, and innovative networking like I6PN and RDMA for distributed training. The conversation also covers Modal's contributions to LLM inference with speculative decoding and its capacity management approach in the competitive AI compute market.
- Modal has open-sourced Dlash, a block-based speculative decoding technique designed to accelerate LLM inference without compromising output quality. Read →
- Modal's SDK team pivoted from Developer Experience (DX) to Agent Experience (AX), recognizing that AI agents are now the primary consumers of infrastructure. Read →
- Modal developed I6PN, a private IPv6 overlay network, to enable secure communication between containers within the same workspace without public internet exposure. Read →