10+ hours of podcasts, in 5 minutes.
11 quotes from 1 episode on No Priors, each with a timestamped link to the source.
Walter Goodwin argues memory bandwidth limits modern AI capabilities far more than raw compute power. Hardware developers scaled processing operations 1,000,000x over 20 years, yet current architectures still choke on highly sparse models.
Internal hardware programs at major technology firms often exist primarily to force volume discounts from external chip suppliers.
“There's a bit of a joke today that the sort of first party efforts their primary purpose is to reduce the price that people pay Nvidia.”
From Why Custom AI Chips Are a Dangerous Trap for Frontier Labs
Many application-specific hardware designs currently on the market offer nearly identical architectures.
“…if you look across this entire space of AI ASICs, one of the things that is very striking is there is a lot of relatively identical chips out there.”
From Why Custom AI Chips Are a Dangerous Trap for Frontier Labs
Developers want to build highly sparse mixture-of-expert architectures routing 1 in 256 parameters. Standard graphics processing units cannot serve these designs efficiently because memory delivery starves the compute cores.
“These mixture of expert models, it's pretty well known that actually ideally we would make them sparser and sparser and sparser. So for like ISO intelligence, you will save a ton of flops if you go from being like 1 in 16 sparse on your to 1 in 128, 1 in 256.”
“One of the challenges, one of the headwinds to doing that is actually that it becomes incredibly prohibitive on today's HBM based GPUs, XPUs to serve those models efficiently. You end up often bandwidth bottlenecked…”
A lab running entirely on custom hardware faces a 9-month delay if a competitor discovers a computational breakthrough. That delay occurs while waiting to deploy hardware capable of running the updated algorithms.
“Suppose I'm lab one and I've gone all in on some proprietary silicon and then lab 2 discovers some new computational breakthrough that delivers far better computational efficiencies for the same level of intelligence but it only works on the chip that they've decided to deploy. I could die in the 9 months before I get to deploy enough of that chip that I've also now gained that kind of 5x in computational efficiency.”
From Why Custom AI Chips Are a Dangerous Trap for Frontier Labs
Foundries enforce a physical minimum of 3 to 5 months to manufacture and return a silicon design. Compressing the time between a technical observation and high-volume delivery captures market value.
“…from the moment you send the chip to them to getting it back is you know 3 to 5 months even in a kind of super hot lot scenario. And so these are the kind of innate latencies I guess in the industry.”
“…the shorter you can make that latency, that gap between an observation and realizing that bet in volume, that is where there's an, you know, an enormous amount of value to be captured.”
“For the first kind of two years or so of the company's life…”
From Why Fractile Ditched SRAM for Custom High-Bandwidth DRAM
“There are two things that grow with AI today…”
From Why Fractile Ditched SRAM for Custom High-Bandwidth DRAM
“Like many technical challenges, it boils down to a slightly mundane technical observation, which is we need chips that have this kind of particular ineffable property, which is incredibly high bandwidth to memory.”
From Why Fractile Ditched SRAM for Custom High-Bandwidth DRAM
“We've scaled flops like a millionfold in the last 20 years…”
How we attribute quotes. Every quote was matched against the episode transcript, so the words and the timestamp are real (we trim filler words like "um", nothing else). The name comes from our written summary of the episode, and we use it only when a separate check of the captions finds that person on the episode. YouTube gives us no voice-by-voice transcript, so open the timestamp to hear who is talking. See a wrong name? Tell us and we fix or remove it.
10+ hours of podcasts, distilled into one 5-minute read. Free, every Sunday.
For now, every subscriber gets both newsletters. No spam. Unsubscribe with one click.