The best ideas from 30+ top podcasts, every week.
Host, Dwarkesh Podcast
4 quotes from 3 episodes on Dwarkesh Podcast, each with a timestamped link to the source.
Dwarkesh Patel argues that physical hardware availability constrains AI compute scaling, even as agent swarms find ways to train models faster. ASML builds fewer than 100 extreme ultraviolet lithography tools each year, restricting global wafer output growth to just 2x annually.
A 50% API price increase at 200,000 tokens exposes memory bandwidth restrictions inside the physical cluster hardware.
“With longer context, Gemini 3.1 is 50% more expensive if you go over 200k tokens than if you're below 200k tokens.”
From LLM API Costs: Context Length Reveals Memory Bottlenecks
Long context windows will allow models to fully automate the work of top AI researchers by 2028.
“Here's why I don't think that the current lack of continual learning in these models will stop full automation of the best AI researchers by 2028…”
AI compute demand grows 3x each year, while equipment availability restricts progress. ASML manufactures under 100 extreme ultraviolet lithography tools annually, capping leading-edge wafer output growth at roughly 2x.
“For this EUV tool, ASML produces fewer than 100 of them a year.”
Swarms of agents test smaller changes to bypass physical hardware constraints. By late 2027, top companies could provide enough daily compute for 10,000 individual agents to each run GPT-3 scale experiments.
“And if this trend continues, then by the end of 2027, OpenAI could almost give 10,000 agents, so the amount that were involved in solving the Millennium Prize problem, each enough compute to run a GPT3 size training run every single day.”
How we attribute quotes. Every quote was matched against the episode transcript, so the words and the timestamp are real (we trim filler words like "um", nothing else). The name comes from our written summary of the episode, and we use it only when a separate check of the captions finds that person on the episode. YouTube gives us no voice-by-voice transcript, so open the timestamp to hear who is talking. See a wrong name? Tell us and we fix or remove it.
The week's best AI, tech, business, science, and culture podcasts in one short read. Free, every Sunday.
You get both newsletters, The Throughline and The Carry. No spam. Unsubscribe with one click.