33 quotes from 2 episodes on 20VC with Harry Stebbings and No Priors, each with a timestamped link to the source.
33 quotes2 episodes
The short version
Walter Goodwin argues that memory bandwidth physically limits the speed of AI inference and autonomous agents. Raw computing power grew by a factor of 1,000,000 over two decades while bandwidth grew 40 times, forcing a transition to vertical memory stacking to support multi-day tasks.
Most interesting insights
Current high-bandwidth memory architectures starve compute cores and fail to serve highly sparse mixture-of-expert models.
“One of the challenges, one of the headwinds to doing that is actually that it becomes incredibly prohibitive on today's HBM based GPUs, XPUs to serve those models efficiently. You end up often bandwidth bottlenecked…”
Committing to proprietary lab silicon carries a 9-month risk of death if a rival discovers an algorithmic breakthrough tied to different hardware.
“Suppose I'm lab one and I've gone all in on some proprietary silicon and then lab 2 discovers some new computational breakthrough that delivers far better computational efficiencies for the same level of intelligence but it only works on the chip that they've decided to deploy. I could die in the 9 months before I get to deploy enough of that chip that I've also now gained that kind of 5x in computational efficiency.”
Static random-access memory chips lack the physical capacity and economic viability to process large language model inference or store long context windows.
“You cannot solve LLM inference with these SRAM based chips. There isn't enough capacity. The cost to achieve speed is too high. The constraint on the context lengths you can operate on is very very stark.”
Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 1:02:12 ↗
Memory bandwidth severely lags behind raw compute scaling
Over the last 20 years, computing power grew by a factor of 1,000,000 while memory bandwidth increased only 40 times. Autoregressive generation forces processors to pull all model weights across narrow physical wires for every single word.
“We've scaled flops like a millionfold in the last 20 years…”
“…bandwidth by contrast is kind of much more intrinsically physics limited. We've scaled it about 40fold in the same time. So you know it's lagged behind the flop scaling about 25,000 times.”
Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 15:27 ↗
Vertical memory stacking clears the token generation bottleneck
Placing memory directly on top of the logic die replaces restricted lateral connections with millions of copper wires running straight down. Walter Goodwin notes this two-dimensional surface area physically removes the restriction on data flow.
“…what we've been working on at Fractile over the past couple of years is uh essentially stacking that memory on top of the logic.”
Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 19:44 ↗
“When you put this on top, suddenly you have this whole two-dimensional area where you can have essentially those copper wires are now not going laterally. you have millions of them coming straight down.”
Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 21:05 ↗
Deploying 10,000 parallel agents to run reasoning loops for 80 hours requires millions of sequential token generations. Walter Goodwin states that no current silicon can handle these multi-day wait times.
“…the many many millions of sequential token generation means you're waiting days and days and days to get one of these responses. And it's a space which somewhat tantalizingly hasn't been cracked at all by any of the silicon any of the chips that we have today.”
Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 24:51 ↗
Hardware startups and hyperscalers frequently use Broadcom to build their physical chips, turning specialized accelerators into generic platform plays. Sharing the same backend partner restricts custom builders to surface-level changes.
“A lot of these chips today are built with a backend partner, for example, whether it's the TPU or Jalapeno MTIA from Meta, MatX or Etched. All of those are rumored to be using Broadcom.”
Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 27:51 ↗
“A lot of the sort of building blocks are dictated by the backend partner because it's a platform technology play. You get to change the color of the paint basically.”
Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 30:23 ↗
Finite data center power forces rapid hardware turnover
Operators face a direct trade-off between peak token speed and token delivery per megawatt of power. Strict energy caps mean companies swap out their entire chip fleets every few years for denser architectures.
“On these chips, you can either be slow and efficient in generating tokens or fast but deeply inefficient: far fewer tokens per second per megawatt of power…”
Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 38:10 ↗
“Deepseek had done so well in building an inference stack to run their models on GPUs that they were getting, you know, much higher throughputs, much better interactivity, better speeds than than everybody else who had built a Deepseek inference server.”
Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 7:17 ↗
“What is commoditizing and I don't think anybody has ever argued it won't be is kind of you know the whole space of of cloud compute. You know one neocloud that is competent is much like another neocloud that is competent.”
Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 8:03 ↗
“The reason that over the last 30 years we've externalized memory into this this thing called DRAM and put it on different silicon. These are built in completely different foundries.”
Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 17:57 ↗
“The memory stacking work, for instance, is incredibly complex and painful and requires convening multiple much larger supply chain partners to get them to do something that is highly custom…”
Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 30:42 ↗
“I think one of the reasons why, you know, I am very long on chips in terms of where that value occurs or being one big place where that value occurs is actually because of the nature of the challenge of placing the right chip bet.”
Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 52:05 ↗
“…heterogeneity is actually a really challenging thing and I think it's a transient response to the fact that you don't today have a chip that can kind of do it all.”
Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 59:06 ↗
“…if you look across this entire space of AI ASICs, one of the things that is very striking is there is a lot of relatively identical chips out there.”
“Like many technical challenges, it boils down to a slightly mundane technical observation, which is we need chips that have this kind of particular ineffable property, which is incredibly high bandwidth to memory.”
“…from the moment you send the chip to them to getting it back is you know 3 to 5 months even in a kind of super hot lot scenario. And so these are the kind of innate latencies I guess in the industry.”
“…the shorter you can make that latency, that gap between an observation and realizing that bet in volume, that is where there's an, you know, an enormous amount of value to be captured.”
“These mixture of expert models, it's pretty well known that actually ideally we would make them sparser and sparser and sparser. So for like ISO intelligence, you will save a ton of flops if you go from being like 1 in 16 sparse on your to 1 in 128, 1 in 256.”
Neocloud GPU hosting is commoditizing rapidly because competent hosting providers offer almost zero technical differentiation once clusters are online.
Model creators protect software margins by co-designing the runtime stack. DeepSeek achieved higher throughput and faster interactivity on identical GPUs by custom-building their inference server.
Model training is shifting into a fixed upfront cost, making inference and test-time compute the primary operational expense of frontier intelligence.
Tackling heavy math and scientific challenges, such as the Navier-Stokes Millennium Prize problem, demands swarms of 10,000 agents generating tokens across 80 hours of continuous execution.
Hyperscalers and startups alike, including Google for TPU, Meta for MTIA (Jalapeno), MatX, and Etched, rely on Broadcom to handle backend physical design and foundry execution.
Backend design partners run platform plays; outsourcing physical design reduces architectural differentiation to what Walter Goodwin calls "changing the color of the paint."
The primary ceiling on AI expansion has shifted from GPU allocation and wafer supply to grid capacity, physical land, and substation hookups.
Chip operators face a direct trade-off between generation speed and power efficiency, forcing a choice between peak token velocity and tokens delivered per megawatt.
Data centers that pair GPUs like Nvidia Vera Rubin with SRAM chips like Groq face severe utilization mismatches when AI workloads shift.
Disaggregating prefill, decode, and attention across separate hardware creates a brittle orchestration layer that Walter Goodwin expects will collapse over time.
How we attribute quotes. Every quote was matched against the episode transcript, so the words and the timestamp are real (we trim filler words like "um", nothing else). The name comes from our written summary of the episode, and we use it only when a separate check of the captions finds that person on the episode. YouTube gives us no voice-by-voice transcript, so open the timestamp to hear who is talking. See a wrong name? Tell us and we fix or remove it.