Issue No. 46Week ending Sunday, November 15, 2026844 episodes · 3,675 write-ups
The Podcast Summary.

The best ideas from 30+ top podcasts, every week.

AI infrastructure and compute

Walter Goodwin on AI infrastructure and compute

33 quotes from 2 episodes on 20VC with Harry Stebbings and No Priors, each with a timestamped link to the source.

33 quotes2 episodes

The short version

Walter Goodwin argues that memory bandwidth physically limits the speed of AI inference and autonomous agents. Raw computing power grew by a factor of 1,000,000 over two decades while bandwidth grew 40 times, forcing a transition to vertical memory stacking to support multi-day tasks.

Most interesting insights

Current high-bandwidth memory architectures starve compute cores and fail to serve highly sparse mixture-of-expert models.

“One of the challenges, one of the headwinds to doing that is actually that it becomes incredibly prohibitive on today's HBM based GPUs, XPUs to serve those models efficiently. You end up often bandwidth bottlenecked…”

Walter Goodwin, No Priors · October 2026 · Watch at 30:06 ↗

From Why Memory Bandwidth Bottlenecks Sparse MoE Models

Committing to proprietary lab silicon carries a 9-month risk of death if a rival discovers an algorithmic breakthrough tied to different hardware.

“Suppose I'm lab one and I've gone all in on some proprietary silicon and then lab 2 discovers some new computational breakthrough that delivers far better computational efficiencies for the same level of intelligence but it only works on the chip that they've decided to deploy. I could die in the 9 months before I get to deploy enough of that chip that I've also now gained that kind of 5x in computational efficiency.”

Walter Goodwin, No Priors · October 2026 · Watch at 34:27 ↗

From Why Custom AI Chips Are a Dangerous Trap for Frontier Labs

Static random-access memory chips lack the physical capacity and economic viability to process large language model inference or store long context windows.

“You cannot solve LLM inference with these SRAM based chips. There isn't enough capacity. The cost to achieve speed is too high. The constraint on the context lengths you can operate on is very very stark.”

Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 1:02:12 ↗

From Why Mixed AI Chip Fleets Will Strand Capital

Top talking points

  1. Memory bandwidth severely lags behind raw compute scaling

    Over the last 20 years, computing power grew by a factor of 1,000,000 while memory bandwidth increased only 40 times. Autoregressive generation forces processors to pull all model weights across narrow physical wires for every single word.

    “We've scaled flops like a millionfold in the last 20 years…”

    Walter Goodwin, No Priors · October 2026 · Watch at 30:54 ↗

    From Why Memory Bandwidth Bottlenecks Sparse MoE Models

    “…bandwidth by contrast is kind of much more intrinsically physics limited. We've scaled it about 40fold in the same time. So you know it's lagged behind the flop scaling about 25,000 times.”

    Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 15:27 ↗

    From Walter Goodwin Says Memory Bandwidth Lags Compute by 25,000x

    “Every time I go and generate one of those words, I'm loading from that memory over that limited bandwidth all of the weights of my model.”

    Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 16:07 ↗

    From Walter Goodwin Says Memory Bandwidth Lags Compute by 25,000x

  2. Vertical memory stacking clears the token generation bottleneck

    Placing memory directly on top of the logic die replaces restricted lateral connections with millions of copper wires running straight down. Walter Goodwin notes this two-dimensional surface area physically removes the restriction on data flow.

    “…what we've been working on at Fractile over the past couple of years is uh essentially stacking that memory on top of the logic.”

    Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 19:44 ↗

    From Walter Goodwin Says Memory Bandwidth Lags Compute by 25,000x

    “When you put this on top, suddenly you have this whole two-dimensional area where you can have essentially those copper wires are now not going laterally. you have millions of them coming straight down.”

    Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 21:05 ↗

    From Walter Goodwin Says Memory Bandwidth Lags Compute by 25,000x

  3. Persistent reasoning tasks break existing hardware architectures

    Deploying 10,000 parallel agents to run reasoning loops for 80 hours requires millions of sequential token generations. Walter Goodwin states that no current silicon can handle these multi-day wait times.

    “…the many many millions of sequential token generation means you're waiting days and days and days to get one of these responses. And it's a space which somewhat tantalizingly hasn't been cracked at all by any of the silicon any of the chips that we have today.”

    Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 24:51 ↗

    From Why AI Silicon Breaks on Long-Running Swarms

    “Where I've both got 10,000 parallel agents all generating their own chains of thought and then they're running for 80 hours…”

    Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 44:51 ↗

    From Why AI Silicon Breaks on Long-Running Swarms

  4. Outsourcing backend chip design eliminates technical advantages

    Hardware startups and hyperscalers frequently use Broadcom to build their physical chips, turning specialized accelerators into generic platform plays. Sharing the same backend partner restricts custom builders to surface-level changes.

    “A lot of these chips today are built with a backend partner, for example, whether it's the TPU or Jalapeno MTIA from Meta, MatX or Etched. All of those are rumored to be using Broadcom.”

    Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 27:51 ↗

    From Walter Goodwin on Why Fractile Bypasses Broadcom

    “A lot of the sort of building blocks are dictated by the backend partner because it's a platform technology play. You get to change the color of the paint basically.”

    Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 30:23 ↗

    From Walter Goodwin on Why Fractile Bypasses Broadcom

  5. Finite data center power forces rapid hardware turnover

    Operators face a direct trade-off between peak token speed and token delivery per megawatt of power. Strict energy caps mean companies swap out their entire chip fleets every few years for denser architectures.

    “On these chips, you can either be slow and efficient in generating tokens or fast but deeply inefficient: far fewer tokens per second per megawatt of power…”

    Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 38:10 ↗

    From Walter Goodwin on Why Power Caps Will Clear Out Old AI Chips

    “If I have finite numbers of megawatts, I need to replace my chips every few years with better chips…”

    Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 1:13:26 ↗

    From Walter Goodwin on Why Power Caps Will Clear Out Old AI Chips

19 more quotes from Walter Goodwin

“Deepseek had done so well in building an inference stack to run their models on GPUs that they were getting, you know, much higher throughputs, much better interactivity, better speeds than than everybody else who had built a Deepseek inference server.”

Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 7:17 ↗

From Where Value Settles in the AI Stack

“What is commoditizing and I don't think anybody has ever argued it won't be is kind of you know the whole space of of cloud compute. You know one neocloud that is competent is much like another neocloud that is competent.”

Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 8:03 ↗

From Where Value Settles in the AI Stack

“The reason that over the last 30 years we've externalized memory into this this thing called DRAM and put it on different silicon. These are built in completely different foundries.”

Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 17:57 ↗

From Walter Goodwin Says Memory Bandwidth Lags Compute by 25,000x

“The memory stacking work, for instance, is incredibly complex and painful and requires convening multiple much larger supply chain partners to get them to do something that is highly custom…”

Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 30:42 ↗

From Walter Goodwin on Why Fractile Bypasses Broadcom

“…a different process node, maybe a slightly different DRAM node or DRAM generation as well that might be going into frontier HBM.”

Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 31:16 ↗

From Walter Goodwin on Why Fractile Bypasses Broadcom

“I think this is one of the places where China has somewhat stolen the march on the West…”

Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 40:22 ↗

From Walter Goodwin on Why Power Caps Will Clear Out Old AI Chips

“…training becomes a fixed cost and the deployment, inference, is just the use of these models to do useful things. It's the only part that matters.”

Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 42:51 ↗

From Why AI Silicon Breaks on Long-Running Swarms

“…the knob that we have to turn to get greater capability.”

Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 46:07 ↗

From Why AI Silicon Breaks on Long-Running Swarms

“I think one of the reasons why, you know, I am very long on chips in terms of where that value occurs or being one big place where that value occurs is actually because of the nature of the challenge of placing the right chip bet.”

Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 52:05 ↗

From Where Value Settles in the AI Stack

“And so you're very likely to end up in a place where some portion of that capacity in your data center is stranded…”

Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 58:37 ↗

From Why Mixed AI Chip Fleets Will Strand Capital

“…heterogeneity is actually a really challenging thing and I think it's a transient response to the fact that you don't today have a chip that can kind of do it all.”

Walter Goodwin, 20VC with Harry Stebbings · November 2026 · Watch at 59:06 ↗

From Why Mixed AI Chip Fleets Will Strand Capital

“…if you look across this entire space of AI ASICs, one of the things that is very striking is there is a lot of relatively identical chips out there.”

Walter Goodwin, No Priors · October 2026 · Watch at 2:45 ↗

From Why Custom AI Chips Are a Dangerous Trap for Frontier Labs

“For the first kind of two years or so of the company's life…”

Walter Goodwin, No Priors · October 2026 · Watch at 10:48 ↗

From Why Fractile Ditched SRAM for Custom High-Bandwidth DRAM

“There are two things that grow with AI today…”

Walter Goodwin, No Priors · October 2026 · Watch at 11:36 ↗

From Why Fractile Ditched SRAM for Custom High-Bandwidth DRAM

“Like many technical challenges, it boils down to a slightly mundane technical observation, which is we need chips that have this kind of particular ineffable property, which is incredibly high bandwidth to memory.”

Walter Goodwin, No Priors · October 2026 · Watch at 14:15 ↗

From Why Fractile Ditched SRAM for Custom High-Bandwidth DRAM

“…from the moment you send the chip to them to getting it back is you know 3 to 5 months even in a kind of super hot lot scenario. And so these are the kind of innate latencies I guess in the industry.”

Walter Goodwin, No Priors · October 2026 · Watch at 18:56 ↗

From Why AI Chip Startups Need a 3 to 6 Month Lead

“…the shorter you can make that latency, that gap between an observation and realizing that bet in volume, that is where there's an, you know, an enormous amount of value to be captured.”

Walter Goodwin, No Priors · October 2026 · Watch at 20:55 ↗

From Why AI Chip Startups Need a 3 to 6 Month Lead

“These mixture of expert models, it's pretty well known that actually ideally we would make them sparser and sparser and sparser. So for like ISO intelligence, you will save a ton of flops if you go from being like 1 in 16 sparse on your to 1 in 128, 1 in 256.”

Walter Goodwin, No Priors · October 2026 · Watch at 29:46 ↗

From Why Memory Bandwidth Bottlenecks Sparse MoE Models

“There's a bit of a joke today that the sort of first party efforts their primary purpose is to reduce the price that people pay Nvidia.”

Walter Goodwin, No Priors · October 2026 · Watch at 32:10 ↗

From Why Custom AI Chips Are a Dangerous Trap for Frontier Labs

Key takeaways from these write-ups

Where Value Settles in the AI Stack

  • Neocloud GPU hosting is commoditizing rapidly because competent hosting providers offer almost zero technical differentiation once clusters are online.
  • Model creators protect software margins by co-designing the runtime stack. DeepSeek achieved higher throughput and faster interactivity on identical GPUs by custom-building their inference server.

Why AI Silicon Breaks on Long-Running Swarms

  • Model training is shifting into a fixed upfront cost, making inference and test-time compute the primary operational expense of frontier intelligence.
  • Tackling heavy math and scientific challenges, such as the Navier-Stokes Millennium Prize problem, demands swarms of 10,000 agents generating tokens across 80 hours of continuous execution.

Walter Goodwin on Why Fractile Bypasses Broadcom

  • Hyperscalers and startups alike, including Google for TPU, Meta for MTIA (Jalapeno), MatX, and Etched, rely on Broadcom to handle backend physical design and foundry execution.
  • Backend design partners run platform plays; outsourcing physical design reduces architectural differentiation to what Walter Goodwin calls "changing the color of the paint."

Walter Goodwin on Why Power Caps Will Clear Out Old AI Chips

  • The primary ceiling on AI expansion has shifted from GPU allocation and wafer supply to grid capacity, physical land, and substation hookups.
  • Chip operators face a direct trade-off between generation speed and power efficiency, forcing a choice between peak token velocity and tokens delivered per megawatt.

Why Mixed AI Chip Fleets Will Strand Capital

  • Data centers that pair GPUs like Nvidia Vera Rubin with SRAM chips like Groq face severe utilization mismatches when AI workloads shift.
  • Disaggregating prefill, decode, and attention across separate hardware creates a brittle orchestration layer that Walter Goodwin expects will collapse over time.

How we attribute quotes. Every quote was matched against the episode transcript, so the words and the timestamp are real (we trim filler words like "um", nothing else). The name comes from our written summary of the episode, and we use it only when a separate check of the captions finds that person on the episode. YouTube gives us no voice-by-voice transcript, so open the timestamp to hear who is talking. See a wrong name? Tell us and we fix or remove it.

More

The Sunday Email

Get next Sunday's issue in your inbox.

The week's best AI, tech, business, science, and culture podcasts in one short read. Free, every Sunday.

You get both newsletters, The Throughline and The Carry. No spam. Unsubscribe with one click.