What the top podcasts are saying about Evals.
Evals came up in 10 of 158 episodes (6%) in the four weeks to Nov 15, about the same share as in the earlier weeks (6%).
Share of episodes, four weeks at a time
The 10 episodes behind the newest bar
- Meta’s Muse: How we launched the most successful consumer product this year | Mona Sarant…Lenny's Podcast · week ending Nov 15 · said it 22 times
- What Frontier Labs Buy: A Data Deep Dive with Handshake AI — Garrett Lord & TeamLatent Space · week ending Nov 15 · said it 16 times
- Why Google wants to win the frontier conversation | Tulsee Doshi (Head of Product for Gem…Lenny's Podcast · week ending Nov 8 · said it 6 times
- Building Legal Agents When You Can't See the Data — Jacob Lauritzen, CTO of LegoraLatent Space · week ending Nov 8 · said it 6 times
- ⚡️ The 2026 Software Factory Standard — Zach Lloyd, Warp.devLatent Space · week ending Nov 8 · said it twice
- Abundant Intelligence at the Midwest AI SummitPractical AI · week ending Nov 1 · said it 9 times
- Building ChatGPT for Finance — Joseph Kim, OpenAI's Head of Financial ServicesLatent Space · week ending Nov 1 · said it 11 times
- How Anthropic ships the #1 LLM in the World — with Katelyn Lesse and Angela JiangLatent Space · week ending Nov 1 · said it 10 times
- Satya Nadella: Whether to Trust Frontier Models | Why Companies Need to Own Their Own Int…20VC with Harry Stebbings · week ending Nov 1 · said it 3 times
- Building what's next (Interview)The Changelog · week ending Oct 25 · said it twice
What the shows said about Evals
Key takeaways from our write-ups that name it, newest first, no more than two from one show. Each links to the full write-up.
Non-technical buyers and enterprise executives suffer the most from benchmark manipulation because they lack the time to run hands-on evals in production.
Diogo Almeida on Why Public AI Benchmarks Lie · 20VC with Harry Stebbings · week ending Nov 22Reliable evals start with 5 to 20 hand-crafted prompts on tasks where you have personal taste and current models show partial success.
Tulsee Doshi on Building AI Evals with 20 Prompts · Lenny's Podcast · week ending Nov 8Consumer giant Unilever automated order-to-cash and trade claims by building an agentic workflow around business evals.
Satya Nadella on Why AI Routers Beat Vertical Monopolies · 20VC with Harry Stebbings · week ending Nov 1Writing rigorous evaluations (evals) is the most valuable technical skill an AI product manager can develop.
How OpenAI Product Managers Feed Data to Research · Lenny's Podcast · week ending Oct 4AI models are now so advanced they've saturated traditional benchmarks like the SAT, creating an "evals crisis" at OpenAI, according to Chief Research Officer Mark Chen.
OpenAI's 'Evals Crisis': Why Your AI Benchmarks Are Broken · Latent Space · week ending Jun 28The development cycle becomes a constant loop of refining evals: "Close the session... and then I improve the evals and then I try from scratch again."
AI Engineering: No Excuse for Backlog Zero · How I AI · week ending Jun 21Satya Nadella states that 'private evaluations' (evals) are a critical form of intellectual property, enabling superior, tailored performance beyond generic public benchmarks.
Nadella: Build Specialist AI with Private Evals, Not Just Public Benchmarks · Latent Space · week ending Jun 7
Earliest mentions in our archive
The first episode from each show that said it, oldest first. The matched words are marked.
“…is useful but not everyone loves doing is building evals. You don't need to build hundreds of evals for them to be…”
How Anthropic’s product team moves faster than anyone else | Cat Wu (Head of Product, Cla…
“…transcripts what about evals how in the world do you like this is such a complex product…”
Inside Abridge: The AI Listening to 100 Million Doctor Visits — Abridge's Janie Lee & Cha…
“…they did was they created a test bench and a set of evals to be a financial analyst essentially and tested all of…”
Anthropic's Digital God, Pope vs AI, Job Loss Narrative Flips, Open Source Crackdown Comi…
“…that we can do is create really hard evals. And if you create the right tests and success criteria for a model, then…”
How this startup uses AI agents to eliminate bugs and optimize infrastructure
“…that issue? The fact that some of the evals that you would want to run are both beyond the scope of budget or time…”
Why Traditional Benchmarks Fail Modern AI Models with OpenAI Research Scientist Noam Brown
“…and our customers will log in to see their data and run evals. So, annotation platform called studio marketplace, call…”
Mercor Head of Product on Revenue Concentration from Frontier Labs
Most recent mentions
The newest episodes that count toward the chart, no more than two from one show.
“…it was a complete surprise and so then we were like wait like all of our evals have been over here. We actually need to…”
Meta’s Muse: How we launched the most successful consumer product this year | Mona Sarant…
“…about some of the real nitty-gritty of evals and data sets that you have. Um I just want to introduce Garrett Nishan…”
What Frontier Labs Buy: A Data Deep Dive with Handshake AI — Garrett Lord & Team
“…can catch problems, escalate to a person, constrain answers when something you know, and then recover when something gets through anyways. And then evals are repeatable tests. Evals are really important and they're not talked about as much as they should be, because they are determined whether…”
“…weights. It can be in context. It can be in your evals, right? You may have a benchmark. In fact, one of the things…”
Satya Nadella: Whether to Trust Frontier Models | Why Companies Need to Own Their Own Int…
“…example. Like how can I make sure Resend is easy for Python as well as for JavaScript developers, as well as for Ruby developers. So then we have evals for for different platforms. And then we try to see where do agents get stuck? Oh, is it Do they get stuck here, or there? How we can make sure…”
Which shows say it
| Show | Last four weeks | All weeks counted |
|---|---|---|
| Latent Space | 5 | 18 |
| Lenny's Podcast | 2 | 7 |
| 20VC with Harry Stebbings | 1 | 5 |
| Practical AI | 1 | 1 |
| The Changelog | 1 | 1 |
| How I AI | 0 | 2 |
| All-In Podcast | 0 | 1 |
| Cheeky Pint | 0 | 1 |
| Dwarkesh Podcast | 0 | 1 |
| No Priors | 0 | 1 |
More trends
Episodes in the newest four weeks:
How we count. An episode counts when its transcript says “evals” at least twice; plurals count too. We count a shortened re-upload of an episode we already have only once, and we skip the sponsor reads and show promos we can spot. A model marks them in every transcript and our code checks each mark against the words; we also skip the lines around phrases like “brought to you by” and any passage a show reads word for word in three or more episodes. The share is the counted episodes divided by all episodes from the tech, business and science shows we cover in the same four weeks, so a busy month of releases does not raise it by itself. The mix of shows does change from month to month, and one daily show can move a number. Weeks run Monday to Sunday, and the week in progress is left out until it ends. The chart starts with the week of April 6, 2026, when our archive became steady. A few older episodes can still appear under the earliest mentions, and some weeks have gaps. Transcripts are automatic captions, so we also match the spellings they use. The counts cover only the shows we summarize. Download the counts (JSON) or CSV.