Diogo Almeida on Why Public AI Benchmarks Lie
Diogo Almeida explains why second-tier AI labs game public benchmarks and why Anthropic models beat paper metrics in production.
The best ideas from 30+ top podcasts, every week.
TypeSafe AI co-founder and CEO Diogo Almeida explains why AI reliability, not raw capability or compute, remains the core roadblock to real-world software automation. Almeida details his theory of 'AI's bitterest lesson,' critiques the inflation of benchmark metrics across frontier and open-weight labs, and unpacks the technological economics behind decision models versus generative LLMs.
Diogo Almeida explains why second-tier AI labs game public benchmarks and why Anthropic models beat paper metrics in production.
Diogo Almeida explains why 5% model variance destroys enterprise automation and why judgment models must replace generative LLMs.
Diogo Almeida explains why compute scaling plateaued at GPT-4.5 and why task definition matters far more than data, compute, or algorithms.
TypeSafe AI's Diogo Almeida explains why enterprise AI co-workers fail and why real automation comes from software engineering, not bespoke models.
Diogo Almeida explains why Chinese open-source AI is only a threat if you suffer from mono-model horse blinders, and how to verify tool calls externally.
The week's best AI, tech, business, science, and culture podcasts in one short read. Free, every Sunday.
You get both newsletters, The Throughline and The Carry. No spam. Unsubscribe with one click.