OpenAI's 'Evals Crisis': Why Your AI Benchmarks Are Broken
OpenAI's Mark Chen on the 'evals crisis' for superhuman AI. Learn why traditional benchmarks fail and how OpenAI separates eval creation from model optimization to avoid self-deception.
Cooking with OpenAI’s Research Chief: AGI, o1, Evals, and Scaling Laws — Mark Chen
With Mark Chen, Sean · Sunday, June 28, 2026
OpenAI's Chief Research Officer, Mark Chen, joins Alessio Fanelli for a cooking session while discussing OpenAI's core research strategies. Topics include the ongoing relevance of scaling laws, the 'evals crisis' in AI, the future trajectory of AGI development, and how aspiring researchers can develop crucial 'research taste' within the field.
Key takeaways
- AI models are now so advanced they've saturated traditional benchmarks like the SAT, creating an "evals crisis" at OpenAI, according to Chief Research Officer Mark Chen. Read more →
- OpenAI's Mark Chen predicts AGI is "coming soon," fundamentally shifting innovation generation from humans to autonomous models. Read more →
- Forget the PhD: Mark Chen says a formal machine learning degree isn't a prerequisite for AI research; creative problem-solving and attention to detail are what truly matter, often found in fields like trading. Read more →
3 articles from this episode
OpenAI: AGI Is 'Coming Soon,' Your Role Is 'Vibe Researcher'
OpenAI's Mark Chen says AGI is near, pushing human researchers into 'vibe' roles, orchestrating models with 'good taste' across unified architectures.
OpenAI's Mark Chen: Build AI Research Taste by Replicating Papers
Forget the PhD. OpenAI's Mark Chen says the best way to develop AI 'research taste' is to replicate published papers. Get his concrete method here.
More Latent Space episodes
- The Future of Claude Code: Mods, Mutable Software, & Multiplayer Agents — Thariq Shihipar, AnthropicOct 4 · 4 articles
- Which GPU Clouds Are Actually Good? | ClusterMAX 3.0Oct 4 · 6 articles
- Recursive Language Models — Alex Zhang, MIT PhDOct 4 · 6 articles
- OpenAI’s New Agent Stack: Computer Use, Decisions API, UltraFast, Dots—Ari Weinstein & Nikunj HandaOct 4 · 6 articles