AI researchers debate how close we are to recursive self-improvement
Dwarkesh Patel hosts AI researchers John Schulman, Charlie O’Neill, and Beren Millidge to debate the frontier of artificial intelligence and recursive self-improvement. They discuss the limits of transformers and reinforcement learning, the mechanisms of model distillation and continual learning, and provide timeline predictions for drop-in remote workers and superhuman AI research.
- Post-training reinforcement learning produces very few actual information bits, making top capabilities trivial to distill once seen in public API outputs. Read →
- Iterative fine-tuning on live user traces causes catastrophic forgetting after hundreds of micro-updates, erasing base capabilities. Read →
- Charlie O'Neill estimates autonomous remote workers arrive in roughly one year if granted direct programmatic tools like Slack bots, or two years if restricted to human browser interfaces. Read →
- Beren Millidge points out that poor sample efficiency forces labs into sim-to-real training, because models currently demand thousands of human interactions to master complex tasks. Read →
- Charlie O'Neill questions whether the current recipe of transformers and reinforcement learning can discover major algorithmic jumps or reach the theoretical optimum of a learner on a chip. Read →
- Mid-training on synthetic reasoning datasets does roughly 80 percent of the work before reinforcement learning begins. Read →