10+ hours of podcasts, in 5 minutes.
6 quotes from 1 episode on No Priors, each with a timestamped link to the source.
Noam Brown argues that evaluating AI models requires measuring the test-time compute budget spent on a task. Model capabilities are fundamentally bottlenecked by the weeks or months of processing time required to complete complex work.
Current models act as complements to humans because the systems lack strategic judgment and research taste.
“…they don't have very good research taste right now.”
From No Overnight Explosion: AI's Recursive Self-Improvement is Bottlenecked by Time
Noam Brown describes the concept of a fast intelligence takeoff as a relative term in an industry already moving at high speeds.
“…fast takeoff is relative; things are moving very fast.”
From No Overnight Explosion: AI's Recursive Self-Improvement is Bottlenecked by Time
Plotting model performance against cost, tokens, or time reveals an accurate picture of capabilities. Current scaling frameworks fail to measure this extended processing factor.
“My claim is the proper way to evaluate the models now is you either have some kind of budget for the benchmark whether it's tokens or cost or time or whatever, or you plot the performance as a function of the amount of test-time compute that's going into the model…”
“The current frameworks and responsible scaling policies, they don't really account for the amount of test-time compute…”
Achieving powerful results forces models to run for extended periods. This heavy reliance on test-time compute makes system evolution a drawn-out process.
“If it requires so much test time on compute to unlock the full capabilities of the model…”
From No Overnight Explosion: AI's Recursive Self-Improvement is Bottlenecked by Time
“…then that means you're bottlenecked by time; things can only go so fast because the models need to run for long enough to actually do something really, really powerful.”
From No Overnight Explosion: AI's Recursive Self-Improvement is Bottlenecked by Time
How we attribute quotes. Every quote was matched against the episode transcript, so the words and the timestamp are real (we trim filler words like "um", nothing else). The name comes from our written summary of the episode. YouTube gives us no voice-by-voice transcript, so open the timestamp to hear who is talking. See a wrong name? Tell us and we fix or remove it.
10+ hours of podcasts, distilled into one 5-minute read. Free, every Sunday morning.
One email a week. Unsubscribe with one click.