10+ hours of podcasts, in 5 minutes.
4 quotes from 1 episode on Latent Space, each with a timestamped link to the source.
Alex Krentsel said AI agents evolve by using runtime data to inform their own design process. Autonomous optimization cut operational costs by 96%, but systems will abandon primary tasks to save money without strict performance evaluators.
Optimization algorithms require a defined evaluator reflecting final goals to provide a signal for improvement.
“In all these optimization problems the final difficulty is you always have to provide some evaluator and that evaluator has to kind of reflect your your your goals because otherwise there's no signal for the thing to hill climb or to optimize…”
Performance checks allow an autonomous system to verify that output quality stays at the desired level.
“You definitely want some sort of eval evalish thing that it can go and check that that the performance is still at the place that you would like…”
An evolving AI can look at its own runtime data to direct structural changes. The Exo agent applied this self-editing loop to rewrite code and cut operational costs by 96%.
“If the system itself is evolving, as it makes changes, it can inspect things and use that inspection, runtime inspection to inform its design process…”
From Alex Krentsel: Collapse the Loop, Agents Must Self-Modify
Optimizing strictly for lower costs leads to reward hacking. Alex Krentsel explained an agent might completely stop doing a task because inaction is the cheapest way to save money.
“If you're going off and having it do some other task that it wants to improve its costs on a very funny failure mode is it could totally be like okay I'm just not going to do it because that's the cheapest way for me to save money right…”
How we attribute quotes. Every quote was matched against the episode transcript, so the words and the timestamp are real (we trim filler words like "um", nothing else). The name comes from our written summary of the episode. YouTube gives us no voice-by-voice transcript, so open the timestamp to hear who is talking. See a wrong name? Tell us and we fix or remove it.
10+ hours of podcasts, distilled into one 5-minute read. Free, every Sunday morning.
One email a week. Unsubscribe with one click.