10+ hours of podcasts, in 5 minutes.
5 quotes from 1 episode on Latent Space, each with a timestamped link to the source.
Ian Quigley states the primary test of a scientific AI agent is how efficiently it spends a limited experimental budget. High-performing models abandon failing chemical structures immediately to save resources.
Agents output clear hypotheses in readable language to explain why specific molecules succeed or fail.
“And what they'll do is they'll put out in readable language SAR hypothesis about why they think these molecules are working or not working…”
Testing systems in fully mapped environments exposes models that stop at mediocre discoveries.
“And so then you can start to interrogate how good is the agent at making scientific decisions. Because maybe it only got to like the thing that's half as good as the best thing in the entire environment.”
Better performing AI systems drop failing molecular neighborhoods right away. The rapid exit from bad paths stands out as a surprising result in recent tests.
“It turns out that more performant models are much better at abandoning kind of unproductive neighborhoods. And that was actually a really interesting surprise for us.”
Less capable systems trap themselves by running tiny modifications on low-scoring structures. Ian Quigley notes these models invent optimistic reasons to keep testing compounds with zero chance of success.
“Yeah, and what happens with the dumber models is that they'll just keep running little subtle changes on molecules that aren't ever going to go anywhere. Right? Like, 'Oh, these two, they have a low score, but I feel good about them.' And so I'm going to keep trying stuff.”
Evaluating an autonomous system involves tracking how it allocates limited experimental capital. Budget efficiency serves as a primary metric for improving scientific models.
“A lot of this is about how do you spend a limited experimental budget and then what can you do to do better at that?”
How we attribute quotes. Every quote was matched against the episode transcript, so the words and the timestamp are real (we trim filler words like "um", nothing else). The name comes from our written summary of the episode, and we use it only when a separate check of the captions finds that person on the episode. YouTube gives us no voice-by-voice transcript, so open the timestamp to hear who is talking. See a wrong name? Tell us and we fix or remove it.
10+ hours of podcasts, distilled into one 5-minute read. Free, every Sunday.
For now, every subscriber gets both newsletters. No spam. Unsubscribe with one click.