Key Takeaways

  • Leash Bio built Minos to test automated chemistry agents inside environments mapped in advance with DNA-encoded libraries.
  • Measuring every chemical interaction before running tests removes wet-lab latency and enables 90,000 agent rollouts.
  • The benchmark grades the scientific process: how agents spend a fixed budget, write hypotheses, and choose between exploration and exploitation.
  • Early runs expose three repeated agent failure modes: quitting early, testing molecules that contradict their own written logic, and re-testing the exact same compounds.

Most AI Evaluations Test Answers Instead of Reasoning

When developers evaluate language models on scientific tasks, they usually measure the end result. Did the model predict the correct crystal structure? Did it score a high binding affinity? If the model guesses right, the benchmark marks it as capable.

That creates a blind spot. A model can stumble onto a workable molecule by luck, or it can hallucinate a plausible path while burning through resources. In physical drug discovery, running an assay takes weeks and thousands of dollars. You cannot afford an agent that guesses randomly or squanders its budget on repetitive tests.

Ian Quigley and the team at Leash Bio built Minos to evaluate the intermediate decisions. Rather than sending agents into physical laboratories where cycles take months, or into synthetic simulators that drift from physical reality, Leash runs exhaustive biological experiments upfront using DNA-encoded libraries. They construct an environment where every possible interaction within that space is already measured and cataloged. The platform holds the complete answer key before the agent ever runs a query.

Running 90,000 Rollouts on an Exhaustive Answer Key

Because the biological data is already gathered, the agent experiences zero physical latency. As Brandon noted, “It essentially allows them to create like a virtual sandbox where agents can explore and make decisions in real time as if they were medicinal chemists and get instant feedback but with like a huge amount of exploratory rollouts.”

Leash drops the agent into this closed space and gives it a specific goal alongside a strict experimental budget. The agent must state its structure-activity relationship (SAR) hypothesis in plain text, select a batch of candidate molecules to test, review the resulting binding data, and decide what to test next.

“And what they'll do is they'll put out in readable language SAR hypothesis about why they think these molecules are working or not working,” Quigley explained. “And then they'll go in and with this limited experimental budget, they'll test this hypothesis. And because it's all pre-measured, you can roll out a lot of these. We're up to about 90,000.”

At 90,000 rollouts, clear behavioral flaws surface. You see agents that run two batches and quit before spending their budget. You see agents that write a clear hypothesis, observe contradictory data from a test, and continue selecting molecules that match their disproven idea. Other models burn scarce budget by re-requesting assay results for compounds they tested five turns earlier.

Because Leash knows every data point in the entire space, they can grade the agent against what was counterfactually possible. Quigley pointed out: “And so then you can start to interrogate how good is the agent at making scientific decisions. Because maybe it only got to like the thing that's half as good as the best thing in the entire environment.”

What to Do With This

Audit your internal agent evaluation suite tomorrow morning. If your benchmarks only check whether an agent reached the desired final state, add a budget counter and log the intermediate tool calls. Flag any run where the agent repeats an API call, stops before exhausting cheap retries, or acts against the evidence returned in previous steps.