Key Takeaways

  • Leash Bio trained a classifier on 1,815 prolific chemists in the public ChEMBL database and hit 60% top-five accuracy predicting which scientist built a given molecule.
  • Swapping out real molecular formulas for an 1,800-element vector of author probabilities predicted binding activity just as well as true chemical representations.
  • Models trained on historical lab records pick up human habits, superstitious reagent preferences, and molecular weight bloat instead of physical binding laws.
  • Brandon Anderson pointed out that simple heuristics like sorting by molecular weight frequently serve as the single biggest predictor of binding activity in public benchmarks.

The 1,815 Chemist Experiment

Clever Hans was a nineteenth-century horse that appeared to solve arithmetic by tapping his hoof. In reality, the animal watched subtle, involuntary facial cues from his owner. Once the owner left the room, Hans could not do math.

Ian Quigley, co-founder and CEO of Leash Bio, argues that modern drug discovery models suffer from the exact same trick. Machine learning teams assume their neural nets learn biophysical principles from public repositories like ChEMBL. To test that assumption, Leash isolated molecules authored by 1,815 prolific chemists in ChEMBL and trained a simple classifier to identify the author based only on the compound.

“It's 1,815 chemists and then we train a classifier to predict a chemist from looking at the molecule alone and we got a top five accuracy of 60%,” Quigley said. “It's a very very easy problem.”

Human chemists have distinct aesthetic preferences. They return to familiar reactions, lean on favorite catalog building blocks, and carry individual habits from graduate school. When an algorithm scans those molecules, it does not see generalizable chemistry. It sees the personal signature of an individual working in a specific lab.

The Vector That Replaced Chemistry

The Leash team pushed the experiment one step further. They stripped all chemical structures and molecular formulas from the dataset entirely. In place of the atoms and bonds, they gave the model an 1,800-element vector containing the probability distribution of who created the compound.

“And then we chose to represent the molecules not with any chemical structures or formulas at all, but just a vector of the probability of creator,” Quigley explained. “So now it's this 1,800 element vector and it's got some probability of who made it and you can use that representation and then train a model to predict activity from ChEMBL and it does as well as if you use chemical formulas.”

The implications are chilling for anyone building on historical datasets. If an identity vector predicts binding affinity as accurately as a SMILES string or a graph neural network, the model never learned biophysics. It inferred researcher intent.

“We really think that it's learning a lot of sociology,” Quigley said. “It's not learning ground truth. It is just learning the choices and the fingerprints of the people who are trying to develop these molecules.”

The Molecular Weight Trap

This sociological shortcut gets amplified by standard medicinal chemistry workflows. During an optimization campaign, chemists habitually attach additional functional groups to a lead molecule to improve potency or patentability. Over time, active molecules reliably become heavier.

Brandon Anderson noted how easily models exploit this bias: “It's well known that things like molecular weight will just, you know, all you have to do is sort by molecular weight. That's like one of the single single biggest predictors you can possibly have.”

When a model predicts high binding affinity for a heavy compound, it often is not calculating electrostatic fit or entropic penalties. It is simply recognizing that a chemist spent three months synthesizing heavier analogs around a confirmed hit. As Quigley warned, “if the models can identify the chemist by looking at the molecule alone, then maybe the models can infer chemist intent, which means maybe they don't learn anything at all.”

What to Do With This

Audit your training data for creator leakage before running your next benchmark. Replace your primary input features with metadata like author ID, lab origin, or timestamp, and test whether that dummy metadata matches your production model's accuracy. If it does, discard the public corpus and invest in unbiased, high-throughput physical screens that systematically test negative and random space.