Key Takeaways

  • Hardcoded biological assumptions hurt model performance once data expands. Theis points out that decades of machine learning show rigid a priori constraints consistently lose predictive power to flexible architectures.
  • AlphaFold proved this shift in structural biology. Early iterations relied on explicit rotational symmetry rules, but later iterations discarded those rigid geometric constraints as the network learned symmetries directly from data.
  • Perturbation biology remains trapped in a data-poor regime. Single-cell state catalogs contain millions of observations, but experimental interventions, meaning what researchers physically do to a cell, remain far scarcer.
  • Hybrid architectures are a temporary bridge, not a permanent architecture. Until perturbation datasets reach parity with raw observational data, teams must use hybrid models today while planning to discard handcrafted biological rules tomorrow.

The Disagreement

Computational biology splits into two opposing camps whenever a team starts designing a model.

Side A argues that biology is too complex, noisy, and dangerous to treat as a black box. Proponents build strict physical and biological constraints directly into model architectures. They enforce conservation laws, rotational invariants, and known pathway topologies. The argument sounds responsible: hardcoding biological rules guarantees that the model will never predict an impossible physical state, and it allows models to train on tiny datasets without hallucinating.

Side B bets on Rich Sutton's Bitter Lesson: human cleverness always loses to computation and data. Theis sides firmly with empirical history here. “I think literature has really clearly shown over I think now decades at least for of us all very many data sets that if we build an a priori inductive biases too strongly, that we might lose predictive power,” (play from 18:16) Theis explains. He points directly to structural biology as proof: “And only in later versions, rotation symmetry was automatically found by the model just with more data and was a bit more flexible because whatever physical constraint you put in, it's not always as rigid, and then you sort of gain something if you let the model learn from this.” (play from 19:30)

Who's Right (and When They're Wrong)

Side B is right in the long run, but Side A holds the ground in perturbation biology right now.

The tension comes down to an asymmetry in data collection. Single-cell sequencing has produced hundreds of millions of transcriptomic profiles. Observational cell data is abundant. But perturbation data, such as knocking out genes with CRISPR or exposing cells to thousands of small molecules, is orders of magnitude smaller. As Theis puts it: “And I think in our field with this perturbation space scale, the cell scale I think is big, but the perturbation space scale, like the stuff that we do to it not being as big, I don't think we're yet in the super data rich regime.” (play from 18:52)

When your action space is starved for data, a purely unconstrained model overfits or fails to generalize across unseen compounds. That is why Theis and his team at Helmholtz Munich develop generative approaches like CellFlow and closed-loop platforms like LabCompass while still using hybrid designs. “So, I think in our case these hybrid models might be still the thing to go for,” (play from 19:04) Theis notes, “but more and more of course, and this is I don't know if you can call it sad, but the honest learning from big scale machine learning is that at some point data beats human-made additional concepts in these models.” (play from 19:10)

If you are predicting protein folding or baseline cell states, Side B wins immediately. Build flexible architectures and let scale discover the geometry. But if you are predicting high-dimensional drug perturbations across rare cell types, pure data scaling will fail you this year. Use hybrid models as scaffolding, but design them modularly so you can discard the handcrafted rules as automated wet labs scale your perturbation counts.

What to Do With This

Audit your model architecture this week. Separate the core neural network from any handcrafted physical constraints or biological pathway masks you added to stabilize early training. If removing a biological constraint degrades accuracy on your current benchmark, keep it, but set a hard data threshold where that constraint gets turned off and benchmarked again.