Key Takeaways

  • Simple linear models and mean predictors routinely beat complex neural networks on noisy single-cell biology benchmarks.
  • In the Arc Institute Virtual Cell Challenge, naive scoring metrics allowed basic baselines to expose heavy overfitting in deep perturbation models.
  • Theis published findings five years ago showing that deep learning is unnecessary for cell type prediction within a single organ.
  • Complex architectures like flow matching in CellFlow and spatial models like Nicheformer only earn their keep on subtle, nonlinear tasks across tissues.

Why Deep Models Fail on Cell Benchmarks

Machine learning researchers love complexity. They want multi-layer transformers and massive latent spaces. In biology, that instinct often leads to embarrassing results.

Fabian Theis, Director of the Computational Health Center at Helmholtz Munich, watches this cycle repeat. Teams download single-cell transcriptomics data, train a massive model, and claim state of the art results. Then someone runs a linear regression or predicts the mean, and the deep network loses.

“I think we had a paper maybe 5 or 6 years ago where we had the title very plain, 'Deep learning is not needed for cell type prediction,'” (play from 27:17) Theis recalls. “This is true if it's within an organ.” (play from 27:26)

The issue reached a head during high-profile community competitions. “The cherry on the cake was most definitely that Arc competition which called the thing not just perturbation prediction challenge, but virtual cell challenge,” (play from 28:34) Theis says. The Arc Institute Virtual Cell Challenge asked participants to predict how cells respond to specific perturbations, such as gene knockouts or drug treatments. Many entrants deployed enormous foundation models. Yet flawed evaluation metrics allowed simple shortcuts to win. Models that merely predicted the dataset average or fit a linear trend outperformed deep architectures that hallucinated biological noise.

When biological datasets contain high measurement noise and technical batch effects, deep networks memorize the artifacts. They capture experimental drift instead of genuine biological mechanisms. As Theis explains: “When this really large-scale foundation model representations came out everyone was buying into the hype and maybe bit over buying. Then obviously after this comes what's it called, the sobering up value.” (play from 20:54)

When Complex Architectures Actually Win

Simpler is better until the task demands true biological generalizability. Theis does not reject deep learning; he rejects unearned complexity.

Linear models work when predicting cell identities within one tissue because the gene expression profiles separate cleanly. But when researchers push into cross-tissue transfer, spatial microenvironments, or continuous cellular transitions, linear baselines hit a hard wall. That boundary is where generative tools justify their compute cost.

In the Arc challenge, once the evaluation metrics were cleaned up, sophisticated modeling proved its value. “The winning model that was then in the end crowned as the actual winning model essentially is a flow matching based thing,” (play from 29:09) Theis explains. Techniques like flow matching, implemented in tools like CellFlow, can model dynamic trajectories of cells under perturbation rather than static snapshots. Similarly, models like Nicheformer capture spatial context across neighboring cells that a linear predictor cannot represent.

The difference lies in the setup. If the task is linear and local, deep networks overfit. If the task requires modeling complex regulatory networks across diverse conditions, deep architectures win, provided the baseline is honest.

“I always tell my students, we always need to do, I know you want to do the most fancy deep learning thing, but please have the baseline,” (play from 29:51) Theis says. “Say for drugs, a baseline as simple as just getting better at the mean.” (play from 30:01)

What to Do With This

Run an unweighted mean predictor and a standard linear regression on your core problem before writing your next neural network. If your deep architecture cannot beat a linear baseline by at least fifteen percent on clean validation data, delete the neural net and ship the baseline.