The AI Training Data Market Will Hit $100 Billion

Key Takeaways

• Sahil Bhaiwala estimates the AI training data sector expanded from $1 billion three years ago to $10 billion today, on track to reach $100 billion within three years.
• Frontier labs have shifted from bounding-box annotation to reinforcement learning sandboxes like ATLAS for corporate tax accounting and VIALS for life sciences.
• Effective training environments now replicate full professional operating systems, including mock email accounts, messy folder directories, and domain software.
• Bad training data now damages labs primarily by consuming researcher hours and high-cost compute runs on dead-end checkpoints.

Simple Annotation Cannot Train Knowledge Workers

For years, the training data industry meant paying workers low hourly wages to draw boxes around stop signs or label sentiment in tweets. That model worked when models were learning pattern recognition. It fails completely when models try to do professional knowledge work.

“Three years ago, this was a billion dollar market,”▶ Play (play from 0:00) says Sahil Bhaiwala of Handshake AI. “It's now a $10 billion market. Three years from now, I don't see how it's not a hundred billion market.”▶ Play (play from 0:01)

The reason for that ten-fold expansion is the shift toward expert reinforcement learning. Frontier models do not just need examples of good text. They need to practice actions inside realistic digital environments. Bhaiwala points out that training models to act like professionals requires actual professionals: “You typically need some level of real expertise to be able to train these models, whether that's being an investment banker or that's being a lawyer or a doctor.”▶ 2:50 (play from 2:50)

When a software engineer writes code, a compiler tells the model if the program works. Most white-collar work does not have a compiler. As Bhaiwala puts it: “What's much harder is defining what good looks like. Most knowledge work is not perfectly verifiable. There's some gradient of quality.”▶ 4:17 (play from 4:17) Defining that gradient requires human judges who have spent years doing the job.

Synthetic Offices Replace Static Datasets

To evaluate whether an agent can perform corporate work, Handshake AI builds end-to-end simulated environments. In environments like ATLAS, models face complex financial modeling and tax accounting tasks. In VIALS, they face visual life sciences evaluations.

These setups go far beyond standard text prompts. An agent receives a task, but to complete it, the model must read mock corporate inboxes, search through nested file folders, open spreadsheets, and run calculations inside real software tools. It must deal with messy inputs, irrelevant attachments, and incomplete instructions, just like an associate at an accounting firm.

Building these sandboxes has become the real bottleneck in AI development. Garrett Lord emphasizes that compute alone cannot fix a bad dataset: “What really matters to researchers is just like high quality data.”▶ 2:26:45 (play from 2:26:45)

If a lab trains a model on poorly constructed benchmarks, the financial penalty is severe. “From the lab perspective you really need to make sure that the data you're getting is high quality and you can trust it,”▶ 30:12 (play from 30:12) Bhaiwala says. “The opportunity cost now of getting bad data is just so much higher. It's a ton of researcher time. Compute and researcher time probably the two most valuable resources at a lab.”▶ 30:20 (play from 30:20) Feeding flawed data into a model wastes millions of dollars in GPU clusters and burns months of top researchers diagnosing phantom failures.

What to Do With This

Audit your internal product evaluation dataset this week. Replace your synthetic test questions with five actual email threads, messy customer spreadsheets, and ambiguous briefs your team solved last month. Score your model on whether it can pull data across multiple unformatted files, rather than grading it on clean single-turn prompts.