Key Takeaways
- Dyna trains its base model, Dyna-2, on over one million hours of egocentric human video to give it a physical world model before touching robot hardware.
- Fine-tuning a new manipulation skill, such as opening a refrigerator or chopping food, requires only 45 minutes to two hours of robot teleoperation demonstrations.
- Dyna bridges the embodiment gap between human hands and robot arms by pairing massive video datasets with tiny, targeted slices of hardware interaction.
- The company builds custom semi-humanoid robots instead of bipedal humanoids to handle long tasks, including multi-hour napkin folding at Din Tai Fung.
The Million-Hour Pre-Training Shortcut
Robotics has always had a data bottleneck. Large language models train on trillions of tokens scraped from the open web. Self-driving cars accumulate millions of highway miles by driving around cities. Robot manipulators, by contrast, learn nothing from text and cannot easily drive down public streets. Collecting teleoperation data by hand, moving robotic arms joint by joint, is painful, slow, and expensive.
Dyna takes a different path. Instead of trying to collect a billion hours of robotic teleoperation data, Chief Scientist Jason Ma looks at the largest existing video corpus of physical interaction: egocentric human recordings.
“Our approach is to first train a general model that's training on as much data as we can find,” Ma explains. “So that is when our dino report our model is trained over a million hours of human data as a starting point and then given that strong foundation what we found is we can quickly make the robot or make the model do many many different tasks with a very minimal amount of fine-tuning.”
Human video contains nearly every everyday physical motion. It shows how bread yields to a knife, how cabinet latches click open, and how liquids slosh in a glass. By watching one million hours of people moving through the world, the model acquires physical common sense before it ever fires a motor.
Bridging the Embodiment Gap with 45-Minute Slices
The catch with human video is physical mismatch. A human hand has five soft fingers, a wrist with wide range of motion, and sensory feedback from skin. A robot manipulator might have two rigid parallel plates or a steel three-finger gripper. Ma refers to this mismatch as the embodiment gap.
“Human data is so diverse like it covers every single physical activity you can possibly imagine and that gives robot the model such a strong prior on how things how the world can evolve around a embodied actor human,” Ma says. “But the issue with human data is that before your robot looks exactly like a human there's going to be some gap between human actions and the robot actions.”
To bridge that gap, Dyna couples the video foundation with short collection sessions. Ma found that once the world model understands object physics from video, teaching it to chop food or open a refrigerator does not take thousands of hours. It takes between 45 minutes and two hours of teleoperated robot demonstrations.
After human teleoperators record a few dozen demonstrations, the system starts practicing autonomously. “The robot rolls out, you can score it good, bad, uh somewhat good and then that data, the robot's only experience can be used to train the model.” The model uses its own execution data to refine grip force, approach angles, and recovery actions when an item slips.
Why Din Tai Fung Does Not Need Walking Robots
Silicon Valley obsesses over bipedal humanoid robots that walk like humans. Dyna skips the legs entirely. Real industrial deployments, such as folding napkins for hours at Din Tai Fung or running laundry cycles, do not require walking up flights of stairs. They require stable bases, high-payload upper torsos, and reliable manipulation cycles that run without tipping over.
By placing custom dual-arm setups on wheeled or stationary platforms, Dyna cuts out the balance problem. Every parameter in the model focuses on visual prediction and arm motor control. When a base model already knows how objects behave from a million hours of human video, keeping the hardware simple lets you deploy months before competitors finish calibrating their robot ankles.
What to Do With This
Audit your data pipeline tomorrow morning. If your team spends hundreds of engineering hours collecting manual, domain-specific labels for a narrow task, stop and look for large adjacent video or text corpora that capture the same underlying mechanics. Pre-train a representation layer on the noisy public data first, then spend 10 hours collecting targeted task data to bridge the interface gap.