HuRo: Robotizing Human Videos for Scalable VLA Pretraining

Technical Analysis is a robotization pipeline that turns heterogeneous egocentric human videos into robot-aligned observations and retargeted actions, and use it to construct the HuRo dataset: 630K robotized episodes and 142M frames. Scaling this data improves real-world performance under both in-distribution and out-of-distribution conditions, with the largest gains in the out-of-distribution setting. High-level Analogy: Imagine you have a huge library of home videos showing people doing all sorts of tasks – cooking, building, cleaning. A robot wants to learn from these videos, but it has a different body (mechanical arms instead of human ones) and understands different instructions (robot joint movements instead of human muscle commands). HuRo acts like a sophisticated 'dubbing studio' for these human videos. First, it meticulously tracks what the human hands are doing – their precise movements and paths. This is like analyzing a dancer's moves. Then, it 'translates' these human hand movements into exact instructions for the robot's specific mechanical arms and hands, ensuring the robot can actually perform those actions. This is the 'action dubbing.' Simultaneously, the studio 'edits' the video itself. It digitally removes the human hands and arms from the scene and then overlays a realistic 3D model of the target robot's arms and hands. These robot arms are shown moving exactly as the 'translated' instructions dictate. It also makes sure the camera perspective looks like…