UMI on Legs: Making Manipulation Policies Mobile with Manipulation-Centric Whole-body Controllers

Technical Analysis is a framework for combining real-world human demonstrations with simulation trained whole-body controllers, providing a scalable approach for manipulation skills on robot dogs with arms. High-level Analogy: Imagine you're trying to teach a very agile, four-legged robot (like a robot dog with an arm) to do complex kitchen tasks, like tossing salad or delicately placing a cup on a saucer. Instead of painstakingly programming every single joint movement, or even having the robot practice thousands of times in the real kitchen (which is expensive and messy!), this system works like a 'Chef' and a 'Master Cook' working together. The Chef (Manipulation Policy): This is the high-level brains. We teach it by having a human simply show the desired hand movements with a special hand-held gripper (like a smart 'Chef's hand'). The Chef learns the path the hand needs to take (e.g., 'move the spoon from here to there in an arc') and what the gripper should do (open, close). It doesn't care how the robot's body moves, just the goal for its 'hand'. The Master Cook (Whole-Body Controller): This is the robot's expert body control system. Given the Chef's high-level hand path, the Master Cook figures out exactly how to move every single leg, joint, and the arm to make the hand follow that path smoothly and stably, even if the floor is slippery or the robot needs to lean to reach. It anticipates movements and coordinates the entire body, much like a master chef balancing…