TWIST: Teleoperated Whole-Body Imitation System

Technical Analysis Caption Humanoid robotics keeps progressing and the shown capabilities of robots come from approaches for motion retargeting, maning mapping human motion to robot motions. Here comes "TWIST: Teleoperated Whole-Body Imitation System", a sophisticated system designed to allow humans to control every part of a humanoid robot simultaneously and in real-time, simply by performing the desired movement themselves. High-level Analogy: Imagine training a dancer (the robot) to perfectly mirror a highly skilled choreographer (the human MoCap data). Instead of just showing the dancer one pose at a time (pure RL), the TWIST system first uses a "privileged expert" (the Teacher Policy) who can see the next few seconds of the routine. This expert learns how to transition smoothly. Then, the actual working dancer (the Student Policy) learns from this expert using a combination of observing the expert's smooth moves (Behavior Cloning) and practicing with constant feedback (Reinforcement Learning). This hybrid training allows the student to perform complex moves smoothly in real-time, even if they can only see the current pose. Motivation of the Work For general-purpose robots to eventually live and work alongside humans, they must be able to perform versatile, coordinated, whole-body actions, like using a foot to open a door while carrying objects. However, current humanoid teleoperation systems face several limitations: Limited Whole-Body Coordination: Most existing…