TWIST2: Scalable, Portable, and Holistic Humanoid Data Collection System

Technical Analysis "TWIST2" is a portable, mocap-free humanoid teleoperation and data collection system that preserves full whole-body control while advancing scalability. High-level Analogy: Imagine you want to teach a robot to perform complex human-like tasks, like folding laundry or kicking a ball. Instead of programming every tiny movement, this system is like a super-smart 'digital puppeteer' setup. You (the human) wear a lightweight virtual reality headset and some trackers, and these capture your entire body's movements—from your head turns to your finger gestures—without needing a huge, expensive motion capture studio. The robot also has its own 'eyes' (a camera on its head) that stream what it sees directly to your VR headset, so you can guide it precisely, almost as if you are the robot. The system then records all these guided movements, allowing the robot to learn from your examples and eventually perform these intricate tasks autonomously, simply by 'seeing' and reacting to its environment. Motivation of the Work In the world of advanced robotics, massive amounts of data have led to breakthroughs, much like how large language models (like ChatGPT) learned from vast texts. However, humanoid robots, which are designed to mimic humans, haven't seen the same level of progress because it's really hard to collect good, large-scale data for them. Existing systems for controlling humanoid robots (called teleoperation) generally fall into three categories: Decoupled…