In-the-Wild Compliant Manipulation with UMI-FT

Technical Analysis is a handheld data-collection platform that mounts compact, six-axis force/torque sensors on each finger, enabling finger-level wrench measurements alongside RGB, depth, and pose. High-level Analogy: Imagine you're teaching a robot how to do delicate tasks, like wiping a whiteboard or screwing in a lightbulb. Traditionally, you might show it where to move its hand and what it should look at. But humans also feel how much pressure they're applying and how firmly they're gripping something. If a robot can't feel, it might press too hard and break the lightbulb, or not hard enough and drop the eraser. This paper introduces a new 'teaching glove' for robots (UMI-FT) that has tiny, coin-sized 'fingertip sensors.' When a human wears this glove and demonstrates a task, the robot doesn't just see the motion; it feels the exact forces the human is using – how gently they grip, how much pressure they apply against a surface. It's like giving the robot a sense of touch, allowing it to learn the 'just right' amount of force needed for sensitive tasks, making it much more capable and less likely to break things. Motivation of the Work Many everyday tasks require a 'goldilocks' amount of force: not too little (which leads to failure), and not too much (which causes damage). Current methods for teaching robots often miss this crucial aspect because they don't capture force information. Humans naturally use their sense of touch and force feedback to perform delicate…