EgoPHI: Estimating 3D Hand-Object Contact and Force from Egocentric Vision

Technical Analysis is a method that jointly estimates dense contact maps and 3D force distributions on hand and object meshes from a single egocentric RGB image and object geometry. High-level Analogy: Imagine you're wearing a camera on your head, recording yourself interacting with objects. It's easy to see where your hand touches a mug. But EgoPHI goes further: it's like having invisible sensors on your hand and the mug that tell you how hard each part of your hand is pressing, and in what direction, just by looking at the video. It's teaching a computer to 'feel' the pressure and forces involved in an interaction, purely from what it sees, not just where the contact is. Motivation of the Work Understanding how our hands physically interact with objects is fundamental to how we engage with the world. For computers to truly grasp this, they need to go beyond simply detecting contact and start understanding the forces at play. Knowing the magnitude and distribution of these forces can reveal a person's intent, the stability of a grip, or how precisely an object is being controlled. This information is incredibly valuable for applications like robotics, where robots could learn how to manipulate objects more effectively by observing human actions. The main problem is that current vision-based systems struggle to infer these crucial 3D forces, especially on complex, non-flat objects, due to the inherent challenges of egocentric vision (like hand occlusion and ambiguous 3D…