HaWoR: World-Space Hand Motion Reconstruction from Egocentric Videos

<h1 id="technical" Technical Analysis</h1 Jump to Section: Motivation motivation | Results summary-of-results | Conclusions final-conclusions "HaWoR" https://hawor-project.github.io/ is a high-fidelity method for hand motion reconstruction in world coordinates from egocentric videos. We propose to decouple the task by reconstructing the hand motion in the camera space and estimating the camera trajectory in the world coordinate system. High-level Analogy : Imagine you're trying to describe exactly how someone's hand is moving while they're wearing a camera on their head an 'egocentric' video . It's like having two expert detectives working together, then combining their findings: 1. The 'Hand Detective' : This expert watches only the hand, focusing on its intricate movements like finger wiggles and rotations, but only what's visible in the camera frame. If the hand disappears from view like reaching out of frame or being temporarily blocked , this detective has a clever 'storyteller' the Infiller Network that accurately guesses how the hand moved during that missing time, based on its previous and subsequent movements. 2. The 'Camera Detective' : This expert ignores the hands completely Instead, they focus on the static background objects in the video to figure out exactly where the camera itself is moving in the real world like a super-accurate GPS for your head . To make sure the camera's path is super precise, this detective carefully filters out any moving hands so they d