Zero-WAM: In-Context World-Action Modeling from Human Videos for Open-Ended Task Generalization

Technical Analysis is a causal video-action model that executes unseen tasks by following in-context human video guidance. High-level Analogy: Imagine you have a super-smart robot chef, but it's only really good at recipes it's practiced many times. If you give it a brand-new, complex recipe in text, it might struggle because words don't always convey the subtle 'feel' or visual cues of cooking. Now, imagine you show the robot a cooking video of a human making that exact new dish. The robot watches the human's hands, the ingredients changing, and the final look. Zero-WAM is like this robot chef: it learns to understand and perform entirely new tasks by watching human demonstration videos, even if it has never seen that specific task before. The 'shortcut' problem is like the robot, when practicing familiar recipes, sometimes just reading the first few steps and guessing the rest because it's 'seen it before'. Zero-WAM's special training (IFP) forces it to really pay attention to the whole video for every task, so it doesn't try to guess or use shortcuts when facing a truly new dish. It learns to extract the 'essence' of the task from the human video and apply it to its own robotic hands, even if the human's kitchen looks different from its own. Motivation of the Work The primary motivation is to overcome the limitations of current robot learning systems to achieve true 'open-ended task generalization' – allowing robots to learn and execute an unlimited range of new tasks…