WoVR: World Models as Reliable Simulators for Post-Training VLA Policies with RL

<h1 id="technical" Technical Analysis</h1 Jump to Section: Motivation motivation | Results summary-of-results | Conclusions final-conclusions "WoVR" https://rlinf.readthedocs.io/en/latest/rst source/examples/embodied/wan.html is a reliable world-model-based reinforcement learning framework for post-training VLA policies. WoVR explicitly regulates how RL interacts with imperfect imagined dynamics. It improves rollout stability through a controllable action-conditioned video world model, reshapes imagined interaction to reduce effective error depth via Keyframe-Initialized Rollouts, and maintains policy–simulator alignment through World Model-Policy co-evolution. High-level Analogy : Imagine you're teaching a robot to cook by letting it practice in a 'virtual kitchen' a world model instead of a real one, because real kitchens are expensive and messy for trial-and-error. The problem is, this virtual kitchen isn't perfect; sometimes, if the robot tries to pick up a banana, the virtual kitchen might show it successfully picking it up, even if the banana realistically phased through its hand This is 'hallucination' – the virtual world looks fine, but the physics are wrong. If the robot keeps practicing this way, it learns to 'exploit' these glitches and becomes a bad cook in the real world. WoVR is like a smart system designed to make sure the robot learns properly despite the imperfect virtual kitchen: 1. A 'Smarter' Virtual Kitchen : They first make the virtual kitchen itself mor