WoVR: World Models as Reliable Simulators for Post-Training VLA Policies with RL

Technical Analysis is a reliable world-model-based reinforcement learning framework for post-training VLA policies. WoVR explicitly regulates how RL interacts with imperfect imagined dynamics. It improves rollout stability through a controllable action-conditioned video world model, reshapes imagined interaction to reduce effective error depth via Keyframe-Initialized Rollouts, and maintains policy–simulator alignment through World Model-Policy co-evolution. High-level Analogy: Imagine you're teaching a robot to cook by letting it practice in a 'virtual kitchen' (a world model) instead of a real one, because real kitchens are expensive and messy for trial-and-error. The problem is, this virtual kitchen isn't perfect; sometimes, if the robot tries to pick up a banana, the virtual kitchen might show it successfully picking it up, even if the banana realistically phased through its hand! This is 'hallucination' – the virtual world looks fine, but the physics are wrong. If the robot keeps practicing this way, it learns to 'exploit' these glitches and becomes a bad cook in the real world. WoVR is like a smart system designed to make sure the robot learns properly despite the imperfect virtual kitchen: A 'Smarter' Virtual Kitchen: They first make the virtual kitchen itself more stable and responsive to the robot's actions, so objects don't randomly disappear or change shape as often. It’s like upgrading the virtual kitchen's physics engine. Focused Practice: Instead of always…