Evaluating Gemini Robotics Policies in a Veo World Simulator

<h1 id="technical" Technical Analysis</h1 Jump to Section: Motivation motivation | Results summary-of-results | Conclusions final-conclusions <figure class="my-6" <video controls playsinline style="display: block; margin: 0 auto; max-width: 100%; height: auto;" src="$BUCKET URL/VEOWorldEvaluator/videos/post.mp4" </video </figure "Evaluating Gemini Robotics Policies in a Veo World Simulator" https://veo-robotics.github.io/ demonstrates that video models can be used for the entire spectrum of policy evaluation use cases in robotics: from assessing nominal performance to out-of-distribution OOD generalization, and probing physical and semantic safety. High-level Analogy : Imagine you're trying to teach a robot chef to cook. Testing it in a real kitchen is slow, expensive, and dangerous if it makes a mistake like trying to chop a laptop . Traditional virtual kitchens simulators are safer, but it's incredibly hard to make them look perfectly real, especially for squishy food or liquids. Plus, you have to manually design every new ingredient or kitchen layout. This research introduces a 'magic video-making AI' Veo as a super-realistic virtual kitchen. Instead of manually building 3D models, you tell this AI: 'Show me the robot picking up a red apple,' and it generates a photorealistic video of that. But the real magic is, you can also tell it: 'Now, show me the robot picking up a purple banana on a polka-dotted tablecloth next to a sleeping cat ,' and the AI instantly edits the sce