TidyBot-Universe: A Modular and Service-Oriented Framework for Robotic Tidy-Up Tasks

Technical Analysis High-level Analogy: Imagine you have a messy room, and you want a robot to tidy it up. Instead of having one giant, rigid robot that's hard to change, think of 'TidyBot-Universe' as building a specialized 'clean-up crew' out of LEGO blocks. You have different specialized LEGO blocks (called 'services'): one block is great at seeing objects (TidyBot-Vision), another knows how to pick up and place things (TidyBot-Skillset), and a basic movement block (TidyBot-Core). The most important 'block' is like a very smart team leader (TidyBot-LLM). This leader understands your high-level instructions, like 'Put all the books on the shelf and the toys in the box.' The leader then figures out the step-by-step plan, telling the 'vision' block to find a book, then telling the 'skillset' block to pick it up, and so on, coordinating all the other specialized blocks to get the job done. This system makes it easy to swap in new, better blocks as technology improves, or to build a slightly different 'crew' for a different tidying task, without rebuilding everything from scratch. Motivation of the Work Currently, robots can perform many manipulation tasks, like picking up objects and placing them elsewhere (often called 'pick-and-place'). Advanced vision systems help them identify objects, and sophisticated planning algorithms guide their movements. Recent breakthroughs in Large Language Models (LLMs) allow robots to understand more complex human instructions and even 'reason'…