World model evaluation platform WorldOracle released

Recently, WorldOracle, the world's first action-driven world model evaluation platform that uses real execution as the benchmark, was officially released and opened evaluation submissions to developers. The project was jointly completed by researchers from AgiBot, NVIDIA, Princeton University, the University of Texas at Austin, and the University of Toronto. WorldOracle is mainly used to evaluate the accuracy of world models in predicting the outcomes of robot action execution. Unlike traditional evaluation methods that focus on video generation quality, this platform uses real robot operation data as a reference, comparing model-generated predicted videos with the actual execution process, with emphasis on testing the model's ability to predict the outcomes of physical interactions. Currently, WorldOracle has included 335 real operation trajectories, covering tasks such as fruit grasping, towel hanging, pouring water, and drawer opening and closing, of which about 40% of the interaction clips contain failed or suboptimal operations.