"World Model" – a model of how a world evolves in response to agents' actions – has been long explored by robotics practitioners. Various perspectives of world modeling have been studied through the lenses of model-based optimal control, reinforcement learning, controllable video generation, dynamic 3D reconstructions, and so on. And it creates significant impacts on the recent development of dexterous manipulation, locomotion, and long-horizon navigation. In particular, powered by large data, the recent advances in the generality and precision of learning-based world models – video generation models and differentiable simulators – show tremendous opportunities in transforming robot learning and optimal control.
The goal of this workshop is to create a space for the robot community to discuss different perspectives of world modeling, as well as the growing impact on robotics. The full-day workshop will bring together researchers and practitioners in robot learning, physics-based modeling, video generation and machine learning to explore the intersection of world modeling and robotic systems. We aim to create a mixture of traditional physics-based approaches and modern learning-based methods, with a focus on building more robust and generalizable robotic systems.
We will focus on several key challenges:
- How can we build world models that capture both visual and physical understanding?
- How can we leverage large-scale pre-trained models for robotics applications?
- How can we combine learning-based approaches with physics-based priors?
- How can we evaluate and benchmark world models for robotics tasks?
By inviting leading experts in robot learning, physics simulation, and video generation models, we hope to spark novel ideas to help advance world modeling and robotics.