RoboSimStudio

An open interactive toolkit that accelerates human and agentic
physics-based Real2Sim of Robot Manipulation Tasks

Kaiyuan Chen1*, Shuangyu Xie1*, Uksang Yoo1, Xinyue Zhu1, Cael Magner1, Alvin Zhu1, Daehwa Kim1, Eric Heiden2, Erwin Coumans2, Miles Macklin2, Spencer Huang2, Ken Goldberg1

AUTOLab UC Berkeley NVIDIA
Paper Code Interactive demo ðŸ”¥
@wipe_the_bar.mov
PROMPT
CODING AGENT · 60×
NEWTON · PATH TRACED
MULTIPHYSICS · FEM SPONGE

Connecting…

A smartphone video of the real task, and a one-line prompt to a coding agent.

The agent builds the scene, matches the camera and tunes the physics to the clip. About an hour of work, shown at 60×.

The resulting simulation in Newton: path traced on the left, the multiphysics view with the FEM sponge on the right.

The same simulation, live on a GPU. Drag the orange ball to move the arm, drag empty space to orbit. Pick another task from the snapshots.

Live simulation Open in a tab ↗