Overview

  • Sim2Real means training in a simulator, then using the agent in the real world
  • Sim2Sim is a safe first test that moves from a simple simulator to a more realistic one (SplitNet example)
  • for robot arms, domain randomization trains under many random changes so the real world looks like one more variation

Sim2Real transfer is the process of training an agent in a simulator and then using it in the real world. Simulation is safe, cheap and fast, but the real world looks and behaves differently, so the agent must be made robust to that gap.

Simulation-based pretraining

The pipeline is: an environment (a 3D scene) is loaded into a simulator, a task is defined, the agent is trained, and the result is transferred to real-world robotics.

Sim2Sim

Sim2Sim transfers from a lower-fidelity simulator to a higher-fidelity one. It is a safe test of whether an agent can transfer at all.

In the SplitNet example:

  • the initial training phase uses SUNCG scenes with the task PointNav, goal "go to (2,5)"
  • Sim2Sim transfer keeps the task (PointNav) but moves to Matterport3D (MP3D) scenes, goal "go to (10,8)"
  • Task2Task transfer keeps the scenes (SUNCG) but changes the task to Explore, with the goal "maximise visited locations"

Related work: Sim-Real Joint Reinforcement Transfer for 3D Indoor Navigation, and SplitNet (Sim2Sim and Task2Task transfer for embodied visual navigation).

Sim2Real for navigation

Sim2Real moves the agent to the real world.

The robot uses its first-person view in a real building, while a top view shows the start, the door and the goal. Related work: Cognitive Mapping and Planning for Visual Navigation, and Combining Optimal Control and Learning for Visual Navigation in Novel Environments.

Sim2Real for visual actuation

For robot arms that reach for objects (visual servoing), the main tool is domain randomization: train in simulation under many random variations, so that the real world looks like just one more variation.

Learning view-invariant control in simulation uses:

  • various viewpoints of the camera
  • various objects
  • simulation randomization of object type, object location, texture (of object, table and plane) and lighting

The trained policy then works on a real robot arm from the same viewpoint with various object locations (Sim2Real View Invariant Visual Servoing by Recurrent Control).

Note

Sim2Real is one way to get robot training data. Another way is to train one large model on many real robot datasets, which is the approach of OpenVLA.