Behaviour cloning teaches a robot a task by supervised learning on demonstrations: the policy sees what the demonstrator saw and learns to output the action the demonstrator took. Most current VLA models are fine-tuned this way.
Set-Up
- record demonstrations by teleoperation: at each step, store the observation (images, robot state, instruction) and the action
- train a policy to predict from
- is mean squared error for simple regression, or a generative loss such as diffusion or flow matching
Main Weakness: Compounding Error
- the policy makes a small mistake and reaches a state slightly unlike any demonstration
- there it has no good examples, so it makes a bigger mistake
- errors grow over the episode, which is called covariate shift or distribution shift
- for a task of length , the expected error can grow roughly with instead of
Ways To Reduce It
- more and more varied demonstrations, including small recoveries from mistakes
- action chunking: predict several future actions at once, so the policy commits to smooth motion and makes fewer decisions per second
- DAgger: run the policy, ask the expert what they would do in the states the policy visits, add those labels, retrain
- constrain the task area first (dense demonstrations over a small area work better than thin ones over a large area), then widen it
Multimodal Demonstrations
- if demonstrators sometimes go left round an obstacle and sometimes right, MSE regression averages them and goes straight through
- generative policies (diffusion, flow matching) can output one of the valid options instead
Practical Points
- keep the demonstrator's hands out of the camera view (teleoperate with a joystick or 3D mouse instead of pushing the arm)
- record at a fixed rate and check it, because the policy learns that rate
- validate every episode (equal lengths, increasing timestamps, same image size) before training