A diffusion policy is a diffusion model that generates robot actions instead of pixels. The denoiser is conditioned on the current observation, and what it denoises is a short sequence of future actions.
Why diffusion suits actions
- an action sequence is continuous and multimodal, because more than one motion can be correct for the same scene
- a single regression output averages those motions together and produces something that works for neither
- a diffusion model samples one coherent mode instead of the average
Imitation learning
- the training signal is human demonstrations, and not a reward
- the policy copies what the demonstrator did in each observed situation
- this avoids designing a reward function for a contact-rich task, which is the part traditional control finds hardest
Large behavioral models
- the same recipe is scaled to many tasks and many robots, giving one policy that covers a broad skill set
- the appeal matches the language model story, so broad pretraining first and narrow adaptation second
The general action-and-observation model this sits inside is VLA.