A generative model is given a training set of samples drawn from some real distribution p_data, and it learns to represent an estimate of that distribution, written p_model. Once it has that estimate, it can draw brand new samples that were never in the training set.
What the word estimate means here
- the model never sees
p_dataitself, it only sees samples from it - what it learns is an approximation of the shape of that distribution
- generation is then just sampling from the approximation
Three families
- Variational Autoencoder (VAE) learns a probabilistic encoder and decoder, and samples from a tidy latent space
- Generative Adversarial Networks learn a generator by competing against a discriminator
- diffusion models learn to reverse a noising process, so they start from noise and denoise