MODEL TYPE
What Is a Diffusion Model?
A diffusion model generates images (or video, or audio) by learning to reverse a process that slowly adds noise to data. Trained to undo that noise one step at a time, the same model can start from pure random noise and walk it back into a coherent image.
The core idea: learn to denoise
Take a real image and add a little Gaussian noise, then a little more, over many steps, until it is pure static. Train the model to predict the noise added at each step so it can be subtracted back out. Run that over millions of images and the model learns what real images look like at every level of corruption, without anyone spelling that out.
Generation runs the process backwards. Start from pure random noise. Ask the model to predict and remove a bit of noise, then do it again, for dozens to a couple thousand steps, until an image is left.
Where the prompt comes in
Text-to-image conditions the denoising on a text embedding. At each step the model does not predict generic noise. It predicts the noise removal that pushes the result toward the prompt's embedding. That conditioning is what turns a model that generates any plausible image into one you can steer with words.
Most current systems run diffusion in a compressed latent space instead of on raw pixels, an idea from the latent diffusion work. That is a big part of why generation got fast enough to ship.
Why so many steps, and how that got faster
Early diffusion models needed around 1,000 denoising steps for a good image. Too slow to use. Newer sampling methods and distillation cut that to tens of steps, and in some turbo and few-step models down to one, giving up some quality and diversity for a large jump in speed.
FAQ
- Is a diffusion model the same as a GAN?
- No. A GAN (generative adversarial network) trains a generator against a discriminator and produces an image in one forward pass. A diffusion model trains a denoiser and builds the image over many steps. Diffusion has mostly replaced GANs for high-quality image generation because it trains more stably and covers a wider range of outputs.
- Why do diffusion models sometimes get hands wrong?
- A hand varies a lot from image to image: finger count on show, how they overlap, the pose. Any single configuration is thinly represented in training data compared with a face, which is more common and more consistent, so the denoiser has a harder time landing hands reliably.
Sources
Related
Last updated 2026-09-06