Is a diffusion model the same as a GAN?

No. A GAN (generative adversarial network) trains a generator against a discriminator and produces an image in one forward pass. A diffusion model trains a denoiser and builds the image over many steps. Diffusion has mostly replaced GANs for high-quality image generation because it trains more stably and covers a wider range of outputs.

Why do diffusion models sometimes get hands wrong?

A hand varies a lot from image to image: finger count on show, how they overlap, the pose. Any single configuration is thinly represented in training data compared with a face, which is more common and more consistent, so the denoiser has a harder time landing hands reliably.

What is the 'score function' in a diffusion model?

It's the mathematical object the network is really learning: at any noisy point in the space of all possible images, the score function is the direction that points toward more realistic, less noisy images. Predicting noise (DDPM's training target) and predicting the score turn out to be the same underlying computation, just phrased differently.

MODEL TYPE

What Is a Diffusion Model?

A diffusion model generates images (or video, or audio) by learning to reverse a process that slowly adds noise to data. Trained to undo that noise one step at a time, the same model can start from pure random noise and walk it back into a coherent image.

Walkthrough: one image followed from pure noise to a finished picture, one denoising step at a time.

What the video explains

From pure noise to a picture

You have probably seen an AI turn a sentence into a picture. To see how, the video follows one image from start to finish. It starts as pure noise: random static, like an old television with no signal, with no picture in it at all.

The model cleans it up a little at a time. First, rough blobs of color appear. Then shapes you can start to recognize, then the small details, and finally the finished image. A diffusion model is an AI that makes pictures this way: noise in, picture out, one small step at a time.

What happens inside one step

The same thing happens at every step. Each step starts with two things: a noisy image taken from somewhere along the walk, and your prompt, in this example "a dog on a hill." The words are turned into numbers so the model can read them, and both go into the denoiser.

The denoiser is a neural network with one job: to guess which part of the picture is noise. Then that noise is taken away, but only a small part of it, so the picture comes out a little cleaner. The cleaner image goes back in, and the denoiser guesses the noise again. The early rounds decide the big shapes, and the later rounds add the fine details. The first diffusion models needed about a thousand rounds, and today's models get there in twenty to fifty.

How the denoiser learned, and what to remember

During training, the denoiser was shown millions of real pictures. Each one was buried in noise, a little more at every step, until nothing was left, and the model practiced one thing: guessing the noise that had been added. Once it is good at that, it can walk the other way, from pure noise to a picture.

Three things to remember. A diffusion model is trained to remove noise, which is the one skill it learns. It works in many small steps, not in one big jump. And it is the same process for any prompt: the model does not change, only the words you give it.

The core idea: learn to denoise

Take a real image and add a little Gaussian noise, then a little more, over many steps, until it is pure static. Train the model to predict the noise added at each step so it can be subtracted back out. Run that over millions of images and the model learns what real images look like at every level of corruption, without anyone spelling that out.

Generation runs the process backwards. Start from pure random noise. Ask the model to predict and remove a bit of noise, then do it again, for dozens to a couple thousand steps, until an image is left.

Where the prompt comes in

Text-to-image conditions the denoising on a text embedding. At each step the model does not predict generic noise. It predicts the noise removal that pushes the result toward the prompt's embedding. That conditioning is what turns a model that generates any plausible image into one you can steer with words.

Most current systems run diffusion in a compressed latent space instead of on raw pixels, an idea from the latent diffusion work. That is a big part of why generation got fast enough to ship.

Why so many steps, and how that got faster

Early diffusion models needed around 1,000 denoising steps for a good image. Too slow to use. Newer sampling methods and distillation cut that to tens of steps, and in some turbo and few-step models down to one, giving up some quality and diversity for a large jump in speed.

DDPM, the 2020 paper that made diffusion models practical, trains the network to predict the exact noise added at a given step rather than trying to predict a slightly cleaner image directly. Predicting the noise turns out to be mathematically equivalent to predicting the direction back toward the real data, but it trains with far lower variance, which is a big part of why diffusion training scales.

A newer framing (Song et al.'s score-based formulation) treats this as continuous rather than step-by-step: the network learns a score function, a vector field defined at every point in the space of possible images, that always points toward higher data density. Diffusion Denoising Implicit Models (DDIM) showed that following this field deterministically, with no randomness at each step, still lands on the same overall distribution of results, and that a deterministic path can be walked in much larger jumps. That's the mechanism behind cutting 1,000 steps down to 20 to 50 without retraining the model, and it's the theoretical basis modern flow-based generators build on.

The random noise DDPM adds back in, and why

One counterintuitive detail in the original DDPM sampling process: at each step, after the network's prediction moves the image toward a cleaner state, the process adds a small amount of fresh random noise back in before the next step. Skipping that step, and always taking the network's single best guess, seems like it should give a better result. It doesn't.

The network predicts the mean of a range of plausible less-noisy images, not the one true answer, because many different real images are consistent with the same noisy input. Removing the added randomness collapses every generation onto that mean, and the mean of many different plausible images is a blurry average of them, not a sharp result. The added noise at each step is what lets the process land on one specific, detailed point in that range instead of their blended average.

FAQ

Is a diffusion model the same as a GAN?: No. A GAN (generative adversarial network) trains a generator against a discriminator and produces an image in one forward pass. A diffusion model trains a denoiser and builds the image over many steps. Diffusion has mostly replaced GANs for high-quality image generation because it trains more stably and covers a wider range of outputs.
Why do diffusion models sometimes get hands wrong?: A hand varies a lot from image to image: finger count on show, how they overlap, the pose. Any single configuration is thinly represented in training data compared with a face, which is more common and more consistent, so the denoiser has a harder time landing hands reliably.
What is the 'score function' in a diffusion model?: It's the mathematical object the network is really learning: at any noisy point in the space of all possible images, the score function is the direction that points toward more realistic, less noisy images. Predicting noise (DDPM's training target) and predicting the score turn out to be the same underlying computation, just phrased differently.

Sources

Last updated 2026-10-05