How Does AI Image Generation Work?

A diffusion model never draws. It denoises. During training, the model is shown real images with Gaussian noise added on top, a little at first, then more, over many steps, until the image is indistinguishable from static. It is trained to predict exactly which noise was added at each step, so that noise can be subtracted back out.

Do that over millions of images and the model learns, implicitly, what real images look like at every level of corruption. Nobody labels edges, shapes, or objects for it. The one task, guess the noise, is enough for the structure of real images to fall out of training.

Generation runs the same process in reverse. It starts from a canvas of pure random noise and repeatedly asks the model to predict and remove a small amount of it. After enough steps, a coherent image remains where there was only static.

An unconditioned diffusion model produces a plausible image with no control over what it depicts. Text-to-image adds a second input at every denoising step: an embedding of your prompt, produced by a text encoder trained to place text and images with matching meaning near each other in the same numeric space (see what CLIP is for how that space gets built).

At each step, the model does not predict generic noise. It predicts the noise removal that pushes the image toward whatever your prompt's embedding represents. On its own, that pull is often too weak: tested directly, plain conditioning can produce a technically correct scene that leaves out the specific subject the prompt named, because the model's general sense of a plausible image competes with the comparatively subtle pull of the prompt. Classifier-free guidance (CFG) fixes this by running the prediction twice, once with the prompt and once without, and pushing the result further in the direction the prompt adds, which is why turning guidance strength up makes an image match the prompt more literally, at some cost to variety. See what classifier-free guidance is for the mechanism and why negative prompts use the same trick.

This is also why wording changes results: the prompt only ever acts through its embedding, so two prompts that a person reads as similar but that encode differently can steer the model toward different images.

Running the denoising process directly on a full-resolution image is expensive: every step touches every pixel. Latent diffusion, the approach behind Stable Diffusion and most current systems, first compresses the image into a much smaller grid of numbers using a separately trained autoencoder, then runs the entire denoising process on that compressed representation.

Once denoising finishes, a decoder expands the compressed result back into a full-resolution image. The compression is lossy but trained specifically to preserve what matters for realistic images, so the loss doesn't show up as visible degradation. Working in this smaller space is a large part of why current image generators are fast enough to run in a browser tab instead of taking minutes per image.

The original diffusion papers needed around 1,000 denoising steps to reach a clean image, far too slow for interactive use. Later sampling methods (DDIM and its successors) reformulated the same reverse process to skip steps without retraining the model, cutting that to 20 to 50 steps for most tools today.

Distillation pushes further: a second model is trained to mimic several steps of the original model's denoising in one jump. Few-step and one-step distilled models can generate an image in a single pass, trading some fine detail and prompt-following precision for near-instant results.