How Does AI Image Upscaling Work?

Classic resizing (bicubic or bilinear interpolation) computes each new pixel as a weighted average of its nearest original neighbors. It can only ever produce values that are a smooth blend of information already present, which is exactly why traditionally upscaled images look soft: no new detail was added, the existing detail was just spread across more pixels.

A super-resolution model does something categorically different: it's trained to predict the high-frequency detail, texture, sharp edges, fine grain, that a genuine high-resolution version of a scene would have, then generates that detail directly, rather than interpolating from what's already there.

Training needs paired examples: a high-resolution image and a low-resolution version of the same scene. Since collecting millions of genuinely matched real-world pairs at scale isn't practical, most models instead synthesize the low-resolution side by deliberately degrading real high-resolution images, downscaling, adding compression artifacts, blur, and noise, in ways designed to mimic what a real low-quality photo looks like.

Real-ESRGAN, a widely used open model, goes further and models several rounds of realistic degradation stacked together (blur, resize, noise, compression, repeated), because a single clean downscaling step doesn't reproduce the messy, compounded degradation real-world low-quality images actually have. The model is then trained to reverse exactly that degradation process, which is why upscalers trained mostly on synthetic degradations can still perform worse on real photos with unusual degradation patterns their training never simulated.

Many super-resolution models are trained adversarially: a second network, a discriminator, is trained alongside the upscaler specifically to tell generated detail apart from real detail, and the upscaler is pushed to fool it. This produces convincingly sharp, textured output, but the detail is generated, not recovered, so it can occasionally invent texture that wasn't in the original scene, a plausible-looking pattern on a surface that was actually blank, for instance.

That's the core tradeoff of generative upscaling: it can make a blurry photo look dramatically sharper than any interpolation method could, but 'sharper' doesn't mean 'more accurate to what the camera actually saw'; some of what appears is the model's best guess, not a recovery of lost information.