Crisp › Learn › AI video upscaling

What is AI video upscaling?

AI video upscaling increases a video's resolution by reconstructing detail that was never stored in the file, using a neural network trained on millions of real high-resolution frames. Traditional scaling can only stretch the pixels you already have. That difference — inventing plausible detail versus interpolating existing detail — is the whole subject.

The problem it solves

A 480p frame is 640×480, about 307,000 pixels. A 4K frame is 3840×2160, about 8.3 million. Going from one to the other means producing 27 pixels for every 1 you started with. There is no arithmetic that recovers information which was thrown away at capture or at encode time — the question is only what you put in the empty space.

Traditional scalers answer with a local average. Bicubic looks at the neighbouring pixels and fades between them; Lanczos does something more sophisticated with a windowed sinc filter, but the principle is the same. Both produce a soft, slightly blurry result, because a blur is the safest guess when you know nothing about what the image contains.

What the model does differently

A super-resolution model has been shown enormous numbers of image pairs: a high-quality frame, and a degraded version of that frame. Over training it learns the mapping. It learns what a real eyelash looks like at full resolution when it has been reduced to three grey pixels. It learns that a hard vertical edge in a low-res frame was probably a hard vertical edge and not a gradient.

So when it meets your footage it is not averaging. It is pattern-matching against everything it has seen and producing the most likely high-resolution frame. That is why the output has crisp edges and readable texture where bicubic gives you a smooth smear.

It is also why the output is a guess. A well-trained model guesses well enough that the result looks like a better camera was used. It is still a guess, and that matters in exactly one context: never use upscaled footage as evidence of what something actually looked like. Licence plates and faces are the classic trap — the model will happily produce a sharp, confident, wrong plate.

The four things it genuinely fixes

ProblemFixable?Why
Low resolutionYesThe core case. Detail is missing but structure survives.
Compression damageMostlyBlocking and banding are learnable artefacts with consistent shapes.
Sensor noise / grainYes, with careDenoise first or the model sharpens the noise into detail.
Out-of-focus or motion blurNoThe information is gone from every frame, not just compressed.

That last row is the one nobody wants to hear. If the camera missed focus, no upscaler recovers it — the model will sharpen the blur into a slightly crunchier blur. Same for motion blur from a slow shutter. Those are optical events, not encoding losses.

Frames are not photos, and that's the hard part

Upscaling one photo is a solved problem. Upscaling 1,500 consecutive frames is not, because each frame gets processed independently and the model's guesses differ slightly between them. A texture that the model renders one way on frame 200 and slightly differently on frame 201 produces shimmer — a crawling, boiling quality in flat areas that is instantly obvious in motion and completely invisible in a screenshot comparison.

This is why upscaler reviews built on still-frame comparisons are close to useless, and why temporal stability is the axis that separates good tools from demo-quality ones. Crisp's own evaluation harness measures it directly as an added_flicker score, comparing frame-to-frame variation in the output against the same measure in the source, so a tool that introduces shimmer cannot hide behind a nice still.

Cloud or on-device

Super-resolution models are small by modern standards — a few hundred megabytes — and run well on Apple Silicon's GPU. There is no technical reason a Mac needs to send footage to a server for this, and two good reasons not to: an hour of 4K is a slow upload, and client footage often shouldn't leave the building at all. Crisp runs the whole pipeline locally for both reasons.

The trade is real, though: cloud services can throw far larger models at a frame than a laptop can. For a difficult restoration, a heavy cloud model may simply produce a better frame. Local wins on privacy, on cost, and on anything you'd rather not queue for.

What to actually do with a bad clip

  1. Work out which problem you have. Pause on a still frame and zoom in: soft everywhere and no hard edges anywhere is focus, which is unfixable. Visible square blocks in dark or fast-moving areas is compression, which is fixable.
  2. Denoise before upscaling, never after. Upscaling noise makes the model treat grain as texture and amplify it.
  3. Pick 2x before 4x. If the source is 1080p, 2x to 4K is the same operation and less risky than pushing a 480p file straight to 4K.
  4. Judge the result in motion, at full size. Not on a paused frame, not in a small preview window — shimmer only appears when it plays.

Download Crisp for Mac