Why upscaling to 4K takes so long

People are often surprised that upscaling a short clip can take longer than the clip itself by a large multiple. It is worth understanding where the time goes, because some of it is avoidable and some of it really is not.

Every frame is a separate job

A normal video export re-encodes frames, which modern hardware does in dedicated silicon at extraordinary speed. AI upscaling is a different kind of pipeline. Inside Crisp a render goes probe → extract → upscale → encode → concat → mux: the extract step decodes your video into numbered still images on disk, and the upscale step runs the model binary over that directory. The progress bar you watch is the count of output files that exist so far.

So a ten minute clip is not one job. At 30fps it is eighteen thousand of them, each a full network pass over a whole image. The model has no memory of the previous frame and gets no discount for the frame being similar to it.

The arithmetic, and the number we will not print

A ten minute clip at 30fps is eighteen thousand frames. Longer sources get worse fast: one real render we have on record took a 36.7-minute source and decoded it to 66,120 frames. That figure is just length times frame rate, and it multiplies everything else.

Is the per-frame cost the whole cost, or is there a fixed setup fee a long job would amortise? On Crisp's Max lane we timed three run lengths — 5 frames in 85.3 s, 10 in 206.9 s, 60 in 1348 s. Fit a line through those and the fixed cost comes out negative: the per-frame term explains all of it. We did not record which machine those runs were made on, so read them as a shape rather than a speed to expect. Max is also a far slower engine than the ordinary Enhance path; the point here is the linearity, not the rate.

That is also why this page gives you no minutes-per-hour figure. Crisp's own estimator carries the reason in its source: seconds per frame varies by output resolution and by machine by more than an order of magnitude, so a baked-in number would be wrong for most people. The Max lane projects from the rate it observes on your own footage instead.

Where the rest of the time goes

Decoding, the colour chain and writing frames out are real costs, though smaller. At 4K the model is producing roughly eight million pixels per frame and the encoder is compressing eight million per frame, four times a 1080p export of the same length. Pixel count scales with area, so doubling each dimension quadruples the work — the most counter-intuitive thing about the maths, and the reason "just a bit bigger" never is.

Memory matters more than people expect. Large frames and large models compete for one pool, and when it is tight the work is broken into tiles and reassembled, costing time and a little quality at the seams. Crisp does not guess a tile size: it uses the binary's own automatic behaviour unless a measured profile exists for your machine, because a value that helps a 64GB Mac Studio can starve a 16GB Air.

And the bundled networks are natively 2×, 3× and 4× only. Beyond that Crisp composes real passes — 8× is a 4× pass then a 2× pass — so each pass runs the whole model over every frame again, the second on images the first already enlarged.

A bigger scale factor does not always mean a longer wait

We timed four routes over 60 frames of the same 720p clip, rebuilt two ways: from 320×180 at 4×, and from 640×360 at 2×. Both land back on exactly 1280×720. These seconds are for 60 frames at very small sizes and we have not recorded the machine they were measured on — read the ratios, not the absolute times.

The 4× route finished faster than the 2× route. We are not going to tell you that 4× is quicker than 2×, because this measurement cannot show that. The two runs used different models: Crisp picks the model from the denoise value and the scale together, and it will not run the detail net at 2× or the compact net at 4×, so scale and model cannot be separated without a code change. What the pair does establish is that here the scale factor was not what decided the wall clock — worth knowing, because it is the assumption everyone starts from. The denoise dial is the control that swaps the model, and it is a bigger lever than it looks.

The quality columns are there because "how long" is only half the question. Against a plain lanczos resize the AI route cost six to nine times the wall clock here, and at 2× it also scored 23.4 VMAF points below that resize — at 4× it was within 1.2. Unflattering about our own default, and what the measurement says: the harder the reconstruction, the more the model earns its cost.

What genuinely makes it faster

Trim first. The single biggest lever, and the most ignored. If you only need thirty seconds of a ten minute clip, cutting before upscaling removes 95% of the work. Crisp applies a trim to the source before frames are extracted, so the frames you cut are never decoded and never reach the model.

Pick the target you actually need. 1080p to 4K is four times the output pixels of 1080p to 1440p, and on a phone screen the larger target may be invisible. Crisp's presets clamp toward the size on the label rather than multiplying blindly, so "Enhance → 4K" on a 1080p source writes 3840×2160 and not the 7680×4320 the raw factor would give.

Do not stack lanes you do not need. Smooth motion runs frame interpolation after the upscale, on the already-enlarged frames, so doubling the frame rate means generating roughly as many new 4K frames as you started with — a second model over a bigger picture, on top of the part that was already long.

Let it run unattended. Because everything is on-device there is no per-minute billing and no upload, so a long job overnight costs nothing but electricity. That changes the calculation compared with cloud services, where the incentive is to keep jobs short.

What does not help

Closing other applications rarely makes a measurable difference unless you were genuinely short of memory, and a higher bitrate affects file size rather than the inference that dominates the time. Nor does running a cheap job first to "warm things up": Crisp already pays that cost at launch, so that Vulkan initialisation, shader compilation and paging the model in happen off the critical path. No frame is ever cached, so a second run of the same clip costs the same as the first.

Render time, in detail

Is it normal for a ten minute clip to take over an hour?

For AI upscaling to 4K, yes. Crisp decodes the clip to individual frames and runs the model over each one, and a ten minute clip at 30fps is eighteen thousand frames. The cost is per frame rather than per file, so the total scales with length and with output resolution, and there is no shortcut that preserves the quality.

Does a faster Mac help?

It should, because the upscaler is an ncnn/Vulkan binary that runs on your Mac's GPU through Metal, so GPU speed is what most of the wall clock is waiting on. Memory matters too, since tight memory forces the frame to be processed in tiles. We have not run the same clip across several Macs, so we cannot honestly give you a ratio between two machines.

Why is the first frame slower than the rest?

Before any frame can be processed, Vulkan has to initialise, shaders have to compile and the model has to be paged in. Crisp pays that cold start at launch rather than on your first job. After that the per-frame cost settles into a steady rate, so a progress estimate taken from the first few seconds tends to read pessimistically.

Can I use my Mac while it runs?

Yes, though heavy GPU work will compete with the render and slow both. Inside Crisp only one GPU job runs at a time — deliberately, because two simultaneous GPU processes is a crash rather than a slowdown — so clicking Preview during a render waits for it or tells you the app is busy. Because nothing is uploaded, leaving a long job running overnight costs nothing beyond power.

Related

• What the denoise setting does: it swaps the model
• What is AI video upscaling?
• Why AI upscaling can't rescue a shot that missed focus
• Why phone video looks shaky
• Why VHS tapes degrade

Download Crisp for Mac Free to try, one-time $129 to remove the watermark. Runs entirely on your Mac.