AI video upscaling for film.

What upscaling actually does

There are two ways to make a small picture bigger. The first is to stretch it: every pixel becomes four, the edges go soft, and the result looks like what it is, a small picture seen too close. The second is to redraw it. A learned upscaler has seen a great many pairs of small and large images and has a model of what skin, hair, brick, cloth and type look like at the larger scale. Given a frame, it draws the frame again at the target size, keeping the structure it can see and filling in texture it can only infer. Edges come back crisp, grain comes back as grain rather than blocks, and a surface that was a smear becomes a surface again.

Video adds a problem a still does not have: the next frame. Redraw each frame on its own and the inferred texture changes slightly from one to the next, which the eye reads as shimmer — a wall that boils, hair that crawls. A video upscaler worth the name therefore looks across neighbouring frames, borrows detail that persists between them, and holds its inventions steady over time. That temporal pass is most of the difference between a clip that looks sharper and a clip that looks upscaled, and it is also why the job takes as long as it does: the engine is reading a window of frames for every frame it writes.

It is worth being precise about the word recover. Compression artefacts, under-sampling and the softness of a low render resolution all hide detail that is partly still there, and an upscaler does recover some of it. Detail that was never in the frame — the pattern on a tie the camera could not resolve, the lettering on a sign — is not recovered. It is proposed. The result can be excellent, and it should be looked at as a proposal, which on a production means checking it against what the frame is supposed to contain.

What it can and cannot recover

It can take a take rendered at a modest size and make it sit beside camera footage without announcing itself. It can clean the macroblocking out of a clip that has been through a messaging app. It can sharpen edges that are soft because of sampling rather than because of optics, rebuild fine texture on surfaces whose structure is visible, and keep grain looking like grain. For previs, where the point is to show a client or a crew what a shot is rather than to deliver it, this is usually all that is needed, and it is the difference between a sequence that reads as a plan and one that reads as a proof of concept.

It cannot bring a shot into focus. Optical blur throws away the high frequencies, and an upscaler asked to sharpen it will draw edges where it guesses they were. It cannot remove motion blur, which is the same loss along a direction. It cannot lift detail out of shadows that were crushed to black or highlights that were clipped to white. It cannot read text that was illegible; it will produce letter-shaped marks, which are worse than a blur because they look like information. And it cannot know who a person is. A face that occupies forty pixels in the source will come out as a sharp, convincing face, and it will be a face the engine chose — which, on a production that has locked its cast, is a continuity question before it is a resolution question.

What goes wrong is almost always one of three things. The first is upscaling the pile instead of the select: every take gets sent up before the cut is decided, and the production pays, per second, to sharpen clips it will never use. Upscale after the edit, not before. The second is upscaling as a substitute for rendering: a take that is wrong in composition or performance goes up a rung and comes back as a sharper version of the wrong take. The third is over-shooting the target — an 8K master for a web deliverable — which costs time and credits and gives the colourist nothing to grade that a 4K would not. Decide the delivery first, upscale to it, and stop.

What a good output looks like, read frame by frame: edges that are sharp without a halo; texture that stays put from one frame to the next when the camera is still; grain that is even; no new detail in places you know were plain; and faces, where there are faces, that still read as the people on the board. If any of those fails, the fix is usually one rung lower or a cleaner source, not a different setting.

Where PREVIZ does it, and what it does not

Post is the finishing room. A take from Director or the board, or a clip brought in from outside, goes in with a target rung; the price is quoted per second of footage on the control before anything runs, like every spend on the platform, and plans differ by credits, not by whether the room is open. Progress reads out per frame while the job runs, and a finished clip can be sent up a further rung rather than re-processed from the source, so a small clip climbs in modest steps. A hero still goes a different way: Keyframe carries its own progressive chain for frames bound for print, which the treatment and pitch deck page touches on.

What Post does not do is restoration. It will not stabilise a shot, retime it, denoise it as a separate pass, or repair a frame that is damaged rather than small; those are post-production jobs for post-production tools. It does not replace shooting at the resolution the deliverable needs, and it does not make a previs take into a finished film. It takes the clip you would deliver as it is and makes it the size it needs to be, and it keeps the record of what was done to it, which is the point of doing it inside the production rather than in a separate app. The workflow overview places it as the last step of the delivery stage, and the rooms page describes the six studios the take came through before it got there; the words along the way are in the glossary.

Questions people ask