How does CGI rendering work?

A computer-made picture starts with a virtual camera: a point for the eye and, just in front of it, a flat image plane cut into a grid of pixels. Your screen is the same grid, only much finer.

To colour one pixel, the computer draws a straight line, a ray, from the eye through the middle of that pixel and out into the 3D scene. It works out the first thing the ray hits: the floor, a ball, the block. That thing gives the pixel its colour. Do this for every pixel and you have a picture. This is ray casting, first described by Arthur Appel at IBM in 1968.

From the hit point, a second ray heads for the lamp. If it gets there, the spot is lit; if something is in the way, it is in shadow. Let rays bounce off the mirror ball too and you have ray tracing. A full-HD frame is 1920 × 1080 pixels, so it needs over 2 million rays before any bouncing. Where the scene's shapes come from, the modelling and rigging, is in Anim3DClear.

There are two big ways to turn a 3D scene into pixels, and the monitor draws the same scene both ways, timed on your device.

Rasterising is how games draw. Every object is first cut into flat triangles. The three corners of each triangle are projected onto the screen, and the pixels inside are filled in. Triangles arrive in any order, so a z-buffer remembers how far away each pixel's surface is. A new triangle only paints a pixel if it is nearer. Ed Catmull described this in 1974, and Wolfgang Straßer had the same idea that year. Graphics chips (GPUs) do billions of these triangle-pixel tests a second.

But a triangle is drawn on its own, with no idea what else is in the scene. So a rasteriser gets no true shadows or reflections. Games fake them with clever tricks, like shadow maps and pre-baked reflections.

Ray tracing works the other way round: one ray per pixel, plus rays to the lamp and off mirrors. Shadows and reflections come straight from the physics. It is slower, which is why films, which can wait minutes for a frame, used it first, while a game at 60 frames a second has just 16.7 ms per frame.

To be fair, both sides here use exactly the same triangles. The ray tracer first checks a bubble around each object before testing its triangles; real renderers use a whole tree of such boxes, a BVH, to skip most triangles.

Real light does not stop at the first surface. It bounces: off the floor, onto the walls, into the shadows. That is why a room lit by one lamp is not pitch black in the corners, and why a white wall next to a red sofa looks a little pink. Renderers call this global illumination.

A path tracer copies nature with dice. For each pixel it follows one random path: from the camera to a surface, then off in a random direction, and so on, bounce after bounce. At every stop it also checks the lamp. One path is a wild guess, so the picture is speckled with noise. Average many paths, called samples per pixel, and the guesses settle on the right answer. Jim Kajiya wrote down the maths, the rendering equation, in 1986.

The catch: noise only falls as 1/√N. Four times the samples gives half the noise, so a clean frame takes hundreds or thousands of samples. That is why films now also use denoisers, smart filters, many of them trained by machine learning, that clean up a noisy frame. Our room is a cousin of the famous Cornell box from 1984. How real lamps are placed on set is in LightingClear, and how heat radiates the same way light does is in HeatClear.

When a ray hits a surface, the renderer asks the shader: what happens to the light now? The answer is a rule called a BRDF, which says how much light arriving from one direction leaves in each other direction. The board draws it as a lobe.

A matte surface (the Lambert rule) scatters light evenly everywhere. A mirror sends it one way only. Most things are in between: the roughness slider widens a shiny lobe from a sharp mirror to a soft sheen.

Glass splits light. Some bounces off, and the rest goes in and refracts, bending by an amount set by its index of refraction; SnellClear shows the bending. How much reflects follows the Fresnel equations: straight on, glass reflects only about 4%, but at a glancing angle nearly all of it. That is why a lake is a mirror near the horizon and clear at your feet.

Subsurface scattering is for skin, wax, milk and jade: light slips inside, bounces around, and leaks out nearby, giving a soft glow. Films use it for every face. Lighting a real set is in LightingClear.

A film runs at 24 frames a second, so a 90-minute feature is 129,600 frames. If each one takes hours to render, one computer would need centuries. So studios build a render farm: racks of computers, each rendering different frames at the same time.

Toy Story (1995), the first fully computer-animated feature, had 114,240 frames and took about 800,000 machine-hours, on a farm usually given as 117 Sun computers. For Monsters University (2013), Pixar reportedly averaged about 29 hours per frame on about 2,000 computers with 24,000 cores. Frames got faster, but films asked for ever richer light, so render times stayed long. Today studios also rent machines in the cloud when a deadline looms.

A renderer does not only save the finished picture. It also writes passes (also called AOVs): the diffuse light, the shiny reflections, the shadows, the depth and more, each as its own image. Artists can then brighten a reflection or tint a shadow without rendering again. Putting passes and live footage together is compositing, the subject of VFXClear.

For decades, ray tracing was for films and rasterising was for games. That changed in 2018, when Nvidia launched RTX graphics cards with special RT cores that test rays against triangles in hardware. Nvidia claimed about 10 billion rays a second, roughly ten times its previous card.

Ten billion sounds endless, but a 1080p screen at 60 frames a second has 124 million pixels a second to fill. With a few rays per light path, that leaves only a handful of samples per pixel, far from the hundreds a film uses. So games take one noisy sample per pixel, blend it with the last few frames (temporal accumulation) and run a denoiser, often a neural network, to clean it up. Try it on the monitor: this page is doing it for real, just much slower.

Films borrow the other way too. Game engines now drive virtual production: giant LED walls show a live, engine-rendered world behind the actors, as on The Mandalorian (2019). That is in VirtualProdClear. Offline renders are still richer, but the gap between film and game pictures is narrowing every year.