How do visual effects (VFX) work?

Visual effects (VFX) are anything added to or changed in a shot after it was filmed. Effects done for real on set, like rain machines, fake glass and squibs, are special effects (see SFXClear). VFX start from the real footage, called the plate, and add things that were never there.

A big shot passes through a whole pipeline of departments. Matchmove recovers the real camera's movement. Layout and animation place and move the CG models. FX simulates smoke, water and fire. Lighting gives the CG the same light as the set and renders it (see CGIClear for how rendering works). Last, compositing stacks every layer into one picture.

Blockbusters have thousands of these shots. Baahubali 2 (2017) had close to 2,500 VFX shots made by about 35 studios, and Avengers: Endgame (2019) had 2,496. RRR (2022) had about 2,800. Almost every modern film has some VFX, even quiet dramas: removed signs, extra crowds, a cleaner sky.

Most VFX shots start with actors in front of a big, evenly lit wall of one colour: a green screen (or blue screen). The colour is picked because almost nothing else in the shot, least of all skin, has that much pure green or blue in it.

A keyer goes pixel by pixel. It asks: how much more green is this pixel than its red and blue? Compare that with the bare screen and you get a number from 0 (actor) to 1 (screen). That number becomes the matte, or alpha: white where we keep the picture, black where the new background shows through. Our keyer is a real colour-difference key running on your graphics card, the same idea Petro Vlahos patented for blue screens in the 1960s.

Real shots fight three problems. Spill: the bright screen bounces green light onto the actor's back and hair, so an actor 60 cm from the screen gets far more than one 3 m away. Motion blur: a fast hand is smeared half green, half skin, so the key must be soft there, not hard. Costumes: a green scarf in front of a green screen simply disappears. That is why a film about a green-clad hero is shot on blue.

Why is green the usual choice today? Digital sensors have two green pixels for every red and blue one (the Bayer pattern), and green carries about 72% of the brightness we see. So the green channel is the cleanest and needs the least light. Blue still wins for night scenes, for blond hair and for green costumes, and its spill is easier to hide. The explosions and rain that were really on set are in SFXClear.

When the camera moves, a CG object has to move exactly with the real world, or it slides and floats. A difference of one pixel is enough for your eye to notice. So before anything is added, someone rebuilds the real camera. This is matchmove.

First comes 2D tracking. The crew stick tracking markers (small crosses) around the set, and software follows each one from frame to frame, like following a freckle through a video. Each marker becomes a track: a path across the picture.

Then the camera solve. Near markers slide across the frame faster than far ones (parallax). From how all the tracks move together, the solver works out where each marker sits in 3D and where the camera was on every frame. It needs at least five to eight good tracks, and more is steadier. The result is a virtual camera that copies the real one, and CG is rendered through it.

The solver checks itself with the reprojection error: put the solved 3D points back through the solved camera and see how far they land from the real tracks. Under about half a pixel is a good solve. Actors' bodies are tracked the same way for digital costumes (see MocapClear).

A CG object is not rendered as one picture. It comes in passes: the object itself (the beauty), its shadow on the ground, a Z pass saying how far away each pixel is, and more. Effects like smoke arrive as separate elements. The compositor stacks them on the plate in order, like sheets of glass.

Each layer joins the stack with a blend mode. Over lays one picture on another using its alpha. Add sums light, perfect for glows and fire. Multiply only darkens, so it suits shadows. Screen only brightens, gently, and suits smoke and haze. A matte cuts holes: here a roto matte of the actor holds out the airship, so it passes behind her head instead of in front.

Then the hard part: making CG look filmed. Real air adds haze with distance, so the compositor uses the Z pass to add depth fog to the CG, matching the plate. Real footage never has pure black, because haze and lens flare lift the shadows, and it has grain. Clean CG has neither, and it shows. Matching black levels and grain is what glues it in. The colour grade that follows is in GradeClear.

Most shots have no green screen. To put something behind an actor, a roto artist draws a shape round them with a bezier curve, a line bent by a few handles. The actor moves, so the shape must be moved too. The artist sets it on keyframes and the computer fills the frames between. Fewer keys means less work, but the shape drifts off the body in between.

Paint fixes things that should not be there. Stunts are done on safety wires (the rigs are in SFXClear), and every frame of wire must vanish. With a locked-off camera, the crew film a clean plate: the same shot with nobody in it. The painter copies clean pixels over the wire.

A sky replacement swaps a dull sky for a better one. A luma key picks out the bright, grey sky. But it also grabs anything bright and grey, like a white shirt, so the roto matte is used to protect the actor. Then the actor is graded to suit the new light.

This is slow, careful work. For The Invisible Man (1933), about 63,000 frames were retouched by hand. Roto and paint are now a big part of the work done in India's VFX studios, and the FICCI-EY 2026 report says AI-assisted rotoscoping already saves 40% or more of the time on some films.

Building a whole palace city for real would take years. So film crews build only the part the actors touch: some steps, a gateway, a few metres of wall. Everything beyond is a blue or green screen. Then VFX add the rest. This is set extension.

Far away, a matte painting does the job: a huge painted picture of mountains and sky, placed on a card in 3D so it shifts correctly as the camera moves. Nearer, where the camera would see around corners, the city is built as a CG environment of modelled buildings. The camera's matchmove ties it all to the real set.

Crowds work the same way. Film a few hundred extras, then multiply them: copy the filmed groups into new places (tiling), or fill the streets with CG agents, digital people who each choose how to move. Software like Massive, first used for the armies of The Lord of the Rings, gives each agent a tiny brain.

Indian films use this heavily. For Baahubali, large parts of the kingdom of Mahishmati were built at Ramoji Film City in Hyderabad and extended digitally, and one studio said a scene shot with about 3,000 people was made to look like 50,000. India is also a giant VFX hub for the world: the FICCI-EY 2026 report puts India's animation, VFX and post-production business at about ₹105 billion in 2025, with a lot of it done for foreign films. For filming inside LED screens instead of blue ones, see VirtualProdClear.