Save 80% and get all your credits upfront with the new yearly subscription! See details

5 Shopping Cart
Your cart has been updated
checkout
Categories

Cover image via

Will Real-Time Digital Sets Replace Green Screens?

Mike Maher
Published: Last Updated:

Disney, Netflix, Amazon, Paramount, and more are betting big on digital sets, Volumes, and digital set extensions. Let’s take a look.

Practical meets virtual. Digital sets are here to stay, and thanks to COVID-19, they’re being adopted at a much more rapid pace. Also dubbed XR, or cross-reality, these systems combine physical set pieces, cast, and crew, with digital sets or set extensions.

What’s different from traditional rear projection techniques is that these sets are built in real-time game engines, like Unreal Engine. The sets themselves can be altered during a shoot, which also means there’s a new rising role in the production world.

At ILM (Industrial Light & Magic), they call this the Brain Bar—a group of 3D artists and technicians who work on-set with the camera crew and lighting crew to make these systems work.

Brain Bar Team
A new role in production—the Brain Bar. Image via Disney+ courtesy of ILMVFX.

They work hand-in-hand on set with the gaffers to light the set and match colors, while also working with camera crew to track camera position and lens settings so that the digital displays show at the proper angles, depth of field, exposure, and more.

Tracking Camera Angles
The 3D artists work with the camera crew on tracking camera position, angles, etc. Image via Disney+ courtesy of ILMVFX.

The biggest challenge? There aren’t enough of these people in the industry. This type of work requires someone with working knowledge of both filmmaking and video game production.

The ILM Volume is a circular set with a seventy-five-foot diameter.

Mandalorian Set Volume
The circular set is seventy-five feet in diameter. Image via Disney+ courtesy of ILMVFX.

It was originally designed for interactive light with a green screen backdrop that could follow an actor around. That way, there would be no need to re-light screens between takes.

Mandalorian Set
Originally, it was designed for interactive light with a green screen backdrop. Image via Disney+ courtesy of ILMVFX.

Producer and Director Jon Favreau went to ILM to combine the filmmaking concepts and ideas he had from making Elf, Iron Man, The Lion King, and The Jungle Book.

Testing Lion King Environments
Testing real-time environments during The Lion King production. Image via Disney+ courtesy of Movie Trailer / PTP.

During the production of The Lion King, the film’s VFX team began testing video game engines for digital sets.

The content on the screen was motion tracked to the camera, so that the position of the camera reflects a realistic look to the set through the camera lens. This allows for perspective and parallax changes while filming.

Camera Positioning
The position of the camera reflects the real-time look of the set. Image via Disney+ courtesy of ILMVFX.

Favreau credits LucasFilm’s Kathleen Kennedy to committing the resources into building the Volume for The Mandalorian.

Mandalorian Set Volume
The set is basically a giant lighting box. Image via Disney+ courtesy of ILMVFX.

In the simplest way, it’s like being in a large lighting box. Placing a subject in the space realistically lights your subject. And, with Mandalorian’s suit becoming more reflective over every episode, it gave the production team an incredible amount of freedom to light the Beskar helmet and suit.

The real game-changer is that most of the technology isn’t proprietary. LED screens, game engines, and 3D environments are all tools people have access to.

ILM is currently constructing three more Volumes across the globe, and studio executives at Netflix, Amazon Originals, and Paramount have revealed they’re also already using the technology, and building studios of their own.

It’s now about getting the artists and technicians in place to push the concepts even farther. So, what are you waiting for?


Cover image via Disney+ courtesy of ILMVFX.

For more filmmaking tips, tricks, and advice, check out these articles:

100+ Smoke & Fog Effects
Packed with 143 atmospheric effects, Vapor is an ideal way to infuse your footage with an elegant, cinematic look.
A