Why Virtual Production Could Work for You
Virtual production and shooting on LED volume sets are mostly beyond our budgets. But Hollywood thinks the technology is infinitely scalable.
We’ve all the seen the impressive new virtual production sets, surrounded by banks of LED screens that transport us everywhere without effectively going anywhere. It’s the modern version of the Hollywood backdrop, which in their day were some of the most impressive but, ironically, uncredited pieces of large art in modern times.
See the video below for insight into this lost art.
As content creators, we wish we could use such supreme tech as the new virtual production sets, but if the recent NAB show taught us anything, it’s that the marketeers want to enable all of us to use them. There are obvious barriers to entry into this exclusive LED volume club, but it all depends on what you have in mind.
AJ Wedding, head of Orbital Studios in LA, founded his virtual studio last year to pursue the virtual set market. He was part of a NAB 2022 roundtable discussion about his studio’s work on the FX show Snowfall and had this to say about using virtual sets,
One of the cool things about what we did with Snowfall was that we started with just video plates. The more they got used to it and knew that it could work, they wondered what else they could do. Maybe add some camera tracking to this, and then at the end of the season, they were, “Let’s do a full virtual set.”
– AJ Wedding, Orbital Studios
Snowfall is a show based on the 1985 drug epidemic in LA. All the backplates they used had to have the new buildings painted out to match the time. But those same plates also showed cars from that era; shooting those for real would have been expensive and time-consuming, blocking streets and hiring classic vehicles. So, even with the VFX paintwork, they were already saving money.
Wedding has been quoted as saying that using virtual sets saved up to $49,000 an episode by reducing transportation man-hours, shooting time, and crew loads.
Hollywood cinematographer and VFX specialist Sam Nicholson of Stargate Studios also sees how the technology can scale up and, importantly, down. He recently oversaw a huge LED volume for the HBO Max series Our Flag Means Death, which featured a massive 30ft x 160ft LED wall designed to surround a practical pirate ship built on set, stating,
They’re not all going to be like that. This technology is scalable and can work on a ten-foot screen if needed. We’re doing the same thing for Disney, and we’ve had great success with Station 19 and a show like Solos for Amazon. Virtual Production comes in all sizes and shapes.
– Sam Nicholson, Cinematographer and VFX Specialist
As we navigate this new set tech, looking forward, the exceptions may be the huge LED volumes of Our Flag Means Death. At the other end of the scale, Sam has used a single OLED to represent a moving backdrop for a scene.
It’s all about if it feels real, which is better for actors, directors, and editors to get everything together, so it’s much easier for them.
– Sam Nicholson, Cinematographer and VFX Specialist
Minimizing Virtual Production
Depending on the stage you’re at in your production, you can look to raise the production value by introducing some virtual backdrops. You don’t particularly need tracking or Unreal Engine for this. Just concentrate on what the camera is seeing.
For instance, in Amazon Prime’s Solos, there’s an episode with Helen Mirren where she’s in a small spacecraft. She looks out of the windows and sees space with planets, all coming from screens. In that scenario, all that was needed were two 20-foot screens arranged on wheels so they could fill in where needed on the set around the space capsule.
You could’ve built an entire wall around the set, but the mobile screens were a much cheaper option. It’s still a living backdrop with the help of actors, directors, cinematographers, production designers, and editors. It’s just based on where the cameras are shooting.
Another show called Good Trouble, from Disney, had a 70ft balcony that needed a backdrop, but the production could only afford a 14ft screen. Again, the screen was rolled into position with the cameras shooting to it and, this time, they were tracked so the view made sense when the screen moved. In effect, the volume comes to you but scales down to what you need when you shoot.
In a show like Station 19, small screens are used for several scenes. Balcony scenes, driving scenes, and other scenes might be enhanced by moving assets in the background. You can even re-film scenes after you’ve been on location with empty backplates that you fill later on.
On Station 19, one of the actresses caught COVID and couldn’t be on location, so they did just that. They got a stand-in and then shot the original actress when she recovered with the screens in place.
With some imagination, you’ll start saving money and have more realistic footage, and it’s all through the lens. “Everything doesn’t have to look like The Mandalorian,” concludes Sam Nicholson.
New Virtual Production Applications
There are several new enhancements to the world of LED volume that have been designed to work around budget constraints or limitations on space. One of those is an LA-based truck with a sidecar called the Magic Box. This is a mobile LED volume that can be set up in a few hours.
At the recent trade shows, a lighting company, Quasar Science, was showing how they could map video onto their lights. The idea was that while the front and sides of a backdrop were playing, you still needed lighting from above.
This “image-based lighting,” as they call it, is able to blend the palette of colors ambiently, so the scene doesn’t jar when lighting from the front is hugely different from lighting from above.
We also found a booth at NAB 2022 called Skyglass that allows your iPhone’s LiDAR scanner to work with cloud rendering powered by Unreal Engine. It’s able to place your shots in a photorealistic virtual environment.
In effect, Skyglass is tracking the position and orientation of your phone’s camera and streaming it to the cloud, where a render of a virtual environment is created and sent back.
Be Prepared for a Virtual Project
If you’re currently working on a project that you think could use a virtual backdrop, it might not be wise to jump into it. Throw up a green screen and do it the old-fashioned way—it may be more accessible.
Testing is essential when these new assets are involved, and rehearsals are necessary for everyone involved. You also might have to deal with the amount of light the screens push out and may need to have lower light levels. There are a lot of parameters to deal with.
But, the promise of using even just back plates on a couple of movable screens could elevate your shots hugely. It’s about managing expectation and managing a backdrop—you could even shoot a plate one day and then use it the next.
While the LED volume world has started at the massive end, it’ll soon scale down as people understand how specific scenes can work with smaller screens.
For more on new tech and DIY filmmaking tips and advice, check out these articles:
- Are Holograms Here to Stay?
- A Filmmaker’s Guide to Creating Holograms
- 5 Tutorials You Need for Making a DIY Film Cart
- What to Buy for a Green Screen Kit (and Where to Buy It)
- Depth Scanner for After Effects: Create Depth Maps on Video Instantly
Cover image via Obital Studios
