How an iPhone Facial Capture Choreographed 200 Drones

3D artists, musicians, and a swarm of drones combine to create a monolithic performance high in the Michigan sky. See how it was done.

Here’s something you don’t see every day. Detroit-based design and animation studio Hobbes teamed up with musicians VWLS and Firefly Drone Shows to create a 300-foot-tall, flying, singing face. Just look at it.

The performance used a flying swarm of 200 LED-equipped drones. And, while no strangers to creating drone performances, creating an entire show with a speaking character was a first for Firefly Drone Shows and Hobbes

Let’s find out how the team captured facial data on an iPhone and turned it into an epic, god-like musical performance. But first, check out the music video below:

Creating “High in Heaven”

Since their first collaboration in 2018, both Hobbes and Firefly Drone Shows have continued to push the limits of live drone performances. The two companies set out to create a “drone face” performance and enlisted the help of musicians Josh Epstein and Louie Louie. The goal was to have a drone-driven entity sing a song in the night sky.

To achieve this, the Hobbes team began creating mock-ups and sketches of faces.

They had to find the perfect balance of strong facial lines that the swarm of drones could follow and animate without breaking shape or crashing into one another. The final look had to be driven by confident line work, designed to resemble stone carvings. With the look decided, Hobbes then turned to the musicians to capture their performance.

Using Moves by Maxon, a free iOS app built for Cinema 4D artists, the musicians captured multiple performances and sent their capture data over to the team at Hobbes. That data was then brought into Cinema 4D, where the team used the performance to plot out and drive the drone paths.

Once the performance of all 200 drones was locked, the artists began the painstaking task of coloring the performance, a process that took several weeks, as the color and intensity of every single light on every drone had to be programmed in advance.

The next challenge was figuring out a complicated, highly coordinated flight plan, a process that involved close collaboration between Hobbes and Firefly Drone Shows.

While these individual drones can reach speeds up to sixty miles per hour, there are FAA regulations in place that limit the speed of drones when flown in a swarm. With a legal top speed of fifteen miles per hour, the performance would have to be altered.

The solution? The entire sequence was slowed down to twenty percent speed for the actual flight, and then the performance footage was sped back up five times to match the audio of the song.

On the day of the shoot, the Firefly team showed up bright and early to prepare the 200 drones for filming, all while also social distancing high up on the slopes of a Michigan ski resort.

The flight paths were updated to every drone, and the show was ready to execute. Syncing up and taking off, the drones took to the sky.

With a three-minute song length, and a fifteen-minute max flight time, the slowed-down performance was captured perfectly in the night sky.

You can go further behind the scenes in this mini-documentary of the process from Hobbes.

Cover image from “High in Heavens – Behind the Scenes” via YouTube.