Cover image via

VFX Master Michael Conelly Talks AR and VR Technology

By Paula Goldberg

Published: July 02, 2019 Last Updated: July 02, 2019

VFX world influencer Michael Conelly gave PremiumBeat an in-depth interview on the unlimited possibilities of AR and VR technology.

PremiumBeat: You’ve worked in the VFX world for a long time as a supervisor on projects such as Snow White and the Huntsman and Charlotte’s Web. What is your process? Are you normally involved with the producers and director during pre-production regarding what can be live action and what should be an effect?

Michael Conelly: It varied quite a lot per show. Generally speaking, I was involved very broadly across the whole production spectrum — often on set, often involved in the design phase — but the bulk of my work came in the form of steering very large teams of artists, making sure they were all working in sync with the best tools for each phase of work, and that the kinds of assets they built fit the director’s vision and the producer’s time and budget constraints. It was a very complex job, but very rewarding for the enormous range of work I did from show to show.

One might invent a new workflow or piece of software one day, then hand it off to a team and teach them how to use it. Deep in production, it’s more about making sure predictions are smoothly turning into actuals — that characters perform the way they need to, that render times are manageable, that shot complexity doesn’t overwhelm production, that change orders are folded smoothly into a complex flow of work. There’s a lot we could go into more detail on, but that’s the gist.

VFX Master Michael Conelly Talks AR and VR Technology - Snow White and the Huntsman

Image via Snow White and the Huntsman (Universal Pictures).

PB: What is AR/VR technology, and how is it transforming storytelling?

MC: Broadly speaking, I’m far more interested in VR than AR, particularly as a storyteller. There are very few stories that are best told in my kitchen, or wherever I happen to be. There’s a very good reason that movie theaters go black before the movie starts. We want full access to your senses, we don’t want you distracted, and we don’t want you expecting that a story must somehow integrate with the physical world — where you find yourself — when the “play” button is pressed.

As far as the technology, we could speak for a long time about what all is required for VR and AR to work seamlessly, but broadly speaking, VR technology is based on a computer knowing exactly where your eyes and hands are (the more of you the computer knows about the better). Once the computer knows where your eyes are, we can draw pictures for each eye that mimic the kind of perspective and parallax that you experience in the real world. So, we can make it seem to you (visually) that you’re seeing a whole other world that maps perfectly to how you’re moving your head and eyes and body. This is complex stuff. We need to update the image ninety times a second in order to trick your brain into thinking it’s really somewhere else. And if the images we draw don’t conform exactly to how you’re moving, it starts to feel really weird, really quickly. There are incredible issues with optics that need solving; there are issues with how to track your motions within a large room; there are issues with wanting to make sure you don’t walk into things in the real world, once you have the headset on. It’s all delightful when it syncs up. It’s engrossing and captivating in ways no other form of media can approach. From a storytelling standpoint, it’s an absolutely remarkable medium. I can think of no better way to transmit imagination from my head into yours.

PB: Your Blackthorn Media has created a 360-virtual reality, interactive origin story called Caliban Below. What is the user experience and what are the new special effects — and how do they yield the emotional result you want with the story?

MC: Caliban Below is a short story, which is part of a much larger project called The Abbot’s Book. So there’s some lineage to the piece, but it’s carefully constructed to work as a stand-alone experience, too.

With each of the pieces we’ve made, we’re targeting a broad audience. We want to appeal to gamers (since they’re currently the most likely to have VR gear), but even more, we want to appeal to people who just like movies and TV. This is why we’ve been so pleased to present our work at film festivals. People have heard of VR, but most people still haven’t experienced it. So, the stories we make first and foremost are built for people who have never done VR before. We have an “on ramp” for the piece that orients our visitor — gives them a framework for the experience — it’s 1680AD, Northern Italy. And then, we teach them just the bare minimum necessary to have the story experience. We teach them how to move around with little teleports we call “blinking,” we demonstrate that their hands can interact with the world, and then we tell them that they will play the part of Caliban: “The scion of a broken lineage, struggling to understand his dark inheritance.” It’s a mystery. It’s a world to explore. A story is there to be found, and your place in it yields some wonderful moments of empathy and, I think, a very new and organic way of experiencing a story. People who go into this world don’t know what to expect, and they come out having a pretty sophisticated understanding of a much larger world, and a sense of generations that have preceded their visit to the place.

In terms of “special effects,” I think that’s the most special. There are all kinds of technical things going on under the hood, but the real magic of the thing is that this “recipe” for storytelling is purely experiential in a way that no other medium has previously achieved. You become this character, you live in this world that moves and changes as you go through it. And there’s a bit at the end, in particular, which engrosses and captures the visitor in a very unique way. I don’t want to give away the ending — it’s quite an effective and wonderful moment of living a narrative.

Michael Conelly

PB: With Caliban Below, the technology is engaging the viewer in a very immersive, unconventional way. You’ve collaborated with Game of Thrones sound designer Paula Fairfield – how does the use of movement and locations connect to music and sound?

MC: I’m such a big fan of Paula. We’re great friends now, after years of collaboration, and I can’t say enough great things about her. She’s like a magician. So creative, so interested, so able to produce a tapestry of aural experience that is absolutely crucial to the end result. She really fell in love with the power of VR and audio as a medium along the way. I think she’s pretty hooked now, making her own stuff. I’m hoping to collaborate with her on a whole new project she’s dreamed up — we’ll see how that percolates in the coming months.

Back to your question though, the sound is so massively important to Caliban Below. There are so many tiny details that you feel subconsciously, rather than in the front of your mind. A bird roosting in a ruin, for instance, can be a passing detail, but it instantly fires a host of associations, and helps you understand the world better: “Ah, this place is old; no one cares to clear the birds out. I wonder how long it’s been this way?” Somewhere else, there might be the sound of a trickle of water, and while we may not see it, that too can speak to important backstory. There are thousands of these moments that fly by, and they form a richness that helps sustain a complete illusion.

VFX Master Michael Conelly Talks AR and VR Technology - Caliban Below

Image via Caliban Below.

PB: Where do you see the future of VR?

MC: Oh, great question. We’ve been hoping that VR would take off as a mass medium sooner than it has. I continue to love it, but when you compare the audience size for VR versus TV or movies, the economics of it have been pretty janky.

What I’ve continued to watch for is the “big boys” of the technology continuing to invest in it: Valve, Oculus, HTC, Apple, Google. Their interest in AR and VR evolves over time, but in general, the technology continues to improve. If the tech stagnates, then I think it would be hard to justify sticking with it. As it is, it’s moving slower than I’d like to see. But — and this is a big but — one of these days, it’ll be good enough and cheap enough that I think it finally will break through. That’ll be great to see.

When used right, there really is no medium more potent for transmission of ideas and feelings. And so the future continues to be promising. I think it’ll get here someday, it’s just a question of when. It would be spectacular to see the budget of a Marvel movie funneled into a VR experience. The things you could do with a “proper” budget would be mind-bending and electrifying. With any luck, someday the market economics will support that evolution. Until then, we’ll have to make do. I think the love of the work shines through just fine today. With any luck, more is in store in the not too distant future. Til that fine day!

Looking for more industry interviews? Check these out.