You Can Now Control A Robotic Camera Arm With An iPad – ON SET REVIEW
I have experienced the future of filmmaking firsthand. It’s a mix of Mandalorian-esque virtual production and robotic camera arms that you can now control with an iOS device.
Every once in a while, I’m fortunate enough to experience some type of filmmaking advancement that changes everything. The DSLR boom, editing on laptops and now phones, gimbals, drones, robots, and recently the latest in virtual production.
What I’m about to tell you about is the culmination of all of these technologies into one of the most impressive displays I’ve yet to see in my career. The new Motorized Precision app brings the control of their robotic camera arms onto iPads and iPhones.
A few years ago, at Siggraph, Epic Games invited me to their infamous Virtual Production demo, where they put a motorcycle in front of an LED volume that created a realistic 3D environment. They used a Technocrane with a camera tracker to capture the actor and bike, while the background display rendered movement in real-time. What I didn’t know then is that at the same time, they were already using that exact setup on the set of The Mandalorian.
Today, I am standing with the crews of Shift Dynamics and Gear 7 in Nashville, TN watching them capture another motorcycle on the virtual production stage at Arc Studios. The difference is this time; they are shooting on a Motorized Precision Colossus robotic camera arm. The importance of that I’ll touch on in just a moment.
You may recall a few years back when I talked about Motorized Precision’s KIRA robot that shot a Microsoft commercial using and XBOX controller. The controller would allow the robot operator to use the left joystick to move the robotic arm and the right joystick to pan the camera. You could also use the controller to set keyframes and then program your entire move in the palm of your hands. Then you’d go into MP Studio on your machine and run through the camera moves.
So what’s happened since then? Technological advancements we couldn’t even fathom.
FIRST. Not only are robotic camera arms much more common on commercial and film sets, Motorized Precision’s work specifically has pushed their software to greater heights. While some other robots require spreadsheets and math, MP has always taken a creator-first approach that makes using its tools easier and more accessible to the masses. It’s genuinely their software that sets them apart from the competition.
SECOND. We live in a rapidly integrating world of virtual production. Real-time video game engines are generating cinematic quality visuals on LED panels, and the price of entry into this space keeps dropping. Today, it doesn’t matter if you are in video production, filmmaking, VFX, video games, or a 3D generalist. Everyone is moving into 3D pipelines as the traditionally independent film, and video game entertainment industries begin to merge technologies.
THIRD. Say what you will about how Apple releases new products that tend to be behind the PC GPU rendering speeds or iOS features seen years prior on Android devices. One space they absolutely dominate is the mobile camera, LiDAR, and AR/VR features. The latest iPhones and iPads have built-in tools that just a few years ago would’ve cost you thousands in hardware costs to do similar actions. You can literally use an iPhone to scan any object into 3D and in minutes, start animating and placing those assets in any 3D software.
So, where does that bring us now?
Motorized Precision’s MP AR iOS App
Today at CineGear, Motorized Precision announced their new iOS app, MP AR. When I was told about the app, my expectations were FAR lower than what I was finally shown. I was simply expecting some type of MP Studio integration that made on-set life a little easier. A way to add keyframes from a touch screen. Yes, the app can do that. But it can do SO much more.
Here’s just a quick rundown of what the app can already do:
- Create AR sets with a virtual representation of your robot and set pieces
- Test camera moves, reach, and if a robot will fit on location
- Place keyframes or add camera shake while moving the iOS device
- Previz entire scenes remotely for stunts, VFX, or commercial work
- Use QR codes on objects to target precise points
Let’s dive into these features and why they are both impressive and fun to use.
Previz with AR Sets
Just months ago, I needed a robotic camera arm for a remote shoot, but I didn’t know how big of a robot I could fit on location. I had the client take their iPhone and film the entire room and send me the clip. I then used that footage to recreate an accurate set in 3D and could then figure out how much room I had for a set and robot. It was… a process. Nothing overly complicated, just a little time-consuming.
You can do all of that in the MP AR app in real-time.
Whether you are a location scout, DP, director, or even a client, you can now open the MP AR app and drop a 3D model of the robot in an AR set on your iOS device. You can visualize the set on your iPad or iPhone using the built-in camera, Reality Kit, and LiDAR.
This allows you to visualize AR objects in front and behind real-world objects. Now you can see how a robot will move in the space or see how a shot will look. In addition, you can even previz moves remotely, and with a button tap, send those movements over to MP Studio for the robot tech to get everything ready to shoot.
You can even export clips and send them via the iMessage app for a smoother remote workflow with clients and crew.
Precision Control of Robotic Camera Arm On Set
This is where things get really fun. Not only can you previz moves and send them to MP Studio, but you can also alter moves on set.
It’s not uncommon for there to be some type of communication breakdown on set between clients with creatives and technical personnel. It may be hard to explain the exact type of move a client wants. Now with the app, you can avoid what historically could be hours of setup time and have the client show the exact move with the iPad. Then you can take those keyframes and clean them up in MP Studio. It simplifies the entire creative process and expedites everything.
An added benefit of using an iOS device is adding the elements of a handheld camera shake so you can maintain some organic movement. Watching the test of this feature proved to work best when movement was more subtle with the iPad and then gently cleaned up, removing an excessive shake inside of MP Studio.
For smooth precision moves, you can just set individual keyframes on the iOS device in the Keyframe mode. Additionally, you can use QR codes on set to place precise targets for the robot to hit. Simply scan the code to mark your point.
Virtual Production Integration
This isn’t something covered much in the official press release, but on location is where I really saw some true potential of using robotic camera arms in virtual production. The value of Motorized Precision’s hardware allows all that data to pass through the virtual production servers. Unlike dolly moves with a camera tracker, using a robotic camera arm lets you know every camera position ahead of time.
Virtual Production is not the seamless workflow you see in these behind-the-scenes featurettes. There are still many kinks to iron out, including things like not-quite real-time environments. Depending on the size of the environment, it’s still possible for the LED walls to lag. Meaning your foreground action may be moving too fast for the 3D background to keep up. By using a robotic camera arm, you could counteract the lag by having the environment anticipate the next move by knowing exactly where the camera will end up. And since all that data is already built into the robot, it unleashes your creativity to get the shot.
About the App
Motorized Precision could have easily mailed in this app like so many companies often do. Instead, they have set a new standard. I would suggest using the app on an iPad pro to benefit from the larger screen, allowing more workspace and flexibility.
The app works best on the latest generation of iPhones and iPads. Requires iOS 15.0+. The MP AR iOS app is available now in the App Store.
Cover image by Ryan Green courtesy of Motorized Precision/Shift Dynamics/Gear 7/Arc Studios.
