'The Mandalorian' Behind-The-Scenes Video Reveals The Groundbreaking Tech Used To Create The Series

It's hard to believe, but much of The Mandalorian was, well, not real. Not physically, anyway. The series employed a relatively new technology called StageCraft, which uses a rear-projected LED screen to create a reactive environment that moves with the camera. It's kind of like green screen, but better – and far more advanced. A new The Mandalorian behind-the-scenes video shows off just how this works in all its impressive glory.

Back in the old days, when filmmakers wanted to create entire worlds digitally, they usually relied on green screens (or blue screens, depending on the production). This required actors to work against completely blank backdrops and rely on their imagination to fill in the blanks. But StageCraft goes the extra mile, and renders a backdrop in real-time – as the cameras are rolling. Not only that, but the backgrounds are designed to both light the actors and be photographed – while moving along with the camera. It's hard to wrap your head around, really, but the video above does a great job of showcasing how it all works.

ILM developed StageCraft with "in-camera finals, allowing film-makers immediate and complete creative control of work typically handed-off and re-interpreted in post, improving the quality of visual effects shots with perfectly integrated elements and reducing visual effects requirements in post." This video corresponds with the news that ILM is making StageCraft available for filmmakers, agencies, and showrunners worldwide. This means more and more productions are going to start using this, effectively changing the landscape of filmmaking.

"We've been experimenting with these technologies on my past projects and were finally able to bring a group together with different perspectives to synergize film and gaming advances and test the limits of real-time, in-camera rendering," said The Mandalorian creator Jon Favreau. "We are proud of what was achieved and feel that the system we built was the most efficient way to bring The Mandalorian to life."

Richard Bluff, Visual Effects Supervisor for The Mandalorian, said: "StageCraft [allows] us to shoot principal photography on photoreal, virtual sets that are indistinguishable from their physical counterparts while incorporating physical set pieces and props as needed for interaction. It's truly a game-changer."

Over 50% of the first season of The Mandalorian was filmed using StageCraft, requiring actors to perform in "an immersive and massive 20' high by 270-degree semicircular LED video wall and ceiling with a 75'-diameter performance space, where the practical set pieces were combined with digital extensions on the screens. Digital 3D environments created by ILM played back interactively on the LED walls, edited in real-time during the shoot, which allowed for pixel-accurate tracking and perspective-correct 3D imagery rendered at high resolution via systems powered by NVIDIA GPUs. The environments were lit and rendered from the perspective of the camera to provide parallax in real-time, as if the camera were really capturing the physical environment with accurate interactive light on the actors and practical sets."