'Rogue One' VFX Team Explain The Difficulty Of Resurrecting Old Characters

Last week I had the opportunity to jump on the phone with the visual effects team behind Rogue One: A Star Wars Story. After the jump, you can read a transcript of my full interview with Rogue One executive producer / visual effects supervisor / writer John Knoll, animation supervisor Hal Hickel, and special effects supervisor Neil Corbould.

In the conversation, Knoll talks about his original low-budget pitch for Rogue One. We also find out just how many people and how much time it took to recreate Tarkin, just how much work was put into resurrected Gold and Red Leader from A New Hope, and if we'll end up seeing more deleted scenes with finished visual effects on the home video release, and talk Princess Leia, miniatures, and much more.

Peter Sciretta: John, last time I saw you I was sitting next to you on the set of Jurassic World. I was geeking out with you about visual effects and new Star Wars movies, but at the time I had no idea that you were developing the first Star Wars standalone movie. Before we get to the VFX, I'm interested to hear about your initial pitch. The Art Of book mentions it was a much smaller, lower budget story. Could you elaborate on how it was different?John Knoll: Yeah, well in the original way that Kathy [Kennedy] was describing these standalone stories, she was thinking of them as smaller, scrappier productions with potentially somewhat lower budgets. That could afford to be a little more daring in experiment, tone, and genre. And so in the original conception of this, I was figuring well we can't afford to go to that many planets or to take a lot of vast, expensive things to produce. So I pitched it more as a spy thriller than a mission war movie kind of thing. And gradually over the course of the development of the project, it sort of expanded and expanded. Disney sort of made it clear that no, that we wanna do amazing stories, go for the best things that you could make. You know, and the budget will be there for it.Everybody's talking about how you guys brought Tarkin back with CG A lot of people think of what you guys do in CG as "oh it's done in computers." Computers make all this happen. But the reality is people make all this happen. Right? So how many people worked to create that Tarkin that we see on screen?John: How big was the crew? Like 16 people with all disciplines included?Hal Hickel: Yeah. I mean, for the number of shots. For the difficulty of the work it essentially was a very small creative.... But just really great, talented folks.And how long did that crew work on those shots?John: Well, we knew that we had this character in the movie probably six months before principal photography began. So we started working on building the assets really early on. And the hope was that we could start developing some first concepts early enough that everybody would launch to post with confidence that we could pull it off. But it's a very time-consuming and challenging problem. So it was in development for a very long time. And we were working out animation rigs and how to drive it well pretty late into the game. We started on it in January 2015 or so, and then we shot the scenes, and they edited, then locked down, and we got the scenes turned over. I think the first ones happened in spring of 2016. And then it was flat out from there.Hal: But it this way, we had someone on the crew who was pregnant, and she created an actual human being in less time than it took us to create a digital one.Was there ever any thought of having Carrie Fisher reprise her role as Princess Leia on set and then digitally de-aging her like what had been done with like Robert Downey Jr. in Civil War?John: No. Unfortunately, there's a time window, a time span within which those kinds of techniques can work and when enough time has elapsed you're really not the same person anymore. Your voice isn't the same and the way you walk and carry yourself is really not the same. So we felt like the most successful approach to simulate, to recreate an 18-year-old Carrie Fisher was to have an actress that's closer to the correct age.Was ever any search to find an outtake of Carrie Fisher that could have fit? Or was it always the plan we're gonna do CG Leia?John: No, we always knew we needed to create one. You know, part of the research doing the character, I've seen all of the dailies of everything of Carrie Fisher from New Hope and there really wasn't anything we felt we could use.Fans were very excited to see Red and Gold Leader return. What's the process of scanning and cleaning the old footage to make it look like it fits in a movie from today?John: Yeah, well that certainly was more challenging than it might appear. Because this film was shot on Alexa 65 and the latest generation digital camera with 17 stops of dynamic range and a really good signal to noise ratio that's almost grain free. And to take this 40-year-old footage that was a bit grainy, it was kind of underexposed. The negative had faded somewhat. To try and get that to match in without really jumping in terms of its image quality was a significant compositing challenge anyway.

Now that first shot you see of Red Leader was a particularly hard one because it had been underexposed and the whole side of his face was completely black. There was no detail in there. And the other ones were not as bad, so we were in much better shape. And so what we did is we took one of the other setups of Red Leader where we did have shadow detail side of his face and we kind of cut and pasted in sort of Frankensteined together a frame of Red Leader that had the sort of appropriate detail. And motion slowed that in and that's what repaired that side of his face. It was all de-grained, and there was a lot of color corrections done through mattes to compensate for negative space. And then lastly the same underexposed and grainy problem existed for the rest of the cockpit. And so we sort of decided to cut the losses on that. We rotoscoped around them, and the whole cockpit interior is a render C.G. model.

This year at the Academy Awards, ILM will be presented an Oscar for pioneering the facial performance capture solving system. What exactly is that?John: It works in concert with a capture tool from Disney Research called Medusa. Where we'll get a performer into the Medusa scanner system and take them through a whole series of facial expressions that exercise all the different facial muscles. And allow us to get a very high-resolution shape recovery of all those different expressions. So then that goes through a process of decomposition where we kind of isolate it down to the different muscle pulls. So we have high-resolution models that represent all the different degrees of freedom that your facial motion has. And then the facial motion capture tool, you insert when we're actually recording an actor, then we have a dot pattern that goes on the actor's face. And these head-mounted cameras that see those dots and what you're doing is correlating what you're seeing in the dot pattern to the higher resolution shapes that we've recovered from the Medusa scan. So you see a particular dot pattern, you recognize that as an eyebrow up and even though the dots may be a half inch apart, we have all the details in between those that we get from the Medusa.That makes sense. A lot of the spotlight is on this facial recognition in Rogue One, but I'm wondering did ILM do any like miniature work on Rogue One? Or is that like in the past?John: No, we still do miniature work occasionally. But not on this show. You know, I started off as a model maker, so the first part of my career was a model maker and then a motion control camera operator, so I shot a lot of miniatures. So this is something that's very familiar and near and dear to my heart. And I'm not shy about suggesting using miniatures where appropriate. But really in this case we didn't have a lot of places where miniatures would have represented good value for money.There is a shot in the trailers with Jyn walking towards a TIE Fighter that's not in the movie. And that is obviously something you guys created for the film. How did that come about and why did it get cut?John: Well, obviously we are still very much in editorial while trailers are being figured out. And so it's not the same editors that are cutting the movie is cutting the trailer. They have access to all our dailies and even to all kinds of pre-vis that has been done. And they'll pull stuff that they think is cool from that and cut it together into a trailer. And there are often times when there's stuff in the trailer that's not currently in the cut or maybe it is at the time they cut that, but then subsequently we're shortening the movie, and that goes away. Or we use a different take. So it's not surprising to see differences between what's in a trailer and what's in the final film. And that's an example where that's just not a beat that was in the movie.The extensive reshoots have been reported about everywhere, and we see a lot of this unused footage in the trailers — shots that you guys did a lot of work that probably hit the cutting room floor this time around. Is there any chance we'll see any of that ever?John: Yeah. Well, on every show there's some amount of work that is brought to some state of completion or even finished and then cut out of the movie. There isn't a huge amount of that on this show. Being one of the executive producers on the show, I had a little more direct involvement in where the story was going and how things were changing as we were trying to condense the initial cut of over three hours down to a more rational theatrical run time. And what we needed to do and what we should take out to make explain that or clarify that. So we got a pretty early heads up about all right, you know what, I think we're probably gonna reshoot that part of that scene. Let's hold off on that. Let's not get too much into the work on it yet. This stuff is pretty solid; that's not gonna change. Let's get going on that. So, we didn't waste a lot. So unfortunately, there aren't finished scenes that were omitted that you'll see on the DVD.Will there be any tweaking of those special effects before the home video release? Or is the version we saw in theaters like the version that's gonna be on home video?John: We have no plans to redo the shots.Okay, well thank you guys very much. I appreciate it.John: All right, thank you.