Matt Reeves Defends Andy Serkis' "Digital Make-Up" Comments

You will emerge from viewing Dawn of the Planet of the Apes amazed at the motion capture performance by Andy Serkis. Sure, you've seen him do Gollum in Lord of the Rings and even Caesar in Rise of the Planet of the Apes, but Dawn takes the work to an entirely new level. Recently some people in the computer animation and effects space have gotten angry over comments from Serkis that WETA is just "painting digital makeup onto actors' performances." Many have felt that minimizes the work of many talented visual effects artists. I didn't get a chance to talk to the actor, but I was able to chat with Dawn of the Planet of the Apes director Matt Reeves about the Andy Serkis digital make-up controversy.

First off, here is what Andy Serkis has said about the process of translating his motion capture performance to computer animated characters:

The technology has evolved in the sense that it's become more transparent. You don't really realize that it's there at all anymore. And even more importantly, the perception has changed — the use of the authored performance is much more respected. The technology is one thing, but basically one has to remember that it is only technology. Performance capture is another bunch of cameras. It's 360 degree cameras filming an actor, and I think it's the understanding of that has changed, and that's happened because we've gone from a single character like Gollum to multiple characters in films like Avatar. It suddenly went from being an outside, peripheral activity and a singular activity to virtual production. Avatar was a groundbreaking movie. And [in terms of] performance capture live on set, Rise of the Planet of the Apes was a game changer there because it enabled you to be actually out on location shooting the movie.

And then this movie, Dawn of the Planet of the Apes, is the biggest ever. In Rise we were shooting on sets for the first time. And with this, it's the biggest on-location shoot with performance capture and multiple characters. There's been a significant change. But also the way that Weta digital, whom I've worked with on all of those projects, that they have now schooled their animators to honor the performances that are given by the actors on set. And the teams of people who understand that way of working now are established. And that's something that has really changed. It's a given that they absolutely copy [the performance] to the letter, to the point in effect what they are doing is painting digital makeup onto actors' performances. It's that understanding which has changed as much as anything.

Now one must first understand where Serkis is coming from. Many feel that Serkis was snubbed for not getting a Best Supporting Actor nomination from Academy Award voters for Rise of the Planet of the Apes. The feeling is that Academy voters were split, feeling the performance is partially the work of visual effects artists, and the recognition deserved to be in a non actor category, especially considering Serkis's own face and body never appear in Rise. So you can understand his defensive tone when talking about the idea that his art, especially a performance like the one he delivers in Dawn, should be recognized.

So I asked Dawn director Matt Reeves about the controversy, and the director revealed a shocking fact:

They are up in arms about it, sure. Well here's the thing, that term [Digital Make-up], it's been explained to me was actually coined by Weta.

So visual effects artists are upset over Andy's use of terminology that comes direct from the visual artists at WETA. But I think its more about the minimization of the art involved in post production. Reeves explains the process:

I think there is a misunderstanding about what exactly happens. The performance if you're affected emotionally by what's happening, which you are when you watch Caesar, you're affected by Andy. Andy is one of the best actors I've ever worked with. I think he's incredible. And that performance is recorded on this equipment. And then I think that there are people at outside companies, not Weta, who are up in arms because that description makes it sound as if there's a kind of button that can be pushed that could create that. But there's two levels of the highest kind of artistry that are going on in order to create Caesar. One is a performance. And that is 100 percent Andy.

But the other is a translation of the performance, which is very difficult because as great as Andy's performance is, he is not an ape, and his performance must be translated to a completely different form.

What Weta does is incredible. And actually is an art, because what they're doing is they're taking a performance and they're finding a way to translate it onto anatomy that is nothing like Andy's. So those are choices. This is something that I was engaged in with them. Like we're looking at the eyes of Caesar; do they look like they are expressing what Andy was expressing? And WETA and I, sometimes I would look at something and I'd say, you know, Andy, he looks not only sad like you have here, but he also seems to be angry. And so they'll look at it and go ah, yes, there's something about the shape and tension in his eyelid. We have to make sure that the animators are using that. And so they're doing something that, and also his anatomy not only just facially, but also his body. Like his arms are a different length.

So WETA's digital artists are not only translating Serkis' captured performance but making artistic choices.

It's not a direct one to one translation. It's a translation that requires artistry in order to realize. And the artists at Weta are better than anyone, they are at the forefront of this. I think that they will tell you that the methodology is they need the performance captured in order to do the work. So there was never a case where we're like, you know, hey, see that background guy? Could he just be doing this? The way that they did that is that they said, well we need a piece of capture to do that. We need the performance. Even if it came down to the animators themselves play, like they would send me some things like there's a background guy. Terry Notary ends up doing a lot of the background guys. Like guy scratching himself, guy eating. He's incredible. And by the way, that's just as important too to the performance. He teaches all the actors. That's a perfect example of how important the performance is.

And Reeves gave me an example how Rise of the Planet of the Apes suffered when WETA didn't have motion capture performances to translate:

We had stuntmen who were all Parkour specialists. We did that because the idea was in the last film there were moments where mo-cap wasn't used for things that human beings couldn't do. And as a result there was a section where the apes felt like they were grounded by actual laws of gravity. And then there were other ones where it was just really hard to exactly match the gravity. And so you would see them doing things that looked a little bit more animated. And so not that the animators weren't amazing, but we were going wow, there's a disparity. Like this feels totally real and this is a little hyper real. And so on this movie, we used Parkour performers who could do those things. But even they couldn't just do what they did. We needed Terry to teach them how to be and do like apes. So they would do these jumps like you could never believe, but they did them as apes. And that was the basis for what all those performances came from. So you need that performance. It is what drives everything.

But back to the differences between the origin and the intended translation –

Weta strives to be as faithful as possible to the emotions that are being expressed, but that faithfulness comes from being interpretive about how to express it because the shapes are not the same. Andy's mouth is so expressive but Caesar has a muzzle, not a mouth. So how do you get that shape on there? And how do you get that same emotion? They're following Andy's shapes, Andy's expression, Andy's emotions, but they're finding the way that only artists that are as talented as they can a way to express it that is faithful.

But it takes a lot of work of an artist to make that faithful adaptation.

They could never push a button to make it look like that. And on top of everything else, so that's what they're doing to drive the performance. And then on top of that, they're doing hair simulations. They're doing moisture simulations. They rebuilt their pipeline. These are the most photo realistic C.G. characters that have ever been done. I mean, the level that Weta, the level of artistry from Weta is astonishing. I've now worked with a few different places. Certainly they are the best in the world at what they've done in this movie. And I think that this is a new high water mark. I think they're incredible.

So when awards season comes, who should be recognized? Should Serkis get a nomination for his motion capture performance or should WETA get a nomination for the incredible computer artistry they applied in the translation to another animal species? Or should there be a new special category created for motion capture performances which awards both the actors and the vfx artists simultaneously in the collaboration?

So does that mean there should be a special category for Andy's performance? And then because Weta's part of that? And I say, to me the categories already exist. There's Best Performance by an Actor and that's what Andy does. And there are Best Special Effects in a Motion Picture and that's what Weta does. Those two categories already exist. And both of them I think are at the top of their form.

After I conducted this interview in advance of the release of Dawn of the Planet of the Apes, it seems like Andy Serkis has changed his tune a bit when talking about the translation in the performance capture process. Serkis told International Cinematographer's Guild Magazine the following:

[The technology] has progressed extraordinarily in terms of the fidelity to the authored performance. But it's only partly to do with technology. It's more to do with having a team of people who fully understand the interpolation of a performance that they are seeing onscreen. They're taking the data and wrangling it in a way that honors the performance. The facial pipeline is now so refined by Weta and so understood by the artists that do that very delicate phase between taking the actor's raw underlying emotional performance and translating it into the actual final rendered character. It's fully understood, spectacularly so.

Meanwhile, when asked about the process, Weta Digital senior vfx supervisor Joe Letteri told ScienceFiction.com that its very much a collaboration between the performance and the animators, and that the motion capture technology is just a tool to store information:

Matt [Reeves] is obviously directing it, he's constructing the film, he's constructing the narrative, and he's working with the actors to bring out the moment – what's the heart of the performance. But, yeah, we have to take that all on board and we have to do a lot of that behind the scenes to present it back to him in a finished fashion...Yeah, it's a combination of the actors and the animators. The machine is only there as a tool to allow us to store information, amplify things that we do. None of that is machine generated; it's all done by the artists, either actors or animators. We did a lot of research and studied a lot of real apes... we videotaped them, we photographed them. You start to pick out their individual personalities. Then what we do is go in and the animators will do studies of these characters. They take little moments that they like and say 'Oh, look what happened when the baby crawled on the mother's back here." And they try to reanimate that so they can understand for themselves how the physicality works and also, once you have that down, why you get that same sense of emotion from them when you see them.

One thing is for sure, if Andy Serkis wants to get the recognition he sincerely deserves, he needs to be more gracious about the process and the artistry in the translation. I think once most people learn about the process they'll see that Serkis (and the other Ape performers like Toby Kebbell) deserves credit for his performance, but that the artists at WETA also deserve credit for their work as well.

A couple years ago, PressPlay also did a great video essay on the notion of a new "Collaborative Performance" Oscar, which you can watch below.