/Film Visits ILM To Learn Why The Visual Effects In 'Warcraft' Are Considered "Groundbreaking"

It's not often that I get invited down to Industrial Light & Magic in San Francisco, CA, but when the invite is thrown my way it's usually to see something cool. Earlier this month I traveled up north to see how filmmaker Duncan Jones used ILM to create the orcs in his big screen adaptation Warcraft, based on the popular Blizzard video game series.

I'll be completely honest with you: the marketing for this movie had done little to convince me that it was a film I wanted to see. I'm not a video gamer and have never played World of Warcraft, and to be completely honest, I'm not a big fantasy (orcs, wizards, dungeons and dragons) type of guy. But I'm a big fan of Duncan Jones and that alone had me continuously searching for reasons.

So I hopped on a plane and visited the visual effects house that George Lucas built, hoping I would be wowed. The filmmakers behind the movie pulled the curtain back to show us how the magic was created, and that in itself was not only very interesting but very impressive. So hit the jump and learn why ILM considers the Warcraft facial capture advancements to be groundbreaking.

Duncan Jones knew that when he came on to Warcraft that the biggest challenge would be creating the orcs themselves. He explained to me that they needed to have characters that not only could win the audience's empathy, but could "hold a close-up."

A lot of the design choices in the video game were made over twenty years ago. ILM realized they needed to adapt the more cartoony look of the designs to fit the mandate of presenting what those characters and species would look like if they were real. Wei Wang, a Blizzard concept artist, gave them an initial sketch which became the blueprint on how the orcs should look. And you could easily mistake this initial sketch as art inspired by the movie – it looks that close to what we see in the film.

In the world of Warcraft, orcs have the same size heads as humans but are much wider and stand 6 and a half to 7 feet tall. The artists decided that practical make-up wouldn't work for what they needed, so they instead decided to go all in on a performance and facial capture approach.

By creating the orcs using performance capture, they didn't have to worry about how big, tall, or muscular the actors were, instead focusing purely on performance. The orcs' movements were developed by Terry Notary, who of course also worked on the Lord of the Rings films and the Apes films. Terry Notary played a lot of the "background" characters in the film, as well as some of the hero characters, notably Grommash Hellscream.

ILM had been working on their facial capture technology for a while now and felt they were finally making impressive headway with the work they did on the Hulk in The Avengers. ILM completed some tests to help convince Jones that the technology was at the stage it needed to be to accomplish his vision for Warcraft, but it wasn't until he got back the first completed shot that he was convinced that his bet would pay off. That first shot is one of the most impressive shots in the entire movie, and you'll see it very early in the film. It's not a big impressive battle, but a very emotional scene between an orc and his pregnant wife in their hut.

The emotion that comes from the faces of these computer generated creatures is astounding and it's no surprise that ILM considers their facial capture work on this film to be "groundbreaking." We're seen some impressive facial capture work from WETA, especially on the recent Apes films, and while Andy Serkis has been fighting the battle of what he calls "digital make-up," performances still require animators to rework and manipulate the data captured on set. But this film supposedly features a closer to one-to-one ratio of performance to rendering.

We were shown a very impressive video showing a side-by-side comparison of the human motion capture performances and the final renders, and I know we've seen this sort of thing before on Avatar and the Apes films, but this was even more impressive — it's amazing how much of the facial movements of the human actors made it to the CG versions of the orcs. You can see a short snippet of some of the footage I screened in this new featurette:

The facial motion capture was not approached as something expendable or something to be used as merely reference, but as performances that needed to be captured for the final product. ILM is particularly proud that very little modification was taken to create the completed facial performances in the film. The facial capture was accomplished with two cameras on the face of each hero orc actor, with over 200 motion dots on the face which needed to be tracked. People, assisted with some software tools, have to individually guide the tracking of every dot. They also did a rigid track on the eyeballs and teeth, allowing them to separate out the jaw motion from the skin/head.

Once they had all this data, the ILM animators would reproduce the performance on a 3D scan of the actor to make sure it looked right before they even began to transfer the performances to the model of their orc counterpart.

When modifications needed to be made to the motion capture, it was usually to make sure the performance was coming through, not to make big changes to intention. Most of the animation work done on the heads falls into two categories: creating the neck movement of the character (something that couldn't be tracked, as the size of the actor's necks were so very different from that of the orc characters) and pushing the lips to make room for the species' huge tusk-like teeth (something that just couldn't be mimicked in the real performance).

On set, most of the hero orc actors were given little Invisalign-style mouth implants that pushed their lips out a little. Not only did it help the actors with feeling that aspect of their character's physical design, but it affected their speech just a little which can be heard in the final film. And of course, Skywalker Sound also pitched their voices down a bit.

Industrial Light and Magic got a little help from their sister company across the bay — Pixar Animation Studio. As you know, George Lucas not only created ILM but also Pixar. Both companies are now owned by Disney. ILM was having trouble with the tusk-like teeth that come out of the orc's mouths and Pixar had done some good work on tusks for a character in Monsters University, so ILM traveled across the Bay Bridge to see a demo of a deformer that Pixar had created for that film. ILM was able to adapt it for their use on Warcraft.

ILM was on the set of the movie not only to capture the performances but to facilitate a live but rudimentary version of the orcs rendered on the monitors during the filming of every scene. This allowed the actors to hone their performances to the movement of the CG orc, and also allowed the cinematographer to recompose shots on set.

And because the data was captured digitally, the cinematographer and Duncan Jones could play around with the framing of these shots long after the actors were off the soundstage, choosing new camera placements and seeing how the performances worked in the scene through the viewfinder of their on set camera.

Some of the actors who had done motion capture previously were under the assumption that by the time the process was done, they wouldn't see much of their original performance in the final character. ILM put together a test showing them that their work would be visible in the final rendering, which made them even more invested in the production.

While performance capture is old hat at this point, Warcraft features motion capture performances on the largest motion capture volumes ever filmed. These aren't just performance capture volumes with blue screen, but huge soundstages often filled with big practical sets, which would later be extended with CG — this seems to be the magic formula to getting CG right.

While the facial capture took new leaps, animation augmented the motion capture performance when the actors were not capable of certain movement or action on set. This posed a challenge for animation supervisor Hal Hickel and his team as they needed to ensure that the animation was in line with the performance capture performances.

The humanity of the orcs push them toward uncanny valley, but the monstrous size of the creatures push in the other direction. In the world of computers, almost anything is possible. I remember watching the documentary on The Phantom Menace DVD, which featured a scene where George Lucas was picking a performance from one actor and matching it with a different response from another actor, and digitally creating a composite shot of a take that never existed.

At the time that seemed crazy to me, but in the world of performance capture and CG-created characters, filmmakers have the ability to take the eyes from one performance and the body movement from another performance and the head movement from another performance, picking and choosing to create the exact perfect performance. But what ILM has found is this approach is not only costly but can sometimes result in what Duncan Jones referred to as the opposite of uncanny valley – a weird Frankenstein performance. So on this film they tried not to mix and match, and instead pick one performance from one take for each shot – you know, how things were accomplished before technology made things insane.

One of the biggest challenges in creating the orcs was their long braided hair. As you probably know, hair is something that has been a huge challenge for any CG production. So a lot of research and development went into getting it right. The result is a custom computer program created by ILM called, hilariously, Haircraft.

The simulation program was created to help with the extensive hair rendering in this film, and especially deal with the braids and dreadlocks of the Orcs. They could create tubes of geometry that the simulation would create hair within. As for the look of the hair, the production referenced beards that had a curl and character to them, using images of yellowing to also help inspire the look.

Not only did ILM need to design the hero characters, but they also had to make hundreds of background orcs who would be shown in the massive battle shots and sequences back at camp. They settled on five different orc clans, each of which would have a very different look so that the audience could easily distinguish them in a wide battlefield shot. Jones wanted to capture the sense of a real society, seeing not only the warrior orcs but also elders, peons, teenagers and more in the background of scenes.

Referencing primitive tribes, they were able to develop the horde's varied war paints, gear and accessories. The Frost Wolf clan, for instance, wears mostly light colored wolf furs, sporting blue war-paint. In this clan alone they designed 70 different background clan members. Another clan, the Bleeding Hallow, are from the woods, so all their weapons are made from the forest or pieces of leftover animal bones, all very low-tech. On the other hand, the Blackrock have a lot of metal pieces in their armor, with a color motif of black and red.

ILM also tried to take inspiration from the game lore whenever possible. For example, Kilrogg Deadeye famously sacrificed one of his eyes in battle, so they decided in the movie that his Bleeding Hallow would all honor the legendary orc by either removing their own left eye as part of a ceremonial ritual or sporting black makeup on their left eye to make it look as if they did. This is not a plot point from the movie, but backstory that was created to give this clan a history.

Creating armies of orcs was not easy, and to accomplish this ILM had teams of designers create many variations of helmets, shoulder pads, chest plates and wardrobe for each clan. With these designs they would then be able to create a catalog which would let them mix and match the library putting them on the 52 unique orc bodies they had designed to creature truly unique orcs. They even created a randomizing program and generated dozens of orcs, picking and choosing and making alternations when they could improve on them. They created a library of heads that they used across the models. In addition to the nine main characters in the film, each tribe has a chieftain, and in the end hundreds of orcs were created for the film's battle sequences.

At the end of the day, every orc was given their own unique detail, tattoos, war paint, brushing, hair, veins and more. Visual effects supervisors Christian Alzman and Nigel Sumner say they always treated the background orc characters as if they could be put into a crowd with hundreds if not thousands, but also be shown in close-up if needed. And we saw a few clips of shots that made the final film, which feature some of these characters in the foreground.

When asked if there was anything they couldn't do that they wanted to do, ILM visual effects supervisor Jeff White explained that everything that was envisioned was possible but there were things that would have required more time and money that would have been prohibitive for this production.

Apparently in the extensive third act battle, there were "loads of stuff" that was shot but didn't make it into the final film. These moments were described by Jones as cool pieces of action that didn't serve the need to move the plot forward. But who knows – Jones hopes that if people like the movie, they might be able render these extra bits for an extended edition. But don't expect a much much longer film like the Peter Jackson's Lord of the Rings or Hobbit extended editions.

ILM visual effects supervisor Jason Smith told me that some shots in the film were rendering for months on multiple processors. Because of this, Jones would approve the shots before they were rendered through the simulations, but it would be rare that he would get a final render and want to change it, only usually when lighting would change the intention of a scene or performance.

I feel like you get no sense of the artistry used to craft this film by watching the trailers, especially that one with the god-awful dubstep music. I'll be very upfront with you, I left ILM not having seen much of Warcraft. So I didn't come away WOWed (pun intended) at the film itself but the impressive artistry behind it.