Age of Ultron: How Industrial Light & Magic built a robotic killing machine


Joss Whedon’s Avengers: Age of Ultron cleaned up at the box office this weekend, scoring the second-highest opening weekend of all time (the original Avengers is number one). While audiences certainly want to keep up with the ongoing saga of Iron Man, Black Widow, Captain America, and friends, a large part of the appeal this time around is the titular villain himself: a wise-cracking robot who’s got humanity’s extinction on his mind.

Industrial Light & Magic’s digital models supervisor for the film was Bruce Holcomb, a visual effects veteran who’s been part of every Marvel film the company has worked on going back to 2003’s Hulk. Heading up the digital model shop, Holcomb and his team were responsible for creating everything from the bricks that Iron Man and the Hulk tear up from the city streets to vehicles like the Quinjet — but their most important work was on Ultron himself. I spoke with Holcomb to find out how ILM used Marvel’s designs and James Spader’s motion capture performance to create the metallic monstrosity.

Ultron concept art (ILM)

Bryan Bishop: Between Spader’s work and the various iterations he goes through, Ultron has a lot of different facets. Can you walk me through the process of creating him?

Bruce Holcomb: When we first heard about James Spader playing the role, it was extremely important to Joss that we give him a robot that can perform all the nuances they were capturing from the motion capture. Initially they sent us different takes of Spader kind of walking around in a circle and acting out some parts, and he has this kind of weird head-tilt thing that he does, that’s very unique to his acting style.

So when we first got the artwork [Marvel] had already drawn a guideline of what they were hoping for, from the menacing aspect of him. And then there was another wave of finalized artwork that came, that was supposed to be able to do all the nuances and expressionary things that Spader did, that we started the process on. …They gave us a really good initial model to start with, and then from there it was back and forth, pretty much all the way up until nearly the end of the film. I would say 80 percent of the design that was implemented by us here at ILM was in the facial area and then how he was going to move mechanically, based on the performance that Spader was giving.

"It was extremely important to Joss that we give him a robot that can perform all the nuances they were capturing."

So the task was almost two-fold. You had to capture the experience that he was performing and then the complexity of what they wanted to see with the robot, interior-wise. They kept mentioning a "watch within a watch"-type technology. …And then we implemented that, and we started putting him into shots, where there were many issues with his flexibility and stuff colliding. He couldn’t do a full range of motion like they were hoping for, so we had to solve a lot of problems here on our side. And then it kind of moved up to a fourth stage, to get everything to work together to where Marvel was happy.

Ultron concept drawings (ILM)

He evolves a lot over the course of the movie. How did those different designs develop?

The robot, the way it was described in the story initially to us, is that it was actually Baron Von Strucker’s robot. It didn’t have any Iron Man technology in it. So the first time that you see him is actually part of Tony Stark’s group of robots called the Iron Legion. And that’s that kind of spooky, wiry character that comes out when they’re at the party sequence. But then when he transfers his intelligence over to the other robot, which we called Ultron Prime, it’s a whole different robot not based on Tony Stark’s technology at all. And then he creates these things that we called "sub-Ultrons," which are basically the minions that run around and serve the only purpose of being blown up and destroyed.

Those were different versions of him, lesser versions, but we didn’t go back and forth between Ultron Prime and those versions at all. They didn’t share the same topology. They shared a similar look, but not any of the same forms. And then when he encases himself in vibranium at the end, it was almost like he was not really making new plates that fit on top of his plates, but he was exchanging his armor for this vibranium armor that had more of an elegant feel to it. In the end, to me, he looked like a knight in armor. … He’s bigger, stronger, scarier. He had a lot of points on him, like a claw feel.

He’s also pretty tall — certainly taller than Spader. How do you deal with that differential and make eyelines play?

You had Spader on an apple box for a lot of the shots, and then you had this guy walking out with a stick with a big red dot on the end of it where all the actors could look up to and act to. And then the rest was just filled in by us. There wasn’t a specific formula to how we pulled off his height exchange. Of course, I think in shot-specific stuff we’d scale him up or kind of just make him fit in the shot to make sure everything was jiving right. There wasn’t a real formula to it, so we just played that one by ear.

What are you favorite Ultron moments in the movie? Are there any particular scenes that you watch and say, "We really got that one right?"

I would say when he’s first introduced. I was particularly proud of how that was handled, because a lot of the times we’ll work on assets and we’ll really get to a point where we’re trying to show off a lot of the mechanisms and things that we build into him. But I felt Ultron’s main thing was that he was dark and kind of a scary character. So when you play him in the dark aspects of the film, where there’s not a lot of light — there was a lot of specularity to a lot of the detail that we made, that you can kind of catch. If you go back on different viewings of the film, you can catch how much complexity is in Ultron the more times you actually stare at him. The first time you’ll see it and go, "Wow, that’s pretty cool," and you’re watching the performance, but on different viewings, you can look at his interior geometry as he’s just walking around.

Avengers: Age of Ultron promotional still (DISNEY/MARVEL)

Ultron is this evil character, but he’s making jokes and dropping one-liners throughout the film. He’s very much a Joss Whedon creation in that regard. What were some of the early conversations with Joss about making sure he would have that kind of flexibility?

That’s a really good question. When I talked to Joss about what his intentions were with Ultron’s identity, his exact words were "he wants to be a schizophrenic robot with daddy issues." I never forgot him saying that, because I thought, "Boy, that’s going to be a hard thing to do." But I think he captured it, and I hope people get a kick out of it like we did when we were developing it.

So much of that is dependent on the robot really being able to convey Spader’s performance.

I like when he chops [Andy Serkis’] arm off. That was a facial performance where he looked in camera and said, "Ooh, I bet that hurt," or, "I think that’s going to be okay." There was a neat mechanism there that we toyed around with for quite a while. He’s got these cheeks that are kind of irises. And for the client, I don’t know that they really knew what they wanted to do with those, but there’s a certain profile, if you look at the comic book character of Ultron, where he looks like a metal skull. But of course, in the performance of him we didn’t have any negative space. So they had designed these irises that fit in the bottom parts of his cheeks that were meant to spiral open. But the only problem with them is that they were an oval shape, so you couldn’t get a full 360-degree kind of rotation as you would a regular camera iris. So we had to come up with this crazy formula of how these things would kind of spiral and also rotate up inside of his head to where they disappeared. And then you get this really good look inside of his mouth.

Ultron Face model (ILM)

There’s a ton of geometry in there designed for a power source that makes him have this red glow that travels throughout his entire body. But when those cheek irises opened and you could see inside of him, that was a big wow for all of us here, because that was definitely a point that they wanted to highlight. I don’t think they highlighted it enough, but if you take a look at the film again you’ll see it in a couple of shots. It’s a pretty cool thing.

Avengers: Age of Ultron is now playing. Ultron VFX images courtesy of Industrial Light & Magic.