When it comes to visual effects, the popular refrain over the past few years has been pretty simple: practical effects good, computer-generated effects bad. But the truth is infinitely more interesting, as movies have largely stepped out of the uncanny valley of problematic CG, and entered an era where digital effects are so good, audiences don’t even realize that’s what they’re seeing. (Did you notice Kylo Ren had a digital helmet in that one shot from The Force Awakens? Yeah, neither did I.)
The latest film to put effects to the test is Marvel’s Captain America: Civil War. The movie’s got the usual flashy appearances from Iron Man and the gang, but the centerpiece is an all-out brawl at an airport where two opposing factions of the Avengers face off — in a shot the visual effects team called “the splash panel” — before all hell breaks loose. Spider-Man, Black Panther, and a 50-foot tall Ant-Man all get in on the action, and while it all seems to be seamlessly integrated with footage shot on a physical location, what audiences actually see was created almost entirely at Industrial Light & Magic.
"The airport is a hundred percent digital," ILM visual effects supervisor Russell Earl tells me over the phone. "Spider-Man, Giant-Man, and Black Panther are always one-hundred percent CG. Iron Man, War Machine, and then we’ve got Vision." The exceptions were characters like Black Widow, Captain America, or Winter Soldier — basically, the few people not wearing a mask of some sort. But while directors Joe and Anthony Russo did shoot at an airport in Germany, only a handful of principal cast members were there — with the majority of the live-action footage shot against a giant green screen in Atlanta, Georgia and assembled by the visual effects team later.
From the beginning, the creative mandate from the Russos was to build upon the grounded look and feel they’d established in their previous film. "Winter Soldier, for all intents and purposes, could have been a ‘70s thriller, or a Bourne movie," Earl says. "Getting to Civil War, it was, ‘How is that going to work on this one?’ Because, obviously, there’s more superhero-y characters, with Vision or Wanda. It was thinking about Spider-Man; how do you take that [character], and keep that same sort of real-world feel?"
To achieve that stylistic goal with the airport environment required dispatching a team to the airport location in Leipzig for a week of recon. "We shot thousands and thousands of pictures. We would go out there with LiDAR scanners, which basically provide an exact representation of the geometry. And then we take that, and we rebuild everything." The same went for every car, plane, and vehicle on location, resulting in a digital duplicate that the filmmakers could tweak, augment, or use as they see fit. "We had six hundred-plus assets, and you could be anywhere on that airport," Earl says. "From when you’re with Ant-Man on that stair car at a half-inch scale, anywhere to when he’s 50-feet tall, the digital environment would hold up."
While the digital airport would be considered a success because audiences will never notice it, the stakes were slightly different for Marvel’s new take on a fan favorite: Spider-Man. When photography on Civil War began, it hadn’t been decided how the character would actually be realized, and a camera-ready Spidey suit was built for the shoot. (A parkour artist played the role during shooting, as actor Tom Holland hadn’t even been cast yet.) When the decision to go all-CG was made halfway through the shoot, it fell to Earl and his team to create the definitive Spider-Man.
The Spider-Man suit is completely CG, in every single shot
"Our suit had to look and feel real," he says. ILM and the Russos collaborated with Marvel’s own visual development group and Marvel Studios head Kevin Feige to workshop how the character should look. "We did a lot of tests and studies to hone in. The suit is designed by Stark, so he’s sort of one-upped it a little bit — so how do you bring the Stark tech look into that suit? And obviously, we wanted it to feel like real world materials, and not do anything that was out of that realm. It needed to feel like it was a photographed suit."
To bring it to life, they used a layered process: a cloth simulation handled the fabric, which ran atop a muscle simulation that provided the look of Peter Parker’s body. (Body scans from Tom Holland served as the basis.) The character was animated like any digital creation, but in Civil War, there’s a teenaged kid inside that suit, and it was up to Holland to provide the performance. The actor performed every line, moment, and beat in the film, captured by motion-capture and reference cameras, with those performances then integrated into ILM’s animation. And to avoid the stiff, inexpressive look that Spider-Man’s face has had in some previous film incarnations, the visual effects team devised a version of the mask that stretched when he spoke. "How much does the fabric slide, how much does it stretch, how much do we see his jaw motion? Adding intricacies, like the camera irising to the eyes themselves, so we could get a little bit of movement in the eyes. There’s a lot of little subtle things that all add up to making him feel like he’s there, and part of the team."
A similar process was used for Black Panther, a character whose scenes had all been shot with the full intention that a stunt performer in a suit would end up in the finished film. ILM had planned to just do some touch-up work and wire removal on the photographed suit, until Marvel decided that Black Panther’s proportions should look a little more… hero-ey. "We changed the bulk of his chest. We made his head a little bit smaller, because obviously, if you put a helmet on a guy it makes their head bigger than you want them to be." At that point it simply made sense to go all the way, leading to a late-in-the-game call to make Black Panther a completely digital character as well.
Given how intensive digital effects work has always been, it seems counterintuitive that the most practical solution in these cases was to simply go with digital replacements and environments. But combine the logistical constraints of production, the schedules of 12 different actors, and the massive amount of small digital tweaks that already happen all the time, and things come into focus. "We had one shot with Bucky and Captain America, at the real airport, in a real place, running towards us," Earl laughs, "and we ended up rotoscoping them off of that background, and replacing it." It would have actually required more work to modify the real shot to match the purely digital ones, he explains — an inversion of how we’ve often thought about effects work.
Despite that, and the ever-increasing amount of digital effects that go by, unnoticed, in everything from comedies to TV shows, Earl stresses that there is still incredible value in shooting real actors, on real sets, even if that imagery never makes it into a finished film. "It forces decisions. You build the set, you can photograph the set, and you know the look and feel of it. That is why we went and photographed the airport and shot a bunch of reference. Something real to base it off of is huge." The same goes for performances. Even though the 50-foot high Giant-Man was totally digital, it was the footage of Paul Rudd playing the scene — Earl describes it as "kind of like a big, drunk baby" — that provided the direction for the animators to follow.
The end result, he says, could be digital work that’s so thoroughly convincing that not even the people making the movies themselves can tell the difference. "Maybe I am just fooling myself, but I think a lot of the time not everyone knew exactly what they were looking at," he says. "I don’t know if everyone knew that the airport was always digital. I think by the end they caught on, but that was our goal: that you wouldn’t really know what you were looking at. In a good way."