Interview: Jeff White’s Visual Effects for The Avengers

Jeff White joined the legendary LucasFilm visual effects company Industrial Light and Magic roughly ten years ago, and has since worked on some of the biggest blockbusters of the past decade. His work spans all three Transformers movies, Pirates of the Caribbean: Dead Man’s Chest, and Star Wars: Episode III “Revenge of the Sith.” During the course of those films and many others, White has worked his way up from his initial post as a Creature Technical Director on Van Helsing to being Visual Effects Supervisor on last year’s The Avengers. The adaptation of Marvel’s superhero world from acclaimed filmmaker Joss Whedon (Serenity) has already become the third-highest-grossing film of all time and received near-universal acclaim. The Visual Effects branch of the Academy has recognized White and his fellow visual effects artists with an Oscar-nomination for Best Visual Effects for the work on the film, and I recently spoke with White in celebration of his first Oscar nomination. Here’s what White shared with me about reimagining The Hulk, bringing his own ideas to light while carrying the banner of existing franchises, and assembling the team of The Avengers.

Jackson Truax: You’ve worked on so many massive special-effects films in the past ten years, but this is your first Oscar nomination. What does the recognition from the Visual Effects branch of the Academy mean to you at this point in your career?

Jeff White: It’s pretty incredible… I’m just so honored to have the work recognized… There were probably thousands of artists working on the effects. So the nomination for Best Visual Effects is really a recognition of all the great work that people put into the film.

JT: Like many craftsmen in the film industry, you seem to have climbed up the ranks, spending a lot of time as a Creature Technical Director before becoming a Digital Production Supervisor and then a Visual Effects Supervisor, with some other titles along the way. What was that journey? And how did it lead to you working on The Avengers?

White: When I first interviewed at ILM, they said, “Do you want to be a Creature Technical Director? Or do you want to be a Technical Director, which is more focused on lighting?” I said, “Probably a Technical Director.” They said, “Well, we’re only hiring Creature Technical Directors right now.” I said, “That sounds good…” It was a great place to start in the industry. Because you’re right in the middle of the pipeline. I started off doing a lot of character rigging and simulation. It turned out, many years later, doing The Hulk, all of that became incredibly relevant. One of the things I love about ILM, is there’s so much variety to the work… I worked on a film, xXx: State of the Union, with [Visual Effects Supervisor] Scott Farrar. He ended up giving me a couple of great opportunities…The Chronicles of Narnia: The Lion, the Witch, and the Wardrobe [and] Transformers. We did all three films together… When I heard about The Avengers and I heard Joss [Whedon] was going to be involved, I really wanted to work on the project…and was able to get involved in it. And I feel really lucky to have been able to work on it.

JT: Film critics and audience members alike might refer to The Avengers as being an “effects-driven film.” Does that description feel accurate to you? Were your effects designed to drive the story and the narrative, or did you always feel like they were in service of something greater?

White: I would argue that it’s actually, really a story-driven film… The reason I was so interested in it was because of Joss Whedon and how focused on story and great characters he is. I’ve been a huge fan of his work…in television, and Serenity. The thing that amazed me, was that even in the midst pulling off a film with 2,200 Visual Effects shots, which is a huge undertaking, he was able to really retain the focus on the story and keep the effects in service of this great story he was telling. And that really permeated every decision that he made… In every film we’ve added new things to Iron Man. On this one, we added a rocket pack. It wasn’t just to add a cool new effects gadget on him. But it actually served the purpose that now his hands were free to be pointed straight down. It allowed him to achieve new poses and to do new things in the film. So really, every decision in the visual effects work was really driven by the story and great characters.

JT: You came on to The Avengers without having worked on any of the other recent films set in the Marvel universe. Did you feel any responsibility or desire to create effects that were similar to the previous films? Or did you want to create something audiences hadn’t seen yet?

White: Even though I didn’t personally work on them, ILM had done both Iron Man and Iron Man 2… When I started into this project, even though there were a lot of things that were new like The Hulk and New York City and the alien race, the Chitauri, when we needed to create the Iron Man suit, I had the same modeler who had made all the previous suits for the previous movies. So there was a great wealth of knowledge and history with the Marvel properties already and the artists at ILM. But at the same time, I think The Avengers was an opportunity for us to really tread some new ground, in building the Helicarrier and creating The Hulk. And in creating virtual New York City.

JT: Tony Stark is a character that audiences are very familiar with. Inherent in his being, even without the suit as Tony Stark, is an effect. What challenges does that create, as far as designing something that needs to be consistent throughout the movie and fit in seamlessly with the narrative and the character?

White: It’s really challenging. I may have been at the beginning, at little too [underestimating] of how difficult Iron Man was going to be because we had done two films with him before. It’s really hard. It was a lot of fun creating a new Iron Man suit, especially in the way that he takes it off and puts it on. [We] came up with this great sequence where now, taking the suit off, he can just walk naturally. And the suit-down machine will take it off around him. Sequences like that, or when he puts it on while he’s falling down the side of the building, those are a lot of fun to figure out. And really provide some new challenges and let us explore who this character is even further than had been done in the previous films.

JT: The character that feels like he was really redesigned for this film was The Hulk. What were the ideas about character design that Joss Whedon brought to the character and how did you implement them in designing the effects?

White: Joss made some pretty significant changes to The Hulk that really benefited [the film]. Number one was absolutely casting Mark Ruffalo… He’s been in all kinds of great work. He did an incredible performance for [Bruce] Banner. Which was a key part of creating a believable Hulk. At the same time, he was really willing to do any part of the visual Hulk character and make him successful. When we started out, Joss had made a pretty big change [in that] this was the first Hulk where you could actually see the actor that played him incorporated into the design. So, what’s nice about that is that we were working with Mark Ruffalo, instead of trying to just solve all the problems in computer graphics, we started by creating a digital version of the actor. We did a live-cast of his skin pores. And shot all kinds of photography to match his eyes. Solving all those problems in creating a digital Mark, I think made The Hulk a lot more convincing. What I wanted to show with The Hulk and break new ground on was showing we can take an actor who’s very prolific, done a lot of independent film work and different projects, and the technology’s progressed so far now with Motion Capture effects, he can really provide the soul and the performance to The Hulk. At the same time, there’s so much great animation that is then is applied with that performance and inspired by it to get it over the finish line.

JT: In the final film, how much of The Hulk was animated, how much was computer-generated, and how much was Motion Capture?

White: Any time you see The Hulk, he’s computer-generated. For most of the shots where he’s giving a performance, maybe it’s dialogue, or interaction with another character, that’s Motion Capture. Anytime he’s doing something that’s completely out of the realm of physicality for Mark, where he’s jumping from building-to-building, that’s straight-ahead Key Frame Animation. But even there, we would show Mark the shots of him jumping from building-to-building. And he would give us a performance of facial animation, straining and flexing, and giving us all this great reference to work from. I think the success of it really comes from that combination of driving everything with performance and having great animation to supplement that. I think the last step of it is, everything you see on The Hulk, even though the skin is green, that’s Mark Ruffalo’s skin. Those are his skin pores. Any blemish we found on Mark we incorporated into The Hulk. The same is very true for the eyes. Even though they’re tinted green, it’s all built around duplicating Mark Ruffalo’s eyes. Instead of just making it up, by basing it on something in the real world, that was a huge advantage for us in terms of getting it to look realistic.

JT: Whether it’s a big battle or a scene with the Avengers on the Helicarrier, what are the biggest challenges of designing a scene with a group of characters, and having each one be of a different size or scope?

White: It’s a real balancing act. That’s where I think a lot of the praise of the movie, and to Joss, and rightly so, was given. In that he was able to balance screentime and character development for each of them. It was interesting on the film, when we have several shots, there’s one in particular where the camera circles around all the Avengers. It’s a very signature hero shot of them assembled on the New York City viaduct. There are some practical problems, in that you’re trying to compose a frame and one of your characters is eight-and-a-half feet tall. It makes for kind of an uncomfortable frame. So we had to do things like…get The Hulk more hunched over and tense and ready to coil in attack. That ended up working nicely as far as getting his head a little lower so that he could fit in the frame with everybody else.

JT: What was the process of designing the Chitauri? How do you go about designing a marauding alien race, when aliens have already been seen on screen in almost every imaginable permutation?

White: We love creating alien races. Because it’s always an opportunity to develop new characters. Down to thinking about, “How do they move?” And “Do they move in packs?…” We incorporated all these weird sort of kicks and bird-like movements into their heads, just to create something new and something different… They’re transported to Earth on these giant Leviathans. Those were great opportunities for bringing this winged character into New York City that has wings that are too wide to fit down a city block. So the whole time they’re flying, they’re ripping through buildings and causing all this destruction, which helps us integrate them into the [story] and is really fun work for us to do.

JT: When audiences think of The Avengers, the first thing that comes to mind is the great battle scene at the end in New York. What were the biggest challenges of creating a grand finale, a battle royale, if you will, with so many characters on such a massive scope?

White: For very practical reasons, we weren’t able to shoot a lot of the end of the film in New York City. So many of the shots that you see where they’re standing on the viaduct or Iron Man’s flying around the city, those were all created digitally. It was one of those effects that we really wanted people not to notice, and for it to really disappear. In the end, there were only three days in New York City. The rest was either a dress set in Cleveland, or a New York City that was entirely digital… On top of that, we wanted to really feel that they were in the middle of complete battle chaos… We added lots of smoke and dust and ash and paper and floating embers and burning fires, all that textual war-movie [stuff] to really punch-up the frames. A lot of the fun was figuring out, “How were the Avengers going to fight the aliens?” There’s a great scene that we call the tie-in shot. It’s basically a shot where you go from Avenger-to-Avenger as they help each other. And you fly through the city as one continuous long-shot. You see all the different ways they can work together, where Iron Man shoots his RT Blasters off of Captain American’s shield to take out a bunch of aliens. Coming up with stuff like that and working with Joss on that portion of the battle was a lot of fun.

JT: The Avengers is now the third highest-grossing film of all time, having been embraced by critics and audiences all over the world. Whether it’s from audiences, critics, or industry peers, what feedback have you received of which you’re the most proud?

White: My kids really enjoyed it. Which is, of course, a high watermark for me. They went to the film opening night, and saw it and really loved it… We really wanted to make the fans happy. Everybody poured a tremendous amount of effort into this project. Especially with a character like The Hulk, who had such a huge following around the world. We wanted to make sure that people were really happy with the Hulk that was created. That he was a very believable, tangible character. Joss gave us all these great moments for him that were really scene-stealing, smashing Loki back-and-forth on the ground, or punching Thor. He was able to bring a lot of comedy to the film. There’s nothing more gratifying that creating a digital character that gets that kind of emotional response out of the audience.