Interview: Behind The Stunning Visual Effects of Kubo And The Two Strings

The Visual Effects team of Kubo And The Two Strings surprised everyone when the Oscars were announced by scoring a nomination for Best Visual Effects. It was nominated alongside The Jungle Book, Star Wars: Rogue One, Doctor Strange, and Deepwater Horizon. What made this nomination a surprise was it was the first animated feature to be nominated in that category since 1993’s Nightmare Before Christmas. Like the Nightmare before Christmas, Kubo and the Two Strings is not just animated, it’s a live animation, a mix of stop-motion and CGI using stunning visual effects to help create the grand, fantasy epic.

The Visual Effects team at Laika are in a jolly mood. Kubo,and the Two Strings is on a roll. It just won the BAFTA for Best Animated Feature. The stop-motion film has actors brought to life one frame at a time, shot on set, sometimes against a green screen, effects are added and software that you typically associate with that of a comic book blockbuster is what helped bring Kubo and the Two Strings to life. Water is made to look real, not through Houdini Software, but by using garbage bags, woodblock texture and animated by hand to create the scenes we see in the film. The same technique goes for creating the costumes and in order for us to see the end product.

The artists who worked on the film have been with Laika through Coraline, Paranorman, Boxtrolls and now Kubo.I caught up with the Laika Visual Effects team Steve Emerson – Visual Effects Supervisor, Brad Schiff – Animation Supervisor, Brian McLean – Director of Rapid Prototype, and Oliver Jones – Animation Rigging Supervisor from their studios in Oregon to discuss how this was bigger than anything they had ever worked on before and got an in-depth look at the technology they created to bring the magical epic set in Ancient Japan to life.

When Travis said that Kubo was your next project, not only was it going to be bigger, but it was going to be different than anything you had ever done before. What was your initial reaction?

Steve Emerson: For Kubo And The Two Strings, the scope of the film from the onset was going to be an enormous challenge. We had completed three films at Laika at that point and in terms of effects, monsters, characters, water systems, fire, and destruction, it was so far beyond anything we had ever accomplished. As is typical here at Laika, we are asked to deliver these films with similar resources from show to show to show. It’s very fortunate that with all the complexity of all these shows that it has continued to grow from film to film, and it’s certainly gotten better as a team here. We are more efficient, we’ve made mistakes together, we’ve learned from those mistakes together. We’re able to absorb a lot of that complexity. So, Kubo seemed impossible at the onset. The flipside of that is when we delivered the film, I don’t think I’ve ever been prouder. You get challenged with something that seems impossible at the onset, and to be able to see it through, and at this point now, we’re seeing that level of acknowledgement for that effort. It’s just incredibly gratifying because we are a small studio working out of a studio in the Pacific Northwest. Being acknowledged alongside these incredible filmmakers and studios with unbelievable and unlimited resources is incredibly gratifying.

Where do you start on a project like this?

Steve: It depends, we start with the script. From there, those become storyboards, those become animated storyboards, the art department is creating artwork, and we all get together as department heads in the conference room and we got through the film shot by shot problem solving how we’re going to create these visuals for the film. Again, that’s something we have gotten better at over the course of the last ten years. We used to sit in conference rooms for half a day going through a sequence trying to figure out how we could possibly create some of these things. Now, it’s only a matter of hours. We’ve got a better understanding of the process. We know the strengths and know when we need visual effects to come into the mix and help out.

Brian McLean: As Steve pointed out in what he just said, we’ve been working together for over ten years, we’ve all been working together in different capacities on all of Laika’s films and with that comes trust and respect for each other and our teams. Over the course of our last few films, we’ve built this tool chest of tricks and things that we solve problems with. What’s amazing with a project like Kubo, is looking at it, in hindsight, when we were sitting down to figure out how to build this stuff, it was a combination of, “Oh, we can try what we did on Boxtrolls, we can adapt this.” We were able to discuss these things, and throw out these ideas and each group was trusted to just run with it. When you’re doing films like this, so much of what we do will be built or engineered for a lot of these problems. We, unfortunately, don’t have the benefit of looking to how other studios have done it and be able to just cookie cut their process into ours, almost everything needs to be finessed and tweaked and reinvented for the medium of stop-motion.

What’s amazing is the problems of Kubo would have been impossible to solve, we did not have the technical capability or the know how, or the expertise to build Kubo five years ago. It took us working together on Coraline, ParaNorman, and Boxtrolls for us to get to this point where we had the ability, the knowledge, the resources, and the expertise to attempt something like Kubo.

Brad Schiff : To that point, any one of the things in this film, whether it’s the skeleton, the moon beast or our main characters, any one of those things is a monumental challenge, and a few films past, one of those things would have been all we would have been able to handle. I think our experience and trust together allowed us to take all those challenges head on and overcome those things.

Brian: Even the animation techniques and animation skill level of Brad’s team of the stop-motion animators, a few years ago, they would have been able to handle the monkey having a heart to heart with Kubo, but the advancement in their technique, their craft, and the improvements and subtlety they’re now able to put into these objects made the story that much more powerful. It made the audience members that much more connected to the story that was unfolding on the screen in front of them.
If the animators didn’t have the skills to be able to bring those puppets to life or it was done in some unconvincing way, then the story would not have been nearly as powerful.

Brad: At Laika, it’s a special place. There’s no other place doing one stop-motion film after the next like this. We actually have the opportunity for artistic growth. With my animation team, typically these projects are put out as satellite operations, everybody comes together and they finish the project and leave with good and bad tricks. Here, as we have a team, we have the ability to work with artists, and after each film, we can talk about what worked and things we want people to work on between films so that animators just get better and better. The bad tricks they take on to other jobs, we’re able to throw away and work on other things that build animators muscles. These guys are animating puppets, and people forget they’re watching puppets, people cry when they watch our movie.

It moved me to tears. I did see some of the puppets up close, but how does animation work relative to scale?

Brad: When we animate, we try to make it look as beautiful and as natural as possible. I don’t think it’s until you see the finished product, that when you see a hand that’s 20 feet tall on screen or a face that’s 40 feet wide, and in reality, it’s just an inch or two inches tall, we don’t even think about that. We try to make it look at good as we can.

Steve: One thing about this place, a lot of the decision making happens, subtlety the first thing I’ll always ask is,”What does the animator want?” So much about the way we work here is about facilitating animation and the best performances that we can get out of these puppets. I would say, even relative to scale, it comes down to the size that is working best for the animators to get these performances. It all goes back to getting the best possible performances.

Brad: On Coraline, we set the scale. On ParaNorman we hit the sweet spot on the size of the puppet and what was small enough to subtlety get what we wanted out of these characters, but not have sets that were too big to fit in this building. 7- or 8-inch scale had really become a sweet spot for us. The sets are big enough that the animators can get everything they need to get out of those puppets.

Oliver Jones: We built a toolkit around the animators and they have apparatus that helps them achieve the nuanced performance. They have a naturalistic style, The frameworks help them produce things like the strings on a marionette to help drive the puppetry into a more nuance and subtle round, compared to something that they would have been able to do purely by hand. There’s been mechanical advantages that we’ve been able to develop with the animators over the course of the four films.

Any development we do, and Steve hit the nail on the head, everything is about the animators. What do the animators need? How can we set them up for success? So much of that is driven by Laika CEO and Director of Kubo, Travis Knight. He’s an animator at heart, and there are so many decisions made to get the best animation possible. If this were done at another studio and you had a normal studio head, he or she would look at the bottom line and think it’s not worth the extra time or energy. Travis is always driving us to produce the best animation we can driving the puppet makers, the animators, the visual effects team, and so much of that is unique to Laika.

So many people have talked about creating the water and visual effects. Let’s talk about the technology:

Steve: Laika has made a name for itself since the beginning of our first film using 3-D printing for facial animation, we’ve slowly advanced that process from film to film. There was a crossroads on Kubo that was pretty daunting. As we started seeing the early artwork come through on some of these characters, there was a realization that the existing technology we had been using was no longer going to work and give the subtlety and performance that was demanding of both the character design and story. This goes back to what I was saying, there’s so much energy and support going into trying new things from the top down, even if it’s untried and not tested, that crossroads looking at the character design and the head of the department that was running the 3-D printing had to go back to Travis and decide whether to change the character design to fit within the mold that we had established, or were we going to try a new hardware that had never been attempted before.

Normally, when you have those discussions with studio heads, they’ll fall on the safe side, but both Travis and Arianne Sutner were all in and all supportive of us going down a path that had never been done before and trying a new technique. That meant there was no guarantee of success, we had to get unreleased hardware, there was extensive testing, we had to rewrite our own software, and the end result is as we hoped, to the audience members they’re not even aware they’re seeing something different. The monkey and beetle puppet were all done with this brand new technique. A few years ago they would not have looked the same or performed the same because at that time if we needed to make Kubo we would not have had the technical capability to do that.

How does a typical day start for you?

Brad: My typical day is different to everyone else’s because I sit in editorial with Travis as we launch the animators. We’ll walk in there and I’ll sit with him and Christopher Murrie. We’ll look at shots and rehearsals and blocks that came in from the night before. We’ll talk about that and what works, what doesn’t. To me, edit is my favorite part of the job. We’re talking about the performance and the characters, and for me there’s no better place. It’s very collaborative. The animator gets their brief, asks questions and they walk out of the room, I’ll follow up to see if they have any other questions. I’ll meet with Oliver and his rigging team. I’ll meet with Steve and his team and see if they have questions.

Steve: That morning session is how my day begins. I’ll get called over to see how to solve problems. A lot of that is going on across the departments.

Brad: It’s really about collaboration at the start of the day.

Oliver: The animators spill out of editorial and we facilitate their day to help achieve what they need to have. We go into the design and build mode. We have 65 individual stages going at one time, supporting two animators and there’s a lot of multiplicity happening at the same time.

Brian: I have a lot more facetime with Travis as we’re building and look at the functionality of how these things are going to move. After that, it turns into a mass production shop where once they’re built and approved, we are producing facial animation on a shot by shot basis and Brad and Travis will review the facial animation. They’ll work with the facial animators and give them notes, from there, we are churning out hundreds and thousands of faces and delivering them to set for the animators. I only get called into editorial if there’s a technical glitch.

Steve: The way we do visual effects here at Laika is distinctive. If you’re on a typical film production they shoot the footage and contract the VFX facility and everyone works with that footage. Here, we have production cycles that span multiple years. The stages will shoot, the VFX team starts working on them and doing the additional work. A great deal of my day is also working with that team. We’ll do dailies twice a day, we’ll look at shots in production and present them to the director two or three times a week until he or she feels they’re ready to go into the film.

Brad: We go over the facial animation and that’s a fun process that carries out from the very beginning to the very end. The facial performances need to be approved months in advance of the animator actually shooting the shot. It’s tweaking smiles and compression in the face to ensure we get that most naturalistic performance.

Steve: It’s really evolved. Back in the days of Coraline, we provide a really low-res computer render of faces moving to the dialogue. So much of that was about lip-synch. As we’ve progressed and gotten better, it’s really a refined acting class, so Brad and Travis are watching these facial performances and able to give subtle notes on getting the exact right emotion on the exact right brain.

https://www.youtube.com/watch?v=RGX_q_2Mo98