CASE STUDY

Client: Atomic Fiction

Need: Pushing the realism of facial replacement to the extreme

Author: Chris McMahon

Excerpt:

How do you transport Joseph Gordon-Levitt 1,300ft above the Manhattan skyline without putting him in danger, while also showcasing him as an accomplished wire walker with a lifetime’s worth of practice? When you’re working with Faceware on your side, anything is possible…

Could there be a more daring stunt than that performed by high-wire artist and daydreaming daredevil Philippe Petit on that fateful New York summer’s morning?

On 7am, August 7 1974, the Frenchman made his first steps out onto a 450-pound steel cable – one that reached out from one of New York City’s highest points to the other – the 1,300ft high 1 and 2 World Trade Centers.

For over 45 minutes this performer, madman, visionary – whatever you chose to call him – walked back and forth across the wire eight times, also taking the time to stop and dance, kneel, and even lay down, all while suspended a quarter of a mile above the stunned crowds below.

Extraordinarily, Petit survived, returning to the bottom and into the handcuffs of a disbelieving NYPD.

It’s a story of such wide-eyed wonder that it should come as no surprise that director Robert Zemeckis selected it as the subject for his 2015 blockbuster The Walk.

076_700_02_v010.01165Of course, with the World Trade Center’s Twin Towers no longer standing, sophisticated visual effects were required to bring the stunning act to life on the big screen. And yet it wasn’t just a case of meticulously recreating the seventies New York City streets in digital form. Other techniques were required to convince the audience that actor Joseph Gordon-Levitt was not to 1,300ft in the air, but that he was also capable of tightrope walking with all the capacity of a seasoned professional.

A little help from Faceware was required…

First Steps

The vast majority of shots in The Walk required some form of digital enhancement – some 672 out of 826 in total. The Oakland, California-based Atomic Fiction led the charge, delivering 250 of those challenging shots under the direction of visual effects supervisor and studio co-founder Kevin Baillie.

133_400_plt01_v002.01945Atomic Fiction was tasked with creating a fully 3D version of New York City, accurate to 1974 as viewed from that vertigo-inspiring elevation point. However, the studio also needed to create a fully digital double of Joseph Gordon-Levitt as Petit to integrate into these shots. In order to really sell the concept of the famous actor performing history’s most famed high-wire stunt, advanced face replacements technology was required.

“It was really important to Zemeckis that we always maintained the illusion of Joseph performing the stunts himself,” begins Rudy Grossman, CG supervisor at Atomic Fiction. “Several shots, which recreated Philippe Petit’s real-life stunts, could only be performed by an experienced master high-wire artists. To achieve this, we had to replace the face of the stunt performer with the performance of Joe’s acting.”

32 shots across The Walk required this process. “We had to take the footage of Jade Kindar-Martin, a former Cirque du Soleil performer turned stuntman, and overlay the face and performance of Gordon-Levitt,” explains Grossman.

A tough task indeed, but one made possible via the application of Faceware’s suite of tools:

“We’ve been using Faceware ever since working on Boardwalk Empire in 2011, and it was absolutely key to achieving the visual fidelity you see in The Walk’s facial performances today,” says Grossman. “Faceware’s performance capture technology gave us a solid foundation for staying true to the essence of the actors’ performances and timing – it was invaluable.”

Facing Reality

040_255_plt01_v001.01023“Because The Walk is so deeply rooted in reality, there’s no room to hide the details of visual effects in fantasy, or in the imagination of what a non-human creature might look like,” explains Grossman, reflecting on the challenges inherent in The Walk. “Everybody is unconsciously an expert on the human face – we spend most of our lives talking to each other, looking at each other’s faces, and looking at our own.

“On top of that the actor, Joseph Gordon-Levitt, has one of the most well-known and recognizable faces in the world,” he continues. “That was our challenge. We needed to convince an audience full of experts that our computer generated face was just the same as the face they already know.”

Along with this already considerable challenge, Atomic Fiction had to work under another demanding proposition – 672 shots might not seem much in a world of 2,000-shot VFX epics, but the lower count was due to The Walk’s much longer takes – an artistic choice designed to emphasize the reality and immersion of the big event.

So – total reality and greater scrutiny. A tall order indeed…

Face Off

In order to deliver the necessary face replacement results, Atomic Fiction’s process began on the last day of shooting The Walk.

On set, the team equipped Gordon-Levitt with Faceware’s Pro HD Headcam, and had the actor perform each of the takes that would require facial replacement. He did so while walking along a strip of tape stretched across the ground, upon which he mimed the act of wire walking without the added risk.

“We probably captured somewhere in the range of 30 minutes’ worth of data with the Pro HD Headcams, and processed what editorial felt was best suited to the face replacements,” explains Grossman.

Screenshot_RT_TheWalk“The process was really simple: Joe just put on the helmet, we hit record using Faceware Analyzer, and he acted,” he continues. “We applied a few ‘feature lines’ and makeup markers to his face in order to serve as reference to our animators, but the Faceware software didn’t really need them to work its magic.”

Once Gordon-Levitt’s facial performance had been captured, it was time to transpose the facial data onto the performance of stuntman Kindar-Martin using Faceware’s Retargeter.

“We would evaluate the performance of the face rig on the stunt double, and then adjust the animation as necessary, using the facial motion data captured with Analyzer as our foundation, and the helmet camera video footage as our reference,” Grossman explains. “Once the animation was approved, we ran out a few initial render tests in lighting, which informed any necessary fine tuning to the face rig.”

Using this captured and retargeted footage, Atomic Fiction could flawlessly place Gordon-Levitt’s face on Kindar-Martin’s body, creating the illusion that he was every bit as physically capable and experienced, without ever having performed the wire-walking actions himself.

A Successful Stunt

For the team at Atomic Fiction, Faceware was a blessing, allowing them to shave large chunks of time from the key frame animation process.

“Using Faceware completely removes the guesswork,” says Grossman. “The actor has provided the performance which the director has already chosen – as such, we know what the director wants, and we know exactly what the actor performed. So the time spent finding the right performance is instead spent making sure the digital face looks exactly as it should, and ensuring it contains the essence of the actor’s performance.”

Screenshot_AN_TheWalkElements such as pupil movement greatly helped the animators when working on the face replacements, as they could offset it to create the exact eye line that the VFX supervisor was looking for. “The animators would also often heavily rely on the initial width and height range for the mouth and brow movement, and then layer in animation from there,” explains Grossman.

Such advantages allowed the Atomic Fiction team to focus on pushing the realism of the facial replacements to the extreme – they could rely on Faceware to work on the core processes, and then have the artists target their energies on the final polish, perfecting the more nuanced details of the performance.

Walking The Walk

The Walk debuted in September 2015 to strong reviews, with effusive praise lavished on the visual effects and the impressive realism of Gordon-Levitt’s high-wire performances. Credit goes Atomic Fiction for their part in that accomplishment, all with that little helping hand from Faceware’s solutions.

“Before Faceware, the face motion capture process for feature film had traditionally involved a huge investment of personnel, hardware/software development, and capture space that could only be afforded by very large studios,” concludes Grossman. “Faceware takes all this and makes it possible for small and medium-sized studios like Atomic Fiction to work on lean-budgeted films such as The Walk.

“So, for us, Faceware was absolutely the right tool for the right job.”