Client: Digic Pictures
Need: Ground-breaking facial animation for gamers’ emotional immersion
Author: Chris McMahon
The world-famous Call of Duty series is synonymous with explosive blockbuster entertainment, but fans of the franchise know Call of Duty: Advanced Warfare is about more than just bullets and high-octane action. This amazing entry spins its narrative around the human element of war, and that required ground-breaking facial animation to achieve the necessary emotional impact in its cinematic cut scenes. Working under the leadership of Advanced Warfare’s development studio, Sledgehammer Games, it was up to Digic Pictures – along with help from Faceware Technologies – to deliver.
The long-running Call of Duty series has been well-established in the gaming canon as one of the most explosive, action-packed titles available on today’s consoles. Gameplay is an adrenaline-fueled rollercoaster ride, with levels rocked by screen-shaking explosions and only those with the keenest reflexes emerging from the other end of the combat.
And yet, for all its aspirations to deliver the most intense gameplay experience available today, Call of Duty has also reached for that other end of the spectrum over the years – quiet, introspective moments of emotion, where combat is traded for moments of poignancy.
Take 2014’s Call of Duty: Advanced Warfare. Following a death-defying escape from an enemy compound, characters Mitchell, Gideon and Ilona stand over the body of the badly wounded Cormack in the back of a flatbed truck, life ebbing from his body. There are no extravagant theatrics here: the drama comes instead through the subtle flickers of emotion that cross over each of the soldiers’ faces – the muted expressions of stern grief and grim determination. In a medium where death is often an unremarkable occurrence, here the passing of one character is filled with the weight and gravitas as the best Hollywood has to offer.
This moment of somber reflection – along with the nearly 30 minutes of Advanced Warfare cinematics – were created by the extremely talented Hungarian studio Digic Pictures working closely with Sledgehammer Games. With experience in video game animation that extends across some of the biggest blockbusters, Digic has built up a strong understanding of what makes for a great character driven sequence – namely, believability.
“This project marked the first time we had worked on a Call of Duty title, and the first time we had worked with developer Sledgehammer Games,” says the team at Digic Pictures. “That’s a lot of big firsts, so we definitely wanted to prove ourselves, especially on the facial animation front.
“By far the most important aspect of our brief was that Sledgehammer Games’ vision for Advanced Warfare was delivered: believable, emotive performances that really shine through, helping gamers engage more deeply with the story and the characters than ever before on a Call of Duty title.”
Even on the basis of that one scene alone, Digic Pictures achieved and surpassed those goals, creating a Call of Duty scene that resonates beyond the explosions. In order to create facial animation that expressed such nuanced emotion, Digic Pictures turned once again to the solutions provided by Faceware Technologies…
“For Advanced Warfare, the studio really wanted to make sure that the players engaged with the characters on a deeper level than ever before,” begins Digic. “The game has a great story that’s not only grand in terms of scope, but is also intimately personal. To convey this, we had to bring the players close to the characters, and the best way to do this is through truly emotive facial performances.”
After a period of extensive internal testing with Sledgehammer Games, Digic began using Faceware’s solutions in production in 2012. The studio has become deeply involved with the software, gaining a deep understanding of its intricacies and the potential it holds. As such, Digic was able to squeeze out every last drop of emotion captured from the on-set performances.
“For Advanced Warfare, the actors not only lent their voices and physical performances – both face and body – to the game characters, but also their actual likenesses,” says Digic. This means that the protagonist Mitchell actually looks like actor Troy Baker, and the same goes for all of the other characters in the game. This meant that the facial performance captured by Sledgehammer Games from the actors didn’t have to be reinterpreted for a different face, which would have a vastly different bone structure and vastly different volumes of tissue.
“Having this direct 1-to-1 correspondence made a huge difference right from the start downstream for us, as it meant we also had perfect reference material of the actual actors performing their lines, which made it very clear what we had to achieve in the game cinematics.”
Each of the game’s actors was fully scanned, with dozens of FACS (Facial Action Coding System) facial poses recorded. After a significant amount of processing and retopologizing – the process of modifying a model’s topology for animation – these FACS could be combined with Digic’s face-rigs to form complex facial animations.
“That’s where Faceware came in: to help drive our face rigs,” says Digic. “The head-mounted camera footage we received on-set was processed using Faceware’s Analyzer, and then applied to our face rigs with Retargeter – Retargeter is 100 percent compatible with our custom face rig, and that means no compromise is required at all when we use Faceware products. It works perfectly, capturing every tic and nuance present on the actors face, and all that feeds into the emotional impact of the cut scenes witnessed on the screen.”
For the facial elements of Advanced Warfare’s cinematics, working from Sledgehammer Games’ vision, Digic created approximately 17 minutes of content – multiplied by the 3-4 characters appearing on screen at any one time – which made for 1 hour of high-quality content that required processing. Even for the most talented keyframe animator, the fidelity required of the facial performances was beyond the reach of human ability alone. Faceware’s solutions were required to help Digic overcome half the battle.
“Unlike our usual projects, for Advanced Warfare we organized our team into smaller specializing units to create a more efficient workflow: one unit processed the captured footage in Analyzer, another unit worked with Retargeter, while the rest of the team worked on keyframe animation.”
As ever, the analyzed on-set footage from Sledgehammer Games got the Digic team some of the way, but it was up to the deft hands of a talented animator to bring the footage to the final quality witnessed in the finished cinematics. However, Faceware’s tools were powerful enough that the original analyzed footage was far enough along for the animators to push it even further than they would have otherwise – and that meant achieving an even higher level of final quality than would have been possible otherwise.
“After using Faceware after a quick retarget phase, the character would suddenly come alive, with the eyes and muscles moving in believable rhythm,” explains Digic. “Simply put, we could achieve basic timing and animation much more quickly and accurately than we would by hand, which left us more time for detailed fine tuning. Faceware had already made enough of an impact that we could just spend more time really focusing on delivering the details.”
“In general, Faceware’s solutions reduced the first part of our animation workflow by 60-80 per cent. Without this important first step, the facial animation wouldn’t be effective, no matter how much detail work went into it.”
In some cases, the Digic team also found Faceware’s results were good enough to be used as is. “There were certainly some shorter scenes that didn’t even require that additional keyframe animation work, such as on background characters, or in certain dynamic action shots. In those cases, Retargeter did enough of the work on its own.”
Part of the reason this high fidelity of facial capture was made possible was the custom development carried out by Digic, in close collaboration with Faceware Technologies, on its facial capture solutions.
“Advanced Warfare presented a real challenge with hard deadlines, so we needed to squeeze as much of that initial information out of the recorded footage as we could,” remembers Digic. “In order to achieve this we had a discussion with the Faceware team about using their software in a different way. The result of this was that the team gave us an option for tracking new parts of the face.”
In the stock Faceware solution facial performance is tracked in three separate groups: the eyes, the brows and the mouth. Digic Pictures was looking for an approach that would capture even more of the actor’s performance, attaining the kind of precision and detail we see in the cinematics.
“Rather than just track the inner relative movements of the face and then re-target them onto given controls on the face rig, we also isolated two new parts: the jaw and the cheeks,” explains Digic. “It was Faceware who made it possible for us to bind these elements to our system, and that was integral to getting that really accurate animation in Advanced Warfare cinematics.”
Adding the jaw and cheek markers helped to really bring out the subtlety required of the character’s performances, as it enabled the team to represent facial motions that otherwise would not have been trackable: “Independent jaw tracking is a very important development, as it allows for handling mouth/lip movements separately from movements of the jaw,” explains the Digic team. “This is important, because it means that actions which happen at an offset from each other can be tracked accurately. An example of this would be when the jaws begin to open yet the mouth itself is momentarily still closed, due to the lips being stuck together for a brief moment. Or, in the case of an M or a B sound, when the jaws are not closed, yet the lips are pressed together.”
This solution has proved so successful that it has now been adopted by teams on various other projects, helping them to also attain a whole new realm of realism.
“The new solution worked really well, and that’s all due to the fact that Faceware was so responsive and helpful with the custom development work,” says Digic. “We could use the new solutions right from the beginning of the project and really hit the ground running.”
Power Changes Everything
The result of the mo-cap sessions, analyzing and retargeting, and the custom development work to get even more out of those initial captures, are moments like that witnessed in the back of a flatbed truck – moments where a glance from one character to another can pack as much of a punch as a .32 slug.
And then there are the moments of soldiers sharing a beer and celebrating a mission gone well; an interrogee falling for the dangerous charms of his interrogator; an analyst staring at a computer screen, wondering if his operatives have fallen to enemy forces. With the help of Faceware, Digic Pictures realized its client’s vision of an emotively driven Call of Duty with great skill, and the results are there for all to see, fitting into studio Sledgehammer Games’ grand vision and design for Advanced Warfare and complementing the thrilling gameplay experienced by gamers.
“Advanced Warfare was a great project to be involved in, and the end result is an incredible achievement,” concludes Digic. “It was the result of true teamwork – the next level indeed – and we greatly looking forward to using the Faceware tools again in future.”