Client: VRWERX

Need: Fast turnaround on terrifyingly lifelike facial animation for VR

Author: Chris McMahon


VRWERX is spearheading a new wave of immersive VR horror with Paranormal Activity: The Lost Soul – and with the Faceware ProHD Headcam and software Creative Suite in its VR pipeline it’s elevating this terror further, enabling a fast turnaround on terrifyingly lifelike animation…

Since 1996’s Resident Evil, video game developers and players alike have grown enthralled by the potential of interactive terror. Yet over the decades these survival horror experiences have been trapped within the confines of monitors and television screens.

Now, the horror hardcore can place themselves inside such experiences, exploring virtual game worlds designed to deliver dread to those willing to don a VR headset. Thanks to the HTC Vive, Oculus Rift and PlayStation VR, survival horror is no longer trapped behind a screen; it’s wrapped around the player.

VRWERX is one of the early pioneers in the emerging VR horror space, having recently teamed up with Paramount Pictures to develop Paranormal Activity: The Lost Soul; an immersive gaming experience inspired by the popular horror film franchise.

In order to deliver big scares while also affording the ability to iterate quickly within a VR pipeline, VRWERX relies on Faceware’s ProHD Headcam and Analyzer and Retargeter softwares.    Using this workflow, VRWERX conjures apparitions realistic enough to scare players witless.

Endurance Horror

Based in the heart of Los Angeles, VRWERX delivers VR experiences from concept to distribution, powered by some of the best and brightest innovators in the 360-degree entertainment space. The studio filmed the world’s first feature-length VR movie at the end of 2015, and followed this up with the Paranormal Activity: The Lost Soul experience.

And quite an experience it is. In it, players explore an abandoned, gloomy house. As they progress, various tricks are employed to up the tension – mirrors shatter, lights flicker, and doors thud – all leading up to the inevitable jump scare climax: specific head movements cause animation triggers, with humanoid creatures launching from the shadows towards the terrified player.

“It may be an overused term, but The Lost Soul really is a visceral experience,” begins Bradan Dotson, VRWERX production manager. “Horror inherently lends itself well to VR – there’s a diehard fanbase that really enjoys the adrenaline rush and masochistic thrill. The Lost Soul is an extension of that thrill-seeking experience.”

The Lost Soul is arguably less survival horror, more endurance horror: how long can players tolerate the intensity of the experience as they wait to be confronted by one of the spectres prowling the environment? Once that confrontation happens, the scares delivered are big – thanks in part to the support of the Faceware offering.

Engineering Fear

Faceware’s solutions have been used across a wide variety of digital entertainment products, from NBA 2K titles to the latest in the Call of Duty franchise, each time ensuring that the digital faces we see on screen feel completely convincing.

In The Lost Soul these facial animations swoop by in moments of sheer terror, striking fear into the heart of those who dare to look. A realistically contorted face is key to that, and that’s what VRWERX delivers with Faceware.

Our FAV reaction video EVER!!!

Characters will interact with you in a traditional game, but when you’re immersed in a 360 virtual environment and they’re right up to the screen, you notice a lot of the details that you wouldn’t otherwise,” says Dotson. “We needed the facial expressions and emotions to be really on point to make sure those performances came through, driving fear into the heart of the player.

VRWERX leverages a number of Faceware products to deliver on these gruesome visages – specifically, the Faceware ProHD Headcam, which is worn by actors when performing the role, and then the duo of Analyzer and Retargeter with their animation team.

Analyzer draws upon Faceware’s sophisticated back-end to capture every nuance of the facial performance – all without the need for markers. From there, Retargeter takes the captured data and applies it to the facial rig, bringing the actor’s performance to life in the game world.

“This process is a great way to accelerate the facial animation pipeline while still capturing detailed facial performances,” explains Dotson. “Running everything through the Analyzer and Retargeter results in great translation right off the bat, which is extremely helpful. In some cases, it allows us to jump straight to the polishing stage, minimizing the amount of key frame work required.

“When you’re working in VR that ease of use is so vital,” he continues. “The player is going to be immersed to the point where they feel like they’re actually standing next to this character. You don’t want to shatter that, so quick developmental iteration is key. Faceware’s solutions allow us to do that, creatively exploring the most terrifying options for the scene.

“We can deliver the most effective final result, all within a complex VR workflow.”

Scarily Good Support

Paranormal Activity: The Lost Soul is an ambitious project, and that comes with its fair share of challenge: not only is immersive virtual horror a new playground without precedent, but developing across multiple VR hardware sets posits a litany of technical hurdles.

“We’ve worked hard to ensure we’re building the game that presents a consistent experience no matter the hardware,” says Dotson. “Working with Faceware has certainly made that an easier experience, providing simple, effective workflows that get the job done. That affords us more time to address the difficult technical details.”

In fact, the support VRWERX has received from Faceware has made for a much smoother development experience overall.

We were truly lucky to have Faceware as a partner on The Lost Soul, especially as it was our first experience using the technology,” says Dotson. “The team came in and supported us throughout the mo-cap process, guiding us through exceptionally quick and seamless setup process. From that point on Faceware was always available to answer any questions we had – they were a wealth of knowledge.

The result has been swift development, fast results, and more scope for creative iteration. And all of that combines to deliver the most important element of this production: really big scares.

“Anything that makes VR development easier and leaves us more time to think about creative end is very welcome,” concludes Dotson. “In that regard, Faceware really did set us up for success.”

Faceware and the Virtual Reality Pipeline

With virtual reality growing more in demand by the month, so too are faster and more streamlined development solutions. Faceware’s Creation Suite Pro Complete System is perfectly positioned to help studios meet the challenge. VP of Business Development, Peter Busch, explains why…

Why is believable facial performance important in virtual reality?

The key to an engaging experience – especially in a new medium such as VR – is connecting users with a powerful story by drawing upon the emotional, character-driven, and nuanced performance from the faces of the characters. Players need to feel emotionally absorbed in the experience as they’re so absolutely immersed in it.

How does Faceware help deliver believable performances in VR?

Using Faceware ProHD headcams and Analyzer 3.0 and Retargeter 5.0 software, studios are able to capture the entire facial performance, including micro expressions, and display those subtleties in convincing ways.

We believe the best facial animation comes from the combination of cutting-edge technology and an intuitive artist-friendly workflow. Our decades of facial motion capture technology research and experience has created a streamlined and flexible workflow – one that enables teams to scale and create high-quality, high-volume animation, all while staying within budget and maximizing artists’ creative control.

How seamlessly does Faceware fit into the VR pipeline?

Our facial motion capture products have been battle tested on hundreds of published projects across six continents. Throughout this time, we’ve designed them to seamlessly integrate into any industry pipelines – including VR. Our 15-year roots as a service company mean you can trust our core product methodology, which respects the dynamic pace of any production thanks to a robust, artist-friendly feature set; our products are designed to be intuitive and get the job done at the quality you need.

Our head-mounted camera systems capture optimal video for use with our facial motion capture software that plugs in to most popular character animation packages used in VR, such as the Autodesk suite of animation software.

Our technology also easily syncs with any body mocap system, while the facial performance capture technology allows for markerless performance recording. Our pixel-based approach samples and tracks the entire face, including subtle eye movement that gives soul and emotion to any character. We have an intuitive pose-based workflow that most artists refer to as their “assistant animator.” The software features allow artists to easily manipulate, change, and embellish the results with ease. All of this feeds in seamlessly to a VR pipeline, resulting in truly immersive, emotive experiences.

What steps should a developer should take when using Faceware within a VR pipeline?

If the developer is familiar with Faceware, then facial animation for VR will be similar to what they already produce for mobile, desktop, or console games. What changes is the user perspective and 3D environment – the developer needs to take that into consideration when developing the game. However, the ultimate output of Faceware’s technology is always facial movement, which remains consistent throughout.

Are there any tips or tricks that can help the Complete System better integrate into a VR pipeline?

Complete Systems easily integrate into a VR pipeline without issues. However, it is important to hire talented actors to draw upon their acting skills in imaginary worlds like VR, delivering the best facial performance possible.