Client: North Carolina Aquariums’ Pine Knoll Shores

Need: Empowering educators to inform while entertaining

Author: Chris McMahon


Static exhibits are growing increasingly obsolete; today’s installations often harness interactive technology to deliver content that entertains as much as it informs. That’s precisely the thinking behind North Carolina Aquariums’ Caretta’s Cove exhibition, where Faceware Live empowers educators to produce a live, dynamic character animation in total real-time…

Installations and exhibits pose a different proposition today to that of ten years ago. A wave of new technology has enabled new, exciting methods of dynamically delivering content to audiences in ways rarely never experienced before.

Case in point: North Carolina Aquariums’ Pine Knoll Shores, which has leveraged such new technology to create Caretta’s Cove; an interactive exhibit designed to teach children about the threats faced by loggerhead sea turtles.

In the exhibit, Children learn by conversing with Caretta herself: a CG-animated turtle who swims across an 80-inch screen and answers questions on the spot.

This live interaction is enabled by an educator behind the curtain, who performs Caretta’s facial movements via mocap solution Faceware Live. Faceware Live translates the nuances of the performance onto the animated turtle in real-time, enabling the actor to deliver an important message while simultaneously capturing the audience’s imagination.

The Birth of Caretta

Pine Knoll Shores first envisioned the Caretta Cove exhibit in 2011, inspired by similar installations such as Robert the Zebra at San Diego Wild Safari Park and Turtle Talk with Finding Nemo’s Crush at various Disney amusement parks.

In order to make the concept a reality in 2016, the aquarium harnessed the talents of an exhibit designer, who researched facial capture software and hardware solutions that could power the installation. Faceware Live was the answer.

Faceware Live is truly best solution out there, the most reliable, and the best price for indie development,” begins Windy Arey-Kent, Pine Knoll Shore’s education curator.

Faceware Live offered the team the perfect balance between price, ease of use, and technical support, particularly given that no additional capture hardware is required to track facial movement: Live is completely markerless technology, capable of converting live video from a camera feed and projecting it onto the turtle rig near instantaneously.

This means no complex set-up for the educators – Live is able to calibrate any performer in one second, simply by having the actor stand in front of a camera in nearly any lighting condition while making a neutral expression. With one click the actor is calibrated and ready to act out Caretta’s interactive performance.

Production on the installation took four months, which included building the Caretta model and her four environments, as well as plugging Faceware Live into the Unity game engine to translate each operator’s facial movements onto the digital puppet.

With some ingenuity and clever development, the team was able to create a setup that any operator could use to power the exhibit while teaching the immersed audience.

Controlling Caretta

To perform the role of Caretta, an operator enters a converted closet approximately 75 feet away from the monitor displaying the CG turtle. Inside this closet sit three setups: one running Unity and the voice modulation software that helps bring Caretta to life, another that reveals the audience to the operator via webcam, and one that runs Faceware’s tools, showing the facial performance translated onto the digital character.

Beyond responding to the audience’s questions – with the operator’s facial movements and speech patterns controlling the output on Caretta – the operator also controls Caretta’s directional movement with an Xbox controller.

Although the technology presented a learning curve to the non-technical North Carolina Aquariums staff, they say that Faceware’s technical support team goes above and beyond to help them make the most out of the facial capture setup. In fact, Faceware’s vice president of business development, Peter Busch, visited the aquarium for a full day, guiding each individual educator through best practice for Faceware Live.

Having Peter and the team come here was absolutely invaluable,” affirms Arey-Kent. “It shows Faceware’s commitment to going above and beyond to ensure that each project is a success. The support has continued since.

Lifelike lessons

Caretta’s Cove opened on Memorial Day in 2016 and has proven a huge success with its visitors: the aquarium runs four daily performances during the busy summer season and two daily performances during winter.

“The character is very lovable – people really enjoy Caretta,” says Arey-Kent. “They’re really excited to be able to talk to this digital character in real-time, learning more about the sea turtles and their habitat.”

Ultimately, the plug-and-play implementation of Faceware Live with Unity, and the ability to quickly and easily setup a performance, means that the staff at Pine Knoll Shores can operate a cutting-edge exhibit, entertaining children while impressing on them the need to respect sea turtle conservation.

“Faceware Live is a great tool for us, as it makes it incredibly easy to deliver our message,” adds Arey-Kent. “You can see the magic when the performances take place – the children are just enthralled by what they’re seeing.”

How Faceware Live powers live broadcast

Live broadcast of digital content can be a challenging prospect – but Faceware Live has been specifically designed to ease the process. Using cutting-edge technology, the real-time mocap solution can deliver instantaneous digital performance without the need for markers or a complex setup. VP of Business Development Peter Busch explains how…

Can you tell us a little about Faceware Live and why it was created?

Faceware Live is facial motion capture software that instantly creates facial animation for any animated character for live events. Our technology enables any new user via a near-instant calibration process, which quickly tracks a user’s face and makes it “the driver” of the resulting facial animation.

Created primarily for pre-visualization and live performances, Live can be used for a variety of public installations and attractions. Digital characters can be “puppeted” in real-time, allowing interaction with live audiences and people in a variety of venues, such as screens and kiosks for theme parks, live studio audiences, museums, concerts and other large entertainment events.

However, Live can be used in many different productions and projects. The nature of the software means it has endless applications.

How does Faceware Live work?

Using patented technology, Faceware Live automatically tracks a performer’s face to stream and create real-time animation on an animated character.

Faceware Live is robust – it works in nearly any lighting condition with any performer. Also, there’s no inherent limit to the amount of characters you can drive; multiple animated characters can interact with each other simultaneously.

This is thanks to the powerful technology that underpins Faceware Live, developed over 15 years of research and development. This focused development effort means the technology is rock solid and, most importantly, stable.

How easy is it to setup and calibrate a Faceware Live performance?

Setting up Faceware Live is an easy process. One unique feature of Faceware Live is the ability to calibrate any performer in one second; there is no need for a lengthy range of motion or complex character setup process.

Performers simply make a single neutral expression and with one click they are calibrated. Faceware Live is robust enough to handle almost any number of factors that can cause other real-time software to fail such as a range of different lighting conditions, ages and skin tones.

With Faceware Live you are running in one second or less – ensuring stability and flexibility in the demands of real-time content creation. Acting talent can arrive minutes before their performance and start streaming right away.

How easy is it to integrate Faceware Live into a development pipeline, such as one powered by Unity?

Faceware Live easily integrates into any development pipeline, including those powered by Unity. Unlike other products that will create content only in a proprietary toolset, Faceware Live will plug in to most popular animation suites. Our software performs the heavy lifting, making content creation an easy process.