iClone 7 Introduces Markerless Facial Capture Tech

Being able to drive facial animation – without tracking markers – is now possible inside the 3d animation software iClone 7 from Reallusion care of a new collaboration with Faceware Technologies, Inc.

That collaboration is designed to enable real-time facial tracking using as little as just a webcam to drive natural human expressions onto animated character faces.

Cartoon Brew looks at how Reallusion and Faceware combined their technologies for the new tool.

If you’re already a Reallusion iClone user, then you’ll know the software leans heavily towards the creation of animated characters and avatars. It already includes several methods for facial mapping and lip-sync, with real-time playback.

Meanwhile, Faceware has been offering facial capture hardware and software for several years – the tool has been used for video games such as Middle-Earth: Shadow of War and The Witcher 3, and in feature film visual effects and animation, and vr projects.

Faceware’s partnership with Reallusion involves plugins for iClone that enable markerless capture using a single PC camera or Faceware’s proprietary head cameras. Users sit through a short calibration process designed to teach Faceware what your face looks like in a neutral pose. It then tracks and captures key facial features as you perform voice tracks and head actions.

This capture can then drive an animated character or avatar, in real-time if necessary. Reallusion revised iClone’s 3d character faces to 60 morph shapes – there had previously been 30 – to make the characters ‘Faceware-ready.’ The resulting animation can be motion edited and intensity adjusted through slider bars. Characters and animation made in iClone are already exportable as FBX files to game engines or 3d applications like Unreal Engine, Unity, Maya, and 3ds Max.

So, what is the benefit of using these new plugins compared to normal facial animation? Well, previously, any facial animation required a traditional blend of lip-sync, puppeteering, and face key editing. Reallusion’s vp of marketing John C Martin II says that now, with Faceware, there is no longer the need to do lengthy character setup and rigging, which can instead be accomplished by “using facial animation-ready iClone Character Creator characters that serve as a base character to morph, dress, or sculpt into a myriad of looks and styles.”

From Faceware’s point of view, the company has been dabbling in many markets, but believes more and more content is now being made directly for immersive experiences – which don’t necessarily have the budgets of games or feature films. The iClone collaboration is intended to be an affordable, but pro, solution for facial mocap.

“With the emergence of these new markets and new makers,” said Peter Busch, vp of business development at Faceware Technologies, “comes a massive need for user-friendly tools that speed up the ease of production and that allow capabilities to be enabled which were previously cost prohibitive.”

If you already have iClone 7, the Faceware plugin suite is available for US$999. There’s also a free trial option and other options to purchase a bundle that includes iClone 7 and other Reallusion character tools.