This Puppet-Like Contraption Lets You Animate CG Characters In Real Time

Tired of the slow pace of animation production? A new tool is on hand — literally — to help speed things up.

Mr. Puppet Animation System is the brainchild of Thinko, a young New York studio with a roster of trendy clients and a penchant for designing its own hardware. The tool is a hand-operated contraption which creators can use to manipulate an animated cg character in real time, like a puppet. Thinko boasts that this can cut a weeks-long production down to hours.

The company breaks the pipeline down as follows. An animator is shot performing with the contraption (20–30 minutes). The team can then clean up the animation with traditional software (30–40 minutes) before the final edit (30–40 minutes). The result is a finished piece of animation — but it isn’t hard to think of more specific applications for Mr. Puppet, like in previsualization or setting production keyframes. While the tool isn’t on general sale, it could potentially benefit client studios in various ways.

Thinko has demonstrated the system in BNN: Bird News Now, a satirical news show starring avian anchor Burt Starling. The series, more a proof of concept than anything else, is visually basic: Burt doesn’t move much and his facial expressions are limited. But Thinko is developing a second season and two other shows, and stresses that its characters will become more dynamic as it upgrades its hardware.

Mr. Puppet brings to mind optical-based augmented reality, whereby computer or phone cameras track movement and use the visual data as the basis for animation. This was, in fact, Thinko’s initial approach — but as Pasquale D’Silva, animator and Thinko co-founder, tells Cartoon Brew, it didn’t suit the studio’s purposes. Below, D’Silva talks us through how Mr. Puppet began, what it can and can’t do, and how it could change storytelling in animation.

The origins

D’Silva: The Mr. Puppet Animation System is the sum of a few influences, the first being a desire to move faster. With animation being such a beautiful but laborious art form, it can be daunting to build up the momentum to pull the idea out of your brain. I wanted to reduce the time from thought to screen loop, so I could play with ideas faster.

The second desire was iterating on a more tangible thread: performing animation. A while back, we were building a product called Lightbox, a 2d animation package. Like other packages, you could create pictures frame by frame (straight ahead) and tween (more like pose to pose). The piece we never got to was mixing in “performed” pictures, giving the artist the ability to directly manipulate the timing and picture.

Though we put the project on ice, the idea of integrating performance capture input into animation workflows kept brewing in my mind. A standard motion-capture approach always felt like too much overhead and overkill. It’s too much data to wrangle. The characters I was interested in performing had always been less literal than humanoids; more like Jim Henson’s Muppets, with simpler, abstracted forms. Our artists work like a hybrid of animator and puppeteer, and we draw inspiration from both disciplines.

The hardware

D’Silva: Before we did the deep dive into building hardware, we tried optical approaches, and boy were they hairy. Optical technologies like the Yo Puppet app work well as fun consumer toys, but are not reliable and quite restrictive. Even with pose estimation, optical data suffers from occlusion when hands get into untrackable poses. Cameras have limited fields of view, which means the performance is constrained to a pretty small physical area. The data is patchy.

Our hardware approach provides continuous data, because the sensors are mechanical. We also inherit tactile feedback and useful constraints from the physicality of our “glove.” Additionally, we’re building more ergonomic companion controllers to complement the core performance, allowing the performer to extend the capability of the characters.

We rapidly prototype and print our hardware in-house using the Ultimaker & Makerbot 3d printers. Game engines like Unity and Unreal have triggered a boom in filmmakers working in real time, and we’re constantly looking at the bleeding edge there. Our friends at Wonder Unit have been inspirational, taking modern consumer technology practices and cross-pollinating them into the film world.

The content

D’Silva: Burt was one of our first prototype characters. In season one of BNN, his range of motion was constrained by [our] own imposed limits. He’s currently getting a big makeover for season two, and will soon be a very articulate, expressive character.

The biggest challenge we face in building these characters is striking a balance between fidelity, the ergonomics of performance, and mixing in the right amount of simulation. Each puppet is different, so we have to tune each one to taste. Thankfully, as we crack problems with various puppets, we can share the technology and techniques across all of them, and they become more robust.

My favorite augmentation Mr. Puppet provides is the introduction of improvisation in the story process. So much of animation is pre-visualizing, pre-planning, pre-everything. When you get to finally seeing a character move on screen, it feels like their destiny is greatly predetermined. In order to make the characters believable, artists then have to reverse-engineer hints of imperfection and spontaneity.

If you’re willing to treat animation more like improvised live action, you can fall down a rabbit hole of happy accidents. I always loved the way the Jim Henson Company would shoot Muppet scenes, with loose outlines and room for interpretation. And of course, it doesn’t all have to be improvised: you can still perform pre-planned scenes or adopt a hybrid approach of writing and acting.

Because the technology can work in real time, there’s opportunity to tell real-time stories. We’re interested in producing shows that are interactive and can be steered by audience participation. We have a musical kids’ show we’re working on with Eric Pringle, animation director on Teen Titans Go!, and a bizarre show called Rad Milk, which features two friends who live in a man cave (a cave inside a giant man floating in space).

(D’Silva’s comments were taken from an interview conducted by email. They have been lightly edited for brevity.)