How They Made ‘Harold Halibut’: A Handcrafted Stop-Motion-Inspired Video Game

Harold Halibut is an unusual fellow.

The lanky figure, with an aquiline nose, suspenders, and janitorial toolbelt is an unassuming hero on board the Fedora – a giant spaceship, controlled by an officious All Water Corp., submerged since crash-landing in an alien ocean 250 years ago. Perhaps the most remarkable thing about Slow Bros.’ computer game Harold Halibut is, for its 12-18-hour running time, the world is populated by what appears to be a cast of animated puppets in miniature settings.

In early previews, pundits struggled to characterize the handcrafted vibe of the game, which appeared on Windows, Xbox, and PlayStation 5 platforms in April. Cartoon Brew connected with Ole Tillmann – one of Slow Bros. key artists who worked with director Onat Hekimoglu in Cologne, Germany. As the art director, co-writer, puppet designer, and illustrator of the game, Tillman discussed the process of crafting Harold his world from handmade objects, which were scanned and digitally manipulated into an atmospheric adventure.

Cartoon Brew: How did you become involved with Slow Bros.?

Ole Tillmann: We met through our sisters. We all are from the same hometown, Aachen, or smaller villages around there. Our sisters knew each other and connected us. I was doing illustration before I met them, and was primarily invited because I knew how to draw and visualize ideas.

Was that while they were building their prototype miniature set, in 2012?

Tillmann: Yes, Fabian Preuschoff, Daniel Beckmann, and Onat [Hekimoglu] had been working together on this idea for a couple of months. They had physically built one ‘room’ because they didn’t know how to draw, or do 3d computer work. That is how they got started on this whole making the game path. And then they invited me to draw the first character, which was Harold.

Your press kit mentions a dinner conversation where Onat discussed the influence of Ray Harryhausen films. How did those ideas inform Harold?

Tillmann: The four of us had a lot of conversations over dinner where we would go crazy, discussing where Harold could live, the greater world, and so forth, coming up with characters and locations. It was very much a world-building enthusiasm reflected through those dinner conversations.

Was the story leading the technique, or vice versa?

Tillmann: I guess both, weaving back and forth over time and then including game design. The first thing we came up with, after a couple of experimental rooms, was a general plotline: we had Harold and a couple of interest groups, and we knew generally what was going to happen between them. We started becoming more detailed along that thread. And it kept going back and forth.

For instance, we knew we needed some type of area where a lot of people regularly gathered. We’d roughly start to build this marketplace. And, while building, we’d decide, oh, maybe we need a bar next to this store? The next time we got together in a story meeting, we’d discuss, okay, it seems we’ve got a bar now – how can we use this to get characters to hang out together? There were several situations like that where the building would inform the writing, and then we would go back into the writing room, and that informed the next steps of building.

How did you decide to set this community on a spaceship in an alien ocean?

Tillmann: We wanted to put a spotlight on an enclosed society. Originally, it was supposed to be more short-story-sized, and it expanded. But we stayed with the idea of a village-sized society of people and looked at the relationships and friendships within that. The spaceship idea came about in combination with the water theme. We wanted it to be underwater.

Let’s focus on Harold. There’s such humanity to the puppets and attention to detail, but they’re also distinctly puppets – how did you arrive at that look?

Tillmann: The characters are the one thing that I can’t point towards a specific inspiration. They were the most intuitive and organically created portion of the game. Harold was where it started, and that was my job, when the three other people who came up with that initial idea of doing the game called me up [and said], ‘We’re making a game over here, we want to build it from real materials, and we’d like to have a main character.’ They described the character of Harold, as far as he was developed at that point. They said, ‘We want him to look a little bit like Adrian Brody, to be approachable, and yet a little bit milquetoasty neutral.’ That idea – to have Harold as a harmless vessel, who was there to look at the other characters – was there early on. He had big, beady glass eyes. I hadn’t done any sculptural work before that, but the more characters I did, the more intuitive the process became. The other factor was our costume designer, Holle Schlickmann, who designed tiny textile costumes that added another layer of storytelling.

Harold appears to have articulated eyes, brow, and jaw motion. Is there an armature, or sculpture wire inside him?

Tillmann: Neither. With Harold, we did stop-motion in the beginning, when we first experimented, so Harold theoretically still has wires and some armatures inside his body. But after Harold, we stopped doing that. The gameplay didn’t work in combination with stop-motion. We calculated the work and it was clear that it would be impossible to make a game in this manner. All the facial animation is completely digital. We sculpted the face physically, and then scanned it, and then all the variations, keyframing, and everything, we did in 3d.

Harold’s body motion at times feels like Japanese bunraku puppetry – how did you animate that digitally?

Tillmann: We went back and forth trying to figure out how [to] embrace the ‘puppetness.’ In facial animation, for example, we reduced the frame rate a lot, chopping in between keyframes rather than interpolating from position to position, which is usually what happens in digital animation.

As for the bunraku element, that’s because we used motion capture suits – capturing human motion in [tracking marker] suits – to move the puppets. We also used our hands to make objects fly. For spaceship motion, for example, an animator would put on the motion-tracking glove and move that to the sky. That created that bunraku-like feeling where you can feel that human touch.

There is video of Onat wearing Xsens motion capture gear – did he drive the body performances for all your characters?

Tillmann: Yes, it made the most sense, for budget reasons, but also for Onat, who had to explain each scene. He would sometimes do rotations over and over in scenes with 10 characters. Onat would rotate through the characters, over and over using reference points [in our studio]. We realized we couldn’t apply natural movement to the characters. So, Onat modified his acting to become a little stop-motion-esque, and then we’d go into keyframes of that.

How did you animate eye-blinks and lip-sync?

Tillmann: Eye blinks were largely automated. And then, we would go back in for certain moments and interrupt them, or add blinks. Eye movement was entirely hand-done. At its lowest level, that worked by using targets that characters could look at and follow. And then, for facial animation and lip-sync, we had a base layer of automation via the audio track, where we fed the actors’ [vocal] recordings. [FaceFX] software would then analyze the audio in combination with the text. Then we would clean that up by hand in some areas. And the nicest thing that worked in combination with the puppet keyframe process was the software [allowed us] to add emojis in between the text lines to trigger specific facial setups. We could add a sad emoji, a calm but sad one, or a widely laughing one, that controlled either the entire face or sometimes just the eyebrows.

As well as your human cast, there are many colorful, bipedal fish characters, the Flumylym – how many of those did you create?

Tillmann: We made seven Flumylum, and an extra five heads to mix them up a little.

When the Flumylym speak, their lips form burbling words, that Harold can understand. Did you work out Flumylym linguistics?

Tillmann: No. We did have some Tolkien-esque ideas in the beginning where we considered hiring people because there’s a community of people who love to invent languages for fantasy projects. But we steered away from that because it would have added another layer of craziness to the project. Plus, we decided that we wanted the audio to function more like a documentary – we’d hear a muffled version of [Flumylym] language in the background, and then the English translation [appears] on top. Since we knew that was going to be the case, it would have been overkill to invent a language. That was mostly people from our team doing [Flumylym] impressions.

Once you worked out your process, how long did it take to complete animation, and how many were on your team?

Tillmann: Recording was only a couple of months. But then we had to hire people to go through all the animation footage, adding hand animation. The hardest part was cleaning up the recorded motion. Ilja Burzev was crucial to the technical approach – he made it possible, essentially, he’s very good at all the 3d stuff. Then Onat, Fabi, me, and Daniel Beckmann came up with the characters and story elements. Danny Wadeson wrote dialog and sharpened everything writing-related. Over time we were a team [that included] four to sixteen people, on and off.

Was every one of the Fedora interiors a physical set?

Tillmann: Yes, and that was the most fun part, getting lost with Fabi and everyone, making all the sets. We were using a specific [photogrammetry] technology to scan the sets that required us to have each wall be collapsible. So we built them to be foldable, in individual parts. We’d scan them, and then put them in a box. We do have a basement chock full. We were fortunate enough to have our first exhibition last year [at the Museum of Design and Visual Communication, in Zürich, Switzerland, March 2023]. That filled a good living-room-sized area with mostly characters and small objects.

How did you create Fedora exteriors?

Tillmann: We adhered to our stop-motion credo and tried to imagine having a stop-motion set, and we considered what effects to use for a stop-motion scenery working at that scale. If we were adding fog, we wanted that to look like it was a fog machine blowing in from the side. And all the lights, especially, are physically correct. Onat and Fabi both have a film background, so they knew exactly what type of lights would be used in each scenario.

It’s an unusual game – meditative, like Harold, but it also contains elements of quirky humor. Did it surprise you how this grew?

Tillmann: It has been surprising, and the biggest reason for that was our group dynamic. I’ve never worked with another group that’s been so naturally and intuitively, full of growth. We would also get into arguments. Meditative versus quirky was an internal back-and-forth that sometimes got stressful. But we always knew one side would appreciate slapstick humor more, whereas the other was a little more into emotions. Our Turkish telenovela subplot was at the extreme end of that. I don’t know if you saw that, but there’s a Turkish telenovela on board [the Fedora] that’s another mini-story in the background, seen on their local television station. It’s one of the only remnants they brought from Earth and, like all telenovelas, it’s overly dramatic. We chose to not translate that. That was one of the directions in our overarching story. The team members would know, okay, I’ll give you this moment of deep emotion, so we can have more slapstick over here. We all enjoyed that seesawing between us.