Team Rolfes on VR puppetry, digital character design and the importance of live performance

VR puppetry is pretty much what it says on the tin. By adorning a VR headset, and recording the movement within an intricately designed digital space, studios like Team Rolfes can push the boundaries of digital expression through body-driven movement. For the New York-based digital performance and image studio, this is its specialism. Founded by Sam Rolfes along with his brother Andy Rolfes back in 2017, the studio arose organically following a collaborative 3D performance with Super Deluxe – as well as a background in music video direction and art direction respectively.

“I began exploring live VR performance simply by palming a VR headset to use as a camera in my first music video for Amnesia Scanner,” explains Sam. “I didn’t really have any other method of animating the POV and motion that I was looking for.” Using VR controllers, motion capture, or a combination of the two, he devised a method that not only accurately represents whole-body movements in the digital realm, but also explores the point where puppetry can further expand on human movement through the digital realm.

In this sense, a human’s subjective gait combines with the digital technology’s preconceived notions of coded movement to create a more personal character. “The specific control scheme that the puppet is built with, and its relationship to the how the hands, head, body et cetera might drive it, in turn, gives us context to the personal choreography of a puppet’s character." Most recently exemplified in the music video for Matthew Dear’s Bunny Dream, the jerky movements of the digital puppets relay the authentic physical control systems that attach each mode of digital motion with human contact.

“Technically, it’s not that different to the traditional methods of rigging up 3D characters for motion” explains Sam. “But the really interesting things happen when you create these novel, counterintuitive ways of controlling the puppets.” Depending on the individual personality traits of each character, Sam and Andy play with the levels of interactivity that connect one part of the puppet’s movement with another.

“Should the head rotation control the hand which is then attached to cables pulling the bodies of another character?” In response to this, Sam adds: “Maybe the arms form the face when they come together and the distance between the hands drives the expression of the eyes.” All in all, the options are endless when it comes to fine tuning a VR puppet’s personality.

For Sam, “the best way to let an animation or story unfold is best done in the moment, responding to the environment and the various stimuli coming in.” Instead of spending copious amounts of time on the labour-intensive traditional methods of animation which Sam finds boring, and ultimately, a delay to creativity, he resultantly employs the impulsiveness of performance as a solution. Recently completing a series of live performances at the likes of MoMA and Tasmania’s Dark Mofo this summer, Team Rolfes continues to probe the boundaries of VR puppetry and the kind of expression it can deliver. To conclude, he tells It’s Nice That: “In a way, live performance is simply building images and then moving between them; a series of paintings that we’re acting out both visually and physically.”