The most startling thing about the music video for British band Duologue’s song Memex isn’t the ghostly way the slow-panning camera turns human skin into a landscape. It’s not how the model, actor Beryl Nesbitt, manages not to blink. And it’s not how at 3:30 in, with just a change in the lighting, the tone of the video abruptly gets dark and sinister.
Rather, the most startling thing about the Memex video is that none of those hyper-realistic shots are photos. It’s a virtual 3-D scan of a human body, in this case, British actor Beryl Nesbitt. The knowledge that it’s virtual reality makes watching the video a little like seeing a Chuck Close for the first time: Your eyes initially registered a black-and-white photograph of monolithic scale, and it takes a few beats before you can believe that it’s a painting.
London-based creative studio Marshmallow Laser Feast---great name!---and visual effects studio Analog made the video for the band (and their friends) Duologue as an deep dive experiment into filmmaking for virtual reality. Hollywood has been using various kinds of 3-D scanning technology for years---films like Avatar, and The Curious Case of Benjamin Button, relied on it. But for the Memex video, the creators used 94 cameras, whereas some Hollywood studios have been reported to use seven, to capture Nesbitt’s glowing skin in the most microscopic detail possible.
“We were trying to push the realism of it,” says Barney Steel of Marshmallow Laser Feast. The team is interested in the combination between photogrammetry—the science of taking 360-degree measurements from photos—and panoramic high dynamic range lighting, which is like photographs of light from all positions and in myriad settings (for example, it could be moon lighting, or desert lighting). Once they recorded both Nesbitt’s skin and a range of light, they worked with Analog to merge all that visual data. The result is a high resolution, immortal, portrait of the actor’s skin.
Of course, the Marshmallow Laser Feast team could have created a similar music video by just filming Nesbitt. “Our ambition was to create the model at a really high resolution so we could take it to a games engine environment,” Steel says. “At the moment we’re treating it like a sculpture. But if you spend enough time you can rig it to be exactly like a face. The software is out there to animate it like a motion capture.”
That said, the Marshmallow Laser Feast team isn’t trying to create a 3-D human scan that will show up in a scripted film or video game. They create large scale, mind-bending installations out of light and sensors, like Light Forest, which turned an entire room into an instrument of light and green lasers. So their objective is to find a way to loop virtual humans into their artwork. “The exciting thing about virtual reality is it can give you a sense of presence that doesn’t exist in other technology—and it feels like we’re on the edge here,” Steel says. “The limiting factor is how realistic those little bubbles are. We’re interested in powerful experiences where you’re not bound by any of the restrictions of reality.”
