People experience art in specific ways: looking at paintings, hearing music, tasting food. Unless you’re a synesthete like Kanye who sees rap songs as colors, this is how it works. But what if you could see music—not as a video or a performance, but a physical object? And what if that object could play the music back for you?
That’s the goal of Reify, a startup from New Inc’s incubator that is Kickstarting its first project. Lead by Allison Wood and Kei Gowda, Reify turns sound waves into 3-D printed sculptures that play the sound back with an augmented reality app. Wood started the company to explore digital synesthesia, a technologically augmented version of the skills Kanye and other syntesthetes were born with.
Music was a natural place to start. Wood began thinking about how she might translate music's ephemeral nature into something tangible and permanent. She came up with the totem, an object 3-D printed in plastic or plated in bronze as a visual, physical representation of a song.
Designing a totem is something of a bespoke process that begins by playing the song through the Echo Nest API to analyze its structure, rhythm, amplitude, and more. This data correlates to physical attributes like height, weight and mass. A fast, loud song could produce a tight, jagged printing pattern, while a mellow song might take a softer, more sinewy form. “It starts to create a new visual language,” says Wood.
A totem’s final form is drawn in equal measure by computerized input and subjective tweaking by the artists (Reify is making them with noise band Health, classical violinist Tim Fain, Yacht and Mae). Early in the process, Wood asks musicians to describe their sound. In the case of Health and its single "Dark Enough," words like “jagged,” “angular,” and “harsh” came up. Its totem has an disorienting form, like a relic in a post-apocalyptic world.
The totems are in some ways a sophisticated music player. The app reads its form much like a stylus on a record or a laser on a CD and plays it through your phone. When the totem enters your phone's camera frame, it begins to animate on your screen, creating a customized augmented reality experience. "It's using your phone like a looking glass into the digital world," says Wood.
Granted, this isn't the future, or even a future, of listening to music—it's all too complex compared to the seamlessness of tapping a playlist or dropping a tonearm. But it is compelling from a design point of view. At its core, Reify is doing what many information designers have done before: visualizing data. "When you really think about it what were doing is translating data from one from into another,” says Wood. Reify just happens to be taking advantage of the latest technology to make it happen.
