Facebook's $2 billion acquisition of Oculus Rift has reopened conversation about the potential of virtual reality, but a key question remains: How will we actually interact with these worlds? Minecraft creator Markus Persson noted that while these tools can enable amazing experiences, moving around and creating in them is far from a solved problem.
A group of Royal College of Art students--Guillaume Couche, Daniela Paredes Fuentes, Pierre Paslier, and Oluwaseyi Sosanya--has developed a tool called GravitySketch that starts tracing an outline of how these systems could work as creative tools.
Their high-tech sketchpad looks a bit like a prop from Tron, with its etched grid of transparent plastic and sternly rectilinear user interface panel, but the goal is to make augmented reality feel as natural and organic as sketching in a notebook. Artists hold the acrylic drawing tablet like its analog counterpart and sketch with a custom stylus. Radio frequency sensors record the movements and coordinates on the pad and send them to an Arduino hidden in the black box on its edge. No lines are actually drawn on the pad, but the artist sees them float in air through a pair of augmented reality glasses wirelessly connected to the Arduino.
>The goal was to create tools that would privilege spatial awareness.
The real trick of GravitySketch is that, rather than requiring artists to master tricks like perspective or foreshortening to suggest depth and physicality, they can simply rotate the tablet and draw lines that connect to their previous marks, creating a sense of form in the process. The drawings hang in space, can be approached from any angle, and can be rotated like physical objects.
A solo sketch can quickly become a jam session if several people surround the drawing, allowing people to collaborate in real time with none of the loss that comes from sharing ideas asynchronously and in traditional formats.
Customized software was developed by the team using the game engine Unity 3D. "We wanted, from the beginning, to have a very minimal user interface," says Couche. "The four of us are users of the conventional CAD tools and having something simple that does not requires you to become an expert in order to create was on strong driver for us." That's not to say the tools are unsophisticated and sketches drawn in the virtual world can be transformed into 3-D printed objects with ease.
A Textbook Solution
GravitySketch is an impressive tech demo, but the inspiration for it didn't come from a session at Siggraph. Rather, it came from Harvard's education school, where Howard Gardner developed his theory of multiple intelligences. Gardner posits that IQ is too narrow a measure of intelligence, which can be embodied in a variety of ways, from the ability to ace standardized tests to a gift for spatial awareness.
Unfortunately, many potentially gifted designers are scared away from the field because they find the tools intimidating. Reducing three-dimensional forms to drawings is a difficult skill to master, and 3-D CAD programs are obfuscated by complex jargon and user interfaces. CAD enables perfection, but at the cost of imagination and serendipity.
The goal here was to create tools that would privilege spatial awareness instead of forcing people through a linguistic and quantitative path that too often favor people with a very specific educational background. "We are passionate designers; we love imagining and making it real," says Couche. "We felt that something was missing in the tools that are available for creation, and we wanted Gravity to be a something that would stay simple to use."
Starting with this mission, the team began experimenting with various techniques. The students stumbled upon their solution after drawing shapes on several pieces of plexiglass and stacking them together to create the sensation of an object suspended in the plastic. Instead of making people experts at 3-D, the team realized that its goal should be bringing a familiar 2-D experience to multiple planes. With that insight established, the team started the long process of transforming its concept into a cool tool.
Instead of testing prototypes exclusively with designers and other traditional audiences, the team talked to dancers and musicians to get a sense of their workflow. They were surprised by how such people approached the tool. "The most interesting is how people use it and what kind of ideas it triggers when they do," says Couche. "For example, we started with the idea to draw volumes in sections and almost accidentally added the possibility to rotate, which now seems very obvious."
Going Virtual
After experimenting with augmented reality glasses, the team realized virtual reality would alter the experience in interesting ways. So it started reworking the experience, optimizing it for the first generation Oculus Rift developer's kit. "The level of immersion is so high that you can free your mind and just create," says Couche. "You also lose some of the interaction with the real environment and the conviviality of the multi-user experience." The experience needs polish, and the team is eager to use the next version, which features head-tracking capabilities. But hardware limitations haven't stopped the team from dreaming up new applications.
Both augmented and virtual reality have their applications with the system. A team of doctors preparing for a complex surgery would be better suited with an AR system that let them draw surgical incisions on a model of a patient, or the patient directly. An architect who sketched out a new doorway would be better suited by the immersive virtual experience provided by VR.
"Now we have a lot of interesting conversations with people who want to do animation with it, on-site design of construction or vehicles, and educational tools," says Couche. "We even discussed with a space exploration expert about the possibilities of visualizing mission architectures and orbital dynamics."
