HOLLYWOOD, California---In his classic book on film editing, In the Blink of an Eye, Walter Murch writes about the violence of the cut. In an instant, everything you see onscreen is erased and replaced with something else. Often the scene jumps to another place or time. "Nothing in our day-to-day experience seems to prepare us for such a thing," Murch writes. And yet, in movies this happens all the time, and we accept it without giving it a second thought.
It's amazing that film editing works, because it's so disruptive to the visual information coming into the brain, says Jeffrey Zacks, a neuroscientist at Washington University in St. Louis. On the other hand, Zacks says, our brains do quite a bit of editing of their own---and we're every bit as oblivious to that as we are to the film editor's cuts.
For example, our eyes are constantly darting around from place to place. We do this thousands of times a day, and every single time, during the fraction of a second that the eyes are in motion, our visual system essentially shuts off to spare us the nausea-inducing blur. The movie playing in your mind's eye may feel smooth and continuous, but that's an illusion manufactured by your brain. It cuts your visual stream of consciousness right before your eyes jump and joins it to what comes right after.
And that's not the only kind of cutting and editing are brains our doing, Zacks said at a recent event here sponsored by the Academy of Motion Picture Arts and Sciences. His research suggests our brains are constantly dividing up the torrent of information streaming in through our senses into more manageable chunks in order to help us make sense of what's happening around us and predict what's likely to happen next.
In one experiment, Zacks had subjects watch a mundane home movie of a guy washing dishes in his kitchen. He asked them to press a button whenever they judged that one meaningful unit of activity had ended and another had begun. "A reasonable fraction of them look at us like we're nuts," Zacks said. But despite the strange instructions, people tended to press the button at roughly the same times. To Zacks that suggests that we intuitively and automatically break our visual stream into discrete scenes, and that our brains do this in a remarkably consistent way.
He thinks this is a manifestation of our brains never-ending effort to predict the future. We have a mental model of what's happening that we use to predict what's likely to happen next. "You do this because it's super adaptive," Zacks said. "If you can anticipate what's coming up in a few seconds you can react adaptively." But whenever the action changes---when the stoplight turns from red to green, say, or when your boss suddenly appears at your desk---you have to update your mental model to reflect what's happening now.
How does your brain do this?
Repeating the dishwashing movie experiment with subjects inside an fMRI scanner Zacks has identified a set of brain regions that become active around event boundaries---the transitions from one event to the other. One is an area at the intersection of the temporal, parietal, and occipital lobes; another is part of the right dorsolateral prefrontal cortex. Exactly what all these brain regions are doing isn't clear, but it's something Zacks wants to figure out.
In another experiment, Zacks and his graduate student Khena Swallow found that people tend to struggle to remember objects they've seen just before an event boundary. Zacks demonstrated this at the Academy event with a scene from the French film Mon Oncle. Not much happens, really. A man walks through a door into a house. But that action represents an event boundary, and when Zacks asked the audience to call out which of two objects---a cat or a chair---was present in the scene before he walked through the door, only about a third of the people correctly picked the chair (another third picked the cat, while another third stayed quiet, presumably because they had no clue). Even though the chair had been plainly visible just a few seconds earlier, most people failed to recognize it.
Zacks has found that when people do successfully reach back across an event boundary to remember something they saw, they activate their hippocampus. That's interesting, he says, because this part of the brain is well-known for its role in encoding long-term memories. "We're used to thinking of this structure as representing the distant past," Zacks says. Once your brain detects an event boundary, even things that happened as recently as 5 seconds ago can get relegated to the distant past.
Zacks says he initially suspected that when people watch movies, the event boundaries detected by their brains might match up with the film makers' cuts. Not so. Most cuts in movies are continuity cuts, as when the camera switches to a different angle on a dinner party or a bank robbery. The view changes, but it's still the same scene, and people tend not to register this as an event boundary, he has found. When the scene changes, however, there's much better alignment.
Continuity cuts and scene breaks may work so well because they mirror our brain's own mechanisms for maintaining visual continuity and breaking up reality into more manageable bites, Zacks says.
"Our brains evolved to take this firehose of sensory information and boil it down to something we can use effectively to survive, and movies leverage that feature of our biology to shape our experience," he said.
This story is part of a series about how scientists are studying cinema for clues about the nature of perception, and how the science might aid film makers as they pursue their art.
