Lamosca
By now, nearly everyone from the president of the United States on down has admitted that the National Security Agency went too far. Documents leaked by Edward Snowden, the rogue NSA contractor who has since gained asylum in Russia, paint a picture of an organization with access to seemingly every word typed or spoken on any electronic device, anywhere in the world. And when news of the NSA's reach became public—as it was surely bound to do at some point—the entire US intelligence apparatus was thrust into what The New York Times recently called a "crisis of purpose and legitimacy."
It was a crisis many years in the making. Over the course of three decades, the NSA slowly transformed itself from the nation's junior spy agency to the centerpiece of the entire intelligence system. As the amount of data in the world doubled, and doubled again, and again, the NSA kept up with it—even as America's human intelligence capability, as typified by old-fashioned CIA spies in the field, struggled to do anything useful with the unprecedented quantities of signals intelligence they had access to. Trained agency linguists capable of parsing massive quantities of Arabic- and Farsi-language intercepts don't scale up nearly as easily as data centers do.
That, however, wasn't the computer geeks' problem. Once it was clear that the NSA could do something, it seemed inarguable that the agency should do it—even after the bounds of information overload (billions of records added to bulging databases every day) or basic decency (spying on allied heads of state, for example) had long since been surpassed. The value of every marginal gigabyte of high tech signals intelligence was, at least in theory, quantifiable. The downside—the inability to prioritize essential intelligence and act on it; the damage to America's democratic legitimacy—was not. As a result, during the past couple of decades spycraft went from being a pursuit driven by human judgment calls to one driven by technical capability.
commentators who "trusted their gut" about mitt romney had their gut kicked by nate silver, the stats whiz who called the election for obama.
This shift in US intelligence mirrors a definite pattern of the past 30 years, one that we can see across fields and institutions. It's the rise of the quants—that is, the ascent to power of people whose native tongue is numbers and algorithms and systems rather than personal relationships or human intuition. Michael Lewis' Moneyball vividly recounts how the quants took over baseball, as statistical analysis trumped traditional scouting and propelled the underfunded Oakland A's to a division-winning 2002 season. More recently we've seen the rise of the quants in politics. Commentators who "trusted their gut" about Mitt Romney's chances had their gut kicked by Nate Silver, the stats whiz who called the election days beforehand as a lock for Obama, down to the very last electoral vote in the very last state.
The reason the quants win is that they're almost always right—at least at first. They find numerical patterns or invent ingenious algorithms that increase profits or solve problems in ways that no amount of subjective experience can match. But what happens after the quants win is not always the data-driven paradise that they and their boosters expected. The more a field is run by a system, the more that system creates incentives for everyone (employees, customers, competitors) to change their behavior in perverse ways—providing more of whatever the system is designed to measure and produce, whether that actually creates any value or not. It's a problem that can't be solved until the quants learn a little bit from the old-fashioned ways of thinking they've displaced.
No matter the discipline or industry, the rise of the quants tends to happen in four stages. Stage one is what you might call pre-disruption, and it's generally best visible in hindsight. Think about quaint dating agencies in the days before the arrival of Match .com and all the other algorithm-powered online replacements. Or think about retail in the era before floor-space management analytics helped quantify exactly which goods ought to go where. For a live example, consider Hollywood, which, for all the money it spends on market research, is still run by a small group of lavishly compensated studio executives, all of whom are well aware that the first rule of Hollywood, as memorably summed up by screenwriter William Goldman, is "Nobody knows anything." On its face, Hollywood is ripe for quantification—there's a huge amount of data to be mined, considering that every movie and TV show can be classified along hundreds of different axes, from stars to genre to running time, and they can all be correlated to box office receipts and other measures of profitability.
Next comes stage two, disruption. In most industries, the rise of the quants is a recent phenomenon, but in the world of finance it began back in the 1980s. The unmistakable sign of this change was hard to miss: the point at which you started getting targeted and personalized offers for credit cards and other financial services based not on the relationship you had with your local bank manager but on what the bank's algorithms deduced about your finances and creditworthiness. Pretty soon, when you went into a branch to inquire about a loan, all they could do was punch numbers into a computer and then give you the computer's answer.
For a present-day example of disruption, think about politics. In the 2012 election, Obama's old-fashioned campaign operatives didn't disappear. But they gave money and freedom to a core group of technologists in Chicago—including Harper Reed, former CTO of the Chicago-based online retailer Threadless—and allowed them to make huge decisions about fund-raising and voter targeting. Whereas earlier campaigns had tried to target segments of the population defined by geography or demographic profile, Obama's team made the campaign granular right down to the individual level. So if a mom in Cedar Rapids was on the fence about who to vote for, or whether to vote at all, then instead of buying yet another TV ad, the Obama campaign would message one of her Facebook friends and try the much more effective personal approach.
Oakland, 2002—when data geeks moneyballed the national pastime. 
Dave Kaup/Getty Images
Most strikingly, the campaign perfected the art of A/B testing—the practice of testing alternate versions—when it came to fund-raising emails. Writing effective language for such appeals, and designing them in the best possible manner, has traditionally been considered an art—but all gut intuition was discarded in favor of raw data about what worked and what didn't. One email in June, headlined "I will be outspent," raised $2.6 million on its own, while projections based on their testing indicated that other emails would have raised less than a fifth of that. Everything was A/B tested, from the background color (yellow worked better than white for some reason) to the greeting, the subject line, and the size of the request.
After disruption, though, there comes at least some version of stage three: overshoot. The most common problem is that all these new systems—metrics, algorithms, automated decisionmaking processes—result in humans gaming the system in rational but often unpredictable ways. Sociologist Donald T. Campbell noted this dynamic back in the '70s, when he articulated what's come to be known as Campbell's law: "The more any quantitative social indicator is used for social decision-making," he wrote, "the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor."
On a managerial level, once the quants come into an industry and disrupt it, they often don't know when to stop. They tend not to have decades of institutional knowledge about the field in which they have found themselves. And once they're empowered, quants tend to create systems that favor something pretty close to cheating. As soon as managers pick a numerical metric as a way to measure whether they're achieving their desired outcome, everybody starts maximizing that metric rather than doing the rest of their job—just as Campbell's law predicts.
once quants disrupt an industry, they often don't know when to stop—and they create systems that encourage cheating.
Policing is a good example, as explained by Harvard sociologist Peter Moskos in his book Cop in the Hood: My Year Policing Baltimore's Eastern District. Most cops have a pretty good idea of what they should be doing, if their goal is public safety: reducing crime, locking up kingpins, confiscating drugs. It involves foot patrols, deep investigations, and building good relations with the community. But under statistically driven regimes, individual officers have almost no incentive to actually do that stuff. Instead, they're all too often judged on results—specifically, arrests. (Not even convictions, just arrests: If a suspect throws away his drugs while fleeing police, the police will chase and arrest him just to get the arrest, even when they know there's no chance of a conviction.)
The same goes for the rise of "teaching to the test" in public schools, or the perverse incentives placed on snowplow operators, who, paid by the quantity of snow cleared, might simply ignore patches of lethal black ice. Even with the 2012 Obama campaign, it became hard to learn about the candidate's positions by visiting his website, because it was so optimized for maximizing donations—an easy and obvious numerical target—that all other functions fell by the wayside.
The most profound example of overshoot, of course, happened in finance, where the rise of quantification could concentrate decisionmaking—and moneymaking—within a relatively small group of people at a bank's headquarters. Soon they were trying to optimize their algorithms to maximize profit, minimize risk, and make millions of dollars for themselves. Global regulators didn't help: In 2004, in sympathy with the over-leveraged, hyper-quantified banking system, the Basel Committee—the Switzerland-based body that oversees world finance—put out the Basel II accord, more than 250 pages of regulations that effectively placed individual banks in the driver's seat. The accord essentially embraced all of the quantitative techniques used by the wizards who would end up blowing up Wall Street, and it allowed banks to operate with astonishingly high levels of debt. As everybody knows, all of that ended in catastrophe in 2008. (You can read more about the particular math of that cataclysm in my March 2009 cover story for WIRED, "A Formula for Disaster.")
It's increasingly clear that for smart organizations, living by numbers alone simply won't work. That's why they arrive at stage four: synthesis—the practice of marrying quantitative insights with old-fashioned subjective experience. Nate Silver himself has written thoughtfully about examples of this in his book, The Signal and the Noise. He cites baseball, which in the post-Moneyball era adopted a "fusion approach" that leans on both statistics and scouting. Silver credits it with delivering the Boston Red Sox's first World Series title in 86 years. Or consider weather forecasting: The National Weather Service employs meteorologists who, understanding the dynamics of weather systems, can improve forecasts by as much as 25 percent compared with computers alone. A similar synthesis holds in economic forecasting: Adding human judgment to statistical methods makes results roughly 15 percent more accurate. And it's even true in chess: While the best computers can now easily beat the best humans, they can in turn be beaten by humans aided by computers.
In finance too we're starting to see at least the outlines of a synthesis. In September 2010, the Basel Committee came out with Basel III, and while it doesn't fully dismantle Basel II, it does add layers of common sense on top of all the rocket science. As well as raising the required capital ratio, it sets a leverage ratio (effectively a maximum size that a bank can grow to, given the amount of capital it has) and liquidity requirements that experienced bankers know create a cushion for the whole system. Essentially, while the algorithms were given free rein under Basel II, there's a host of overrides in Basel III that put power back where it belongs, in the hands of experienced regulators. Basel III isn't perfect, but no international system of bank regulation could ever hope to be. In a few years' time, if and when it gets fully implemented, it's going to be a vast improvement on what preceded it.
That's what a good synthesis of big data and human intuition tends to look like. As long as the humans are in control, and understand what it is they're controlling, we're fine. It's when they become slaves to the numbers that trouble breaks out. So let's celebrate the value of disruption by data—but let's not forget that data isn't everything.
