The Difficulty of Discovery (Shrinking Asteroids Version)

Last weekend, my Head Case column in the WSJ explored the increasing importance of scientific teamwork: Benjamin Jones [an economist at Northwestern] has found that scientific teams have become a far more important part of intellectual production. By analyzing 19.9 million peer-reviewed papers and 2.1 million patents, Mr. Jones and his colleagues at Northwestern were able […]

Last weekend, my Head Casecolumn in the WSJ explored the increasing importance of scientific teamwork:

Benjamin Jones [an economist at Northwestern] has found that scientific teams have become a far more important part of intellectual production. By analyzing 19.9 million peer-reviewed papers and 2.1 million patents, Mr. Jones and his colleagues at Northwestern were able to show that teamwork is a defining trend of modern research. Over the last 50 years, more than 99% of scientific subfields, from computer science to biochemistry, have experienced increased levels of teamwork, with the size of the average team increasing by about 20% per decade.

This shift is even more pronounced among influential papers. The most cited studies in a field used to be the product of lone geniuses, Mr. Jones has shown, but the best research now emerges from groups. It doesn’t matter if the scientists are studying particle physics or human genetics. Papers by multiple authors receive more than twice as many citations as those with one author. This trend is even more apparent when it comes to “home run papers”—those publications with at least 1,000 citations—which are more than six times as likely to come from a team.

One explanation for this shift is the necessity of interdisciplinary collaborations: the most complex problems can no longer be solved by people with expertise in a single field. However, there is another related possibility: science is getting harder. Last year, Samuel Arbesman, a research fellow at Harvard Medical School, published a paper in Scientometrics that documents the increasing difficulty of scientific discovery. By measuring the average size of discovered asteroids, mammalian species and chemical elements, he was able to show that, over the last few hundred years, these three very different scientific fields have been obeying the exact same trend: the size of what they discover has been getting smaller.

Consider asteroids. According to the data, the average diameter of newly discovered asteroids in 1850 was about 250 miles across. By 1950, that size had decreased to about 10 miles, and by 2000 astronomers were forced to look for asteroids in the sky that were less than a mile in diameter. According to Arbesman, it’s not that asteroids are shrinking – it’s just that all the big ones have already been found. As a result, scientists are forced to search far and wide for smaller chunks of cosmic ice and rock.

The same trends hold true for animal species and new elements, which leads Arbesman to conclude that modern scientists must work harder to find new facts. Last year, he summarized the implications of his paper:

If you look back on history, you get the sense that scientific discovery used to be easy. Galileo rolled objects down slopes. Robert Hooke played with a spring to learn about elasticity; Isaac Newton poked around his own eye with a darning needle to understand color perception. It took creativity and knowledge to ask the right questions, but the experiments themselves could be almost trivial.

Today, if you want to make a discovery in physics, it helps to be part of a 10,000 member team that runs a multibillion dollar atom smasher. It takes ever more money, more effort, and more people to find out new things.

What does this have to do with scientific teamwork? The difficulty of modern science means that scientists must work together, pooling their resources and brainpower. They have to share expensive equipment and collaborate with colleagues in different domains. While the pace of discovery has remained fairly constant - we’re still finding new asteroids, for instance – the nature of what we’re discovering has led to dramatic changes within the scientific process.

I think it’s also interesting to contemplate these studies in light of Tyler Cowen’s recent e-book,The Great Stagnation, Cowen argues that our current economic problems are rooted in a larger innovation failure, as the outsized gains of the 20th century (in which living standards doubled every few decades) have given way to a growth plateau:

How did we get into this mess? Imagine a tropical island where the citrus and bananas hang from the trees. Low-hanging literal fruit — you don’t even have to cook the stuff. In a figurative sense, the American economy has enjoyed lots of low-hanging fruit since at least the seventeenth century: free land; immigrant labor; and powerful new technologies. Yet during the last forty years, that low-hanging fruit started disappearing and we started pretending it was still there. We have failed to recognize that we are at a technological plateau and the trees are barer than we would like to think. That’s it. That is what has gone wrong.

While Cowen proposes various fixes for this innovation stagnation, such as increasing the social prestige of scientists, I think it’s also worth contemplating the disturbing possibility that our cresting living standards might ultimately be rooted in the difficulty of making new scientific discoveries. After all, at a certain point the pursuit of reality is subject to diminishing returns – our asteroids will get so small that we’ll stop searching for them.

In his *Scientometrics paper, Arbesman points out that, in a few rare instances, we’ve already reached this “end of discovery” phase. Consider medicine: For thousands of years, humans documented the discovery of new internal organs. But that process of discovery is over - the last new organ to be identified was the parathyroid gland in 1880. *While we’re certainly not close to the end of science – so many profound mysteries remain – we should be prepared to work harder for what we learn next. All the low-hanging facts have been found.

Cross-posted at Ideas Market