How gamblers can save science

A suggestion to segregate good science from bad science

Scientific journals are often considered to be the voice of experts in many fields of science today. But just how reliable are the studies published in the world’s top scientific journals? A major crisis in the scientific world is that the results of most academic studies, including the peer-reviewed ones, do not replicate properly. This is popularly known as the replication, or reproducibility, crisis. The failure of scientific results to replicate suggests that these studies are quite unreliable even though they have the stamp of approval from academic experts. Many critics of the scientific establishment have even blamed the nature of academia, which they liken to a medieval guild dominated by a small group of elites calling the shots, for this crisis.

If the process of academic peer-review cannot be trusted to produce reliable science, what are the alternatives? “Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015,” a 2018 paper by Colin Camerer and others, has found that prediction markets can do a remarkable job when it comes to segregating good science from bad science. A prediction market is any form of open market where people with money are free to bet on various scientific studies. It is believed that such a market could be particularly helpful in saving unpopular scientific ideas. A rebel scientist, for instance, could sell shares in his unpopular idea that has been rejected by universities to interested investors. Investors who bet on such unpopular research could make big bucks if their predictions come true. And if the results of the study are to be believed, investors get things right more often than academics.

Camerer et al tried to replicate the findings of 21 studies published in the world’s top two scientific journals —

Nature and

Science . They found that 13 out of the 21 studies, or 62% of the results, could be replicated.

At the same time, they allowed 206 volunteers to bet on the chances of replicability of the same group of studies. If they were right about their predictions, these volunteers were set to gain money from their bets. To their surprise, the researchers found that this group of volunteers looking to make money also predicted that the results of 63% of the studies would be replicable.

So, according to these findings, getting people to put their money where their mouth is could turn out to be a pretty good way to save science.