Could Gambling Save Science: Encouraging an Honest Consensus

To appear in Social Epistemology, 1992. (version appeared: in Proc.
Eighth Intl. Conf. on Risk and Gambling, London, 7/90.)

C O U L D G A M B L I N G S A V E S C I E N C E?
Encouraging an Honest Consensus

by Robin Hanson
Visiting Researcher, The Foresight Institute
P.O. Box 61058, Palo Alto, CA 94306 USA
[email protected] 510-651-7483

The pace of scientific progress may be hindered by the tendency of our
academic institutions to reward being popular, rather than being right. A
market-based alternative, where scientists can more formally "stake their
reputation", is presented here. It offers clear incentives to be careful
and honest while contributing to a visible, self-consistent consensus on
controversial (or routine) scientific questions. In addition, it allows
patrons to choose questions to be researched without choosing people or
methods. The bulk of this paper is spent examining potential problems with
the proposed approach. After this examination, the idea still seems
plausible and worth further study.

INTRODUCTION

After reviewing the discrepancy between what we want from academic
institutions and what we get from current institutions, a market-based
alternative called "idea futures" is suggested. It is described through
both a set of specific scenarios and a set of detailed procedures. Over
thirty possible problems and objections are examined in detail. Finally, a
development strategy is outlined and the possible advantages are
summarized.

THE PROBLEM

THE SCIENTIFIC REVOLUTION Four centuries ago, some Europeans complained
that the existing academic institutions were biased against them. Insiders,
it was said, were "inflated by letters" and shunned anyone who dared
"speculate on anything out of the common way" [De]. Outsiders --
astrologers, chemists, and people like Bacon and Galileo -- argued that
they and their theories should be judged by how well they agreed with
observations, and not by how they agreed with the authorities of the day
[Gal]. This was the age of utopias [Whi], as these rebels debated possible
academic reforms and imagined whole new social institutions, for both
academia in particular and society in general.

Within a century or so, the intellectual descendants of these outsiders
became the new insiders in a process now called the "Scientific
Revolution". They introduced a new respect for observations along with new
social institutions, such as the Royal Society of London, inspired by those
utopian ideals. Since then science has made impressive progress. Most
controversial issues of four centuries ago seem long settled by now, and
continued research may well settle most of today\'s controversies. Academia
can claim some credit for this, and academic institutions have continued to
evolve in response to perceived problems, formalizing publication in
journals, credit in citations, and evaluation in anonymous peer review.

PROBLEMS WITH ACADEMIA Yet little has really changed. Academia is
still largely a medieval guild, with a few powerful elites, many slave-like
apprentices, and members who hold a monopoly on the research patronage of
princes and the teaching of their sons. Outsiders still complain about
bias, saying their evidence is ignored, and many observers
[Gh,Red,SmP,Syk,Tr,Tul] have noted some long-standing problems with the
research component of academia. footnote: Teaching reform is beyond the
scope of this paper. I am content to observe that there are no obvious
reasons why the changes I will propose should make teaching worse.}

As currently practiced footnote: Early peer reviewer consisted more of
personally observing experiments and trying to reproduce analyses.} peer
review is just another popularity contest, inducing familiar political
games; savvy players criticize outsiders, praise insiders, follow the
fashions insiders indicate, and avoid subjects between or outside the
familiar subjects. It can take surprisingly long for outright lying by
insiders to be exposed [Red]. There are too few incentives to correct for
cognitive [Kah] and social [My] biases, such as wishful thinking,
overconfidence, anchoring [He], and preferring people with a background
similar to your own.

Publication quantity is often the major measure of success, encouraging
redundant publication of "smallest publishable units" by many co-authors.
The need to have one\'s research appear original gives too little incentive
to see if it has already been done elsewhere, as is often the case, and
neglects efforts to integrate previous research. A preoccupation with
"genius" and ideological wars over "true" scientific method [Gh] needlessly
detract from just trying to be useful.

Perhaps the core problem is that academics are rewarded mainly for
telling a good story, rather than for being right. (By "right" I include
not only being literally correct, but also being on the right track, or
enabling work on the right track.)