Detailed History of Experimental Economics at Caltech

Caltech has been pioneering experimental research fueled by rapidly developing theoretical models since the work of Charlie Plott and Vernon Smith in 1970s. The unique feature of research conducted by Caltech-affiliated scholars is the dialogue between theoretical and experimental methods used to provide empirical regularities and ultimately improve the predictive power of economic and political science theories. This dialogue is possible because Caltech researchers treat economic theory as a unifying language across methods and research topics. What follows is a brief survey of the milestone developments in experimental economics that took place at Caltech from the 1970s to the early 2000s.

Market design experiments are the hallmark of experimental economics research at Caltech. These experiments led to important developments in the methodology of conducting laboratory experiments. While the scope of this work is vast, the following examples help to illustrate important questions that these studies brought to the literature.

In Miller, Plott, and Smith's 1977 Quarterly Journal of Economics paper, the authors studied speculative markets and investigated whether the intertemporal competitive equilibrium price and quantity are accurate predictors of the outcomes of certain speculative markets where participants do not have perfect information about demand and supply. Experimental data showed that perfect knowledge about demand and supply is a necessary condition for the competitive model to organize the data well. Even in the presence of uncertainty, competitive forces present in this market led to the predicted outcomes. In a second paper (Review of Economic Studies, 1978), Plott and Smith studied two trading institutions: (a) the one-sided oral auction in which buyers (sellers) make repeated oral price bids (offers) for exchange of many units, one unit at a time, and (b) the posted price mechanism in which the price quotations cannot be altered (at least not without considerable cost) during the exchange period. The empirical regularities documented in this study provided a first step at describing the dynamics of behavior observed in these two institutions—a foundation for later attempts to formulate the unifying theory that explains all the observed regularities.

In the late 1970s, John Ferejohn and his co-authors addressed the problem of designing well-behaved choice mechanisms for simultaneously purchasing more than one discrete public good from among several independent alternatives (Ferejohn et al., Caltech WP, 1977; Research in Experimental Economics, 1979a; Collective Decision Making: Applications from Public Choice Theory, 1979b). The initial example that motivated studying such settings was the selection of roughly 30 television programs of fixed duration and content from more than 100 programs that were proposed to public television stations. In a follow-up study, Ferejohn, Forsythe, Noll, and Palfrey (Experimental Foundations of Political Science, 1982) developed and compared the performance of four collective decision-making procedures for simultaneously determining whether to purchase each of several discrete public goods. All four procedures involved voluntary bids by members of a collective, but they differed in the incentive operating on a member of the collective to misreport the intensities of his or her preferences for the various public goods in order to minimize the proportion of the costs of the public goods that she will have to bear.

Related to the same strand of literature, Smith (Experimental Economics, volume 1, 1979, and Scandinavian Journal of Economics, 1979) compared several public good mechanisms, all of which shared desired properties such as collective excludability, unanimity, and budget balance while differing in their ability to overcome the free-riding problem common in these types of problems. The mechanisms were evaluated experimentally in terms of how much public good they provide on average. All mechanisms improved upon standard free-riding quantity of public good provision, with auction mechanisms standing out as the most efficient.

Much of the early formal theoretical work in political science, beginning in the eighteenth century, examined the effect of agendas and voting procedures on committee decision-making. Not surprisingly, the first experiments in political science dealt with these issues and attempted to use controlled laboratory experimentation to test the predictions of formal theories. A pioneer in this line of experimental research is Charlie Plott. His work identified systematic effects of open and closed rules, the power of a committee chairman, manipulation of the order in which items are voted on, and several other phenomena that political scientists had been speculating about for years. In addition, his research represents a systematic attempt to identify what classes of models predict well in which classes of situations.

One example of this type of research is an influential paper by Fiorina and Plott (American Political Science Review, 1978), which focused on majority-rule decision-making in committees. In this paper, the authors considered 16 different models drawn from game theory, economics, psychology, sociology, and other fields, which ranged from coalition theories to personality-based theories, fairness doctrines, and agenda manipulation models. They found that the one theory that consistently predicted accurately is the majority-rule equilibrium (core), which also coincided with the notion of a Condorcet winner, developed early in social choice theory. When induced preferences resulted in the nonexistence of majority-rule equilibrium, Fiorina and Plott found that while theory predicts chaos theorem, the experimental results were very regular and can be predicted with a small modification of the majority-rule equilibrium prediction.

Another example is a paper by Plott and Levine (American Economic Review, 1978) in which they studied the influence that the agenda setter (chairman) has on outcomes reached by committees and how manipulable these outcomes are. The authors proposed a basic theoretical framework to think about this strategic situation and provided experimental evidence in support of this theory. Experimental results indicated that, within a range of circumstances, the agenda can indeed be used to influence the outcome of a committee decision.

The previously described experiments were concerned with studying group decisions. The next milestone involved looking into the individual decision-making phenomenon called preference reversal documented by psychology literature. According to this phenomenon, people tend to choose one of two available lotteries, while at the same time they attach lower monetary value to the chosen lottery than to its alternative. A paper by Grether and Plott (American Economic Review, 1979) put these empirical findings under the microscope by conducting incentivized laboratory experiments according to all established canons of experimental economics methodology. Surprisingly, they found that preference reversal is not an artifact of the differences in experimental methodology between psychologists and economists but rather a persistent phenomenon characterizing human behavior. Moreover, this phenomenon cannot be explained by applying standard economic theory or some immediate variant of it.

In the early to mid-1980s, a second wave of political economy experiments were initiated and conducted by Richard McKelvey and Peter Ordeshook, involving laboratory investigation of elections. The authors took a poorly informed electorate for granted and examined predictions of formal political science models under this condition (see Collier, McKelvey, Ordeshook, and Williams, Public Choice, 1987; McKelvey and Ordeshook, Public Choice, 1984; Journal of Economic Theory, 1985a; American Journal of Political Science, 1985b; Journal of Politics, 1986). The main research question was whether the lack of information on the part of voters means that the forces of political competition, if they exist at all, are wholly removed from the electoral arena. Surprisingly, their findings showed that the mere presence of a bewildered electorate may pose little threat to the smooth and efficient functioning of democracy. Their experiments identified two basic features that rescue voters from their own political inadequacies: endorsements by well-known interest groups and retrospective voting.

Experimental methodology lends itself naturally to studies of the specifics of market design, which have potential to affect the efficiency of allocation processes. Robert Forsythe, Mark Isaac, and Tom Palfrey (RAND Journal of Economics, 1989) used this approach and reported the results from laboratory markets in which sellers had better information about the quality of an item than any of the potential buyers. In these markets, sellers could either voluntarily reveal this information or, instead, could decide to "blind bid" the item. To fit the data, the authors considered a sequential equilibrium model in which buyers assume the worst. This equilibrium is not attained instantaneously but rather through an unraveling process. The noticeable feature of final allocations in this unraveling process is that they tend to be ex post efficient.

The first experimental tests of the centipede game were conducted here at Caltech in early 1990s by Richard McKelvey and Tom Palfrey. The results of these experiments were reported in their 1992 Econometrica paper. In this game, two players alternate in having a chance to take the larger portion of an escalating pile of money. As soon as one person takes, the game ends with that player getting the larger portion of the pile and the other player getting the smaller portion. In principle, this is a game with complete information in which equilibrium predicts that the first mover should take the large pile in the very first round. However, experimental data showed that this does not happen. The authors proposed an alternative explanation for the data based on the idea that this game is one of incomplete information in which there is some uncertainty over the payoff functions of the players (i.e., some probability that the opponent is an altruist). In this case, the theory predicted that players would adopt mixed strategies in the early rounds of the game, with the probability of taking increasing as the pile gets larger. This model provided a better description of the experimental data compared with the complete information game.

Yet another topic that Caltech researchers put under experimental examination is understanding the effects of political competition in a model of economic growth. Boylan, Ledyard, Lupia, McKelvey, and Ordeshook (Laboratory Research in Political Economy, 1991) conducted a series of experiments on a one-sector model of economic growth in which decisions on consumption and capital accumulation are made by politicians elected in a competitive political process. The basic research question was how political systems, in which candidates have limited tenures, make decisions on issues that involve capital investment planning. The focus of the paper was on comparing the paths generated by a political process with the optimal one that would be chosen by an economic planner and documenting whether there are political business cycles in the data.

More generally, the work of John Ledyard has dealt with issues of (applied) mechanism design both theoretically and experimentally in a variety of settings. For instance, the 1996 Economic Design paper by Ledyard, Noussair, and Porter dealt with the common situation in which the supply of shared resources is limited, creating natural conflicts among members of an organization as to who has priority to use these resources. The authors used a methodology of applied mechanism design to examine alternative processes for the resolution of such conflicts for a particular class of scheduling problems. The focus was on measuring efficiency (the value attained by the resulting allocations as a percentage of the maximum possible value). This paper was motivated by the problem of allocation of time on NASA's Deep Space Network, but the results provided more general insights relevant to many other situations with a similar structure. In a different paper, Ledyard and Szakaly-Moore (Journal of Economic Behavior and Organization, 1994) focused on choosing a permit trading mechanism that is both economically efficient and politically viable. In this paper, they considered an organized trading process and a revenue-neutral auction, both of which involve an initial allocation of permits based on past history. These two systems were tested in a competitive and in a non-competitive environment to determine which mechanism would perform best. The experimental results suggested that, overall, the organized trading process outperforms the revenue-neutral auction. This research was motivated by an interest in using a marketable permits program as a new way to control aggregate pollution emissions.

In many strategic situations, people seem to deviate from playing the equilibrium actions. An infamous example is the class of guessing games (beauty contest games). In a series of papers, Colin Camerer has used these games to study learning processes that govern one's behavior. In their 1998 American Economic Review paper, Ho, Camerer, and Weigelt analyzed how many steps or iterations of dominance players apply. The authors considered experimental data in light of two structural models: one that predicts first-period choices and one that proposes a model of learning across periods. The results showed that these two structural models organize the data well and that a substantial proportion of players behave as adaptive learners and are capable of best responding.

These (and related) findings led to the development of several influential models. The first is the experienced-weighted attraction (EWA) learning model, presented in Camerer and Ho's 1999 Econometrica paper. The EWA combines elements of reinforcement learning models and belief-based learning models, and as a result includes both models as special cases of the more general EWA setup. The authors tested the empirical usefulness of EWA by deriving the maximum-likelihood estimates of it from three data sets: constant-sum games with unique mixed-strategy equilibria, coordination games with multiple Pareto-ranked equilibria, and beauty contest games with unique dominance-solvable equilibria.

The second model is a cognitive hierarchy model that preserves the best-responding feature of equilibrium behavior but relaxes the assumption that all players hold correct beliefs about others. Camerer, Ho, and Chong, in their 2004 Quarterly Journal of Economics paper, developed a general model of this kind in which each player assumes that his strategy is the most sophisticated. Specifically, the cognitive hierarchy model posits decision rules for players that reflect an iterated process of strategic thinking: a player that is capable of k steps of thinking best responds to actions of other players whom she believes are capable of at most k – 1 steps of thinking. The authors applied this model to a few dozen beauty contest experiments conducted in various locations with different populations and different specifications of the game and showed that the estimated parameter of cognitive hierarchy model is quite consistent across these data sets.

Taking a different approach, Richard McKelvey and Tom Palfrey formulated a different model of behavior that incorporates human errors in a coherent way: the quantal response equilibrium (QRE). This equilibrium concept quickly became one of the standard approaches used to fit experimental data. In McKelvey and Palfrey's 1995 Games and Economic Behavior paper, the authors considered players who choose strategies based on relative expected utility and assume that other players do so as well. The QRE is defined as a fixed point of this process, and the authors established its existence. For a LOGIT specification of the error structure, the authors showed that as the error goes to zero, QRE approaches a subset of Nash equilibria and also implies a unique selection from the set of Nash equilibria in generic games. The authors also fitted the model to a variety of experimental data sets using maximum likelihood estimation.

Experimental methodology was used by Caltech researchers to understand network formation processes. Jacob Goeree, Arno Riedl, and Aljaz Ule, in a 2009 Games and Economic Behavior paper, used a controlled laboratory environment to study network formation among homogeneous and heterogeneous agents, basing their design on a modified version of the Bala-Goyal (2000) model. The experiment was designed in such a way that in all treatments, the efficient equilibrium network has a star structure. Experimental data showed that, with homogeneous agents, a star network is almost never formed. However, with heterogeneous agents, stars frequently occur, often with the high-value or low-cost agent in the center. Interestingly, the stars are not born but rather develop over time: dynamic patterns show that with a high-value agent, the network's centrality, stability, and efficiency all increase over time.

Another strand of literature pursued by Caltech researchers in the early 2000s pertains to problems of information aggregation in collective decisions. The theoretical work by Leeat Yariv and Dino Gerardi in their 2007 Games and Economic Behavior paper and the follow-up experimental study conducted in collaboration with Jacob Goeree studied the effects of different institutions (i.e., voting rules) and preference alignment between committee members on outcomes when communication channels are available and group members deliberate the issue at hand before undertaking a vote. Goeree and Yariv, in a 2011 Econometrica paper, documented that without deliberation, different institutions generate significantly different outcomes, consistent with theoretically predicted comparative statics results. However, deliberation significantly diminishes institutional differences and uniformly improves efficiency.

This brief overview cannot cover all of the important experimental work that has been done at Caltech: it shows the tip of the iceberg. The current researchers affiliated with the Institute are continuing this fruitful dialogue between theoretical analysis and experimentation and expanding it to new fields in economics, political science, and social science more generally.