Forecasters as imperfect information processors

Material Information

Forecasters as imperfect information processors experimental and survey evidence
Downs, David L
Place of Publication:
Denver, CO
University of Colorado Denver
Publication Date:
Physical Description:
v, 50 leaves : illustrations ; 29 cm


Subjects / Keywords:
Economic forecasting ( lcsh )
Rational expectations (Economic theory) ( lcsh )
Economic forecasting ( fast )
Rational expectations (Economic theory) ( fast )
bibliography ( marcgt )
theses ( marcgt )
non-fiction ( marcgt )


Includes bibliographical references (leaves 45-47).
Submitted in partial fulfillment of the requirements for the degree, Master of Arts, Economics
General Note:
Department of Economics
Statement of Responsibility:
by David L. Downs.

Record Information

Source Institution:
|University of Colorado Denver
Holding Location:
|Auraria Library
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
37753838 ( OCLC )
LD1190.L53 1996m .D69 ( lcc )


This item has the following downloads:

Full Text


Forecasters As Imperfect Information Processors: Experimental And Survey Evidence by David L. Downs B.S., University of Illinois, M.E., University of Colorado, A thesis submitted to the University of Colorado at Denver in partial fulfillment of the requirements for the degree of Master of Arts Economics


This thesis for the Master of Arts degree by David L. Downs has been approved by Steven R. Beckman W. James Smith Steven G. Medema


Downs, David L. (M.A., Economics) Forecasters As Imperfect Information Processors: Exper-imental And Survey Evidence Thesis directed by Assistant Professor Steven R. Beckman. ABSTRACT One step-ahead forecasts of random walks with four different variances are collected from student subjects. The increasing variance creates a more complex learning environment and engenders forecasts that deviate more from the rational expectation. A one percent increase in the standard deviation of information.released after the forecast date creates a 0.9% increase in the standard deviation of forecasts from rational expectations. The effect is remarkably strong and is even stronger in sur-vey data collected from professional forecasters. One potential application is the stock market as random shocks undermine consensus and induce volatility which undermines consensus. We also find that deviations from rational expectations are serially correlated. This abstract accurately represents the content of the candidate's thesis. I recommend (/ Signed Steven R. Beckman


DEDICATION This thesis represents the combined work of Dr. steve Beckman Ph.D., Assistant Professor of Economics, University of Colorado at Denver (UCD) and myself. This work is based upon a series of economic experiments performed in 1993 and 1995 using several UCD student subjects. Preliminary results of this experiment were presented at the 1993 Economic Science Association meeting held in Tucson, Arizona. Particular thanks are due to Gerald Dwyer, Mahmoud El-Gamal and Richard DeGrandpre and two anonymous referees. A shorter version of the thesis is forthcoming at the Journal of Economic Behavior and organization.


CONTENTS Chapter 1 Introduction . . . . . . . . . . . . . . . . . . 1 2. The Philosophy of Economic Experimentation .... 5 2.1 Standard Economic Experimentation Practices .. 6 2.2 Motivations to Perform Economic Experiments .... 8 3. Literature SurveyExpectation Formation ..... 11 3.1 Bounded and Near Rational Behavior ............ 13 3.1.1 The Impacts of Near Rational Behavior ....... 14 3.1.2 Effects on Experimental Design .............. 15 3.1.3 Previously Performed Experiments .......... 16 3.2 Forecasting in an Asset Market Context ........ 17 4. The Experimental Context ........................ 22 4.1 The Experimental Design and Procedures ..... 23 4.2 Survey and Experimental Results ......... 27 5. Conclusi-on . . . . . . . . . . . . . . . . . . . 41 Appendix A. Instructions . . . . . . . . . . . . . . . . . . 4 5 References References . . . . . . . . . . . . . . . . . . . . . 4 8 v


1. Introduction The collaborationl between John Muth and Herbert Simon at Carnegie Mellon produced two distinct theories of expectation formation. Herbert Simon (1959, p. 272) argued that, "The classical theory is a theory of a man choosing among fixed and known alternatives, to each of which is attached known consequences. But when percep-tion and cognition intervene between the decision-maker and his objective environment, this model no longer proves adequate." In Simon's view, people are exposed to more information than they can process and must simplify. Muth takes the diametrically opposed position that dynamic economic models do not assume enough rationality on the part of decision makers. Expectations, he argues, "are essentially the same as the predictions of the rele-vant economic theory." The distinction between the theo-ries is determined by whether large and systematic deviations of forecasts from theoretically correct predictions can be traced to cognitive limitations. 1 The two collaborated on Planning Production, Inventories and Work Force (1960, Prentice Hall) along with Franco Modigliani and Charles Holt. 1


Both theories have strong theoretical foundations. Muth posits that forecasters should not be assumed to make systematic and costly errors in equilibrium because errors will induce learning and an evolution toward theoretically correct behavior. But truth. is an illusive concept. If objective truth is unobtainable, then the processes people use to simplify and cope with reality may be a worthy area of study. These two arguments alone are sufficiently strong to produce a theoretical stalemate. If one accepts the idea that people must simplify then the possibility that the rational expectations theory represents a useful simplification must be entertained. If we insist that forecasters are capable of learning about the true nature of the economy then they must also be capable of learning about their limitations. The issue then is inherently empirical. The goal is to construct environments where human limitations may lead to interesting and systematic deviations in behavior from strict rationality. In the experiments reported below subjects are placed in a series of environments that, from the point of view of economic theory, are essentially the same and yet place greater or lesser burdens on human cognition. If the 2


change in environments produces no systematic or interesting changes in behavior, then the results will support the rational expectations hypothesis of John Muth. But, if the environmental changes produce predictable and reliable changes in behavior, then the value of research in human cognitive limits will be partially revealed. The core concept behind the experiment is to place subjects in forecasting environments that are different only in the amount of noise added to the series. Rational behavior is still reinforced by monetary incentives but reinforcement weakens as noise increases creating a more difficult learning environment. Any changes in human behavior will be traceable to limitations in the human capacity to learn and respond appropriately. We find that as the objective environment becomes noisier, deviations from rational expectations increase. While simple and obvious ex-post, the behavior is not predicted by the rational expectations theory or a well-known theory based on cognitive limits. Keynes (1936, p. 148) argued that it is "our usual practice .. to take the existing situation and project it into the future, modified only to the extent that we have more or less defi-3


nite reasons for expecting a change." In fact, we do not observe greater and greater emphasis placed on the existing situation as noise is increased.2 We believe our work has a close parallel in stock market research. There is currently an ongoing debate as to why stock prices are excessively volatile as price changes may occur in the absence of any additional market information (Buckley, 1989). David Romer (1993) suggests the stock market is affected by confidence. At the risk of oversimplifying, Romer hypothesizes that large price changes signal that other investors lack confidence and thus contribute to future price changes. Our experiments indicate a similar chain of causation may operate through more erratic valuations as the learning environment becomes more complex. The more erratic valuations may then contribute to more complex stock prices and more erratic valuations. 2 It was Beckman's prior belief, based on great respect for Keynes that subjects' behavior would become more predictable and more heavily anchored on recent events. David Downs used his background as an electrical engineer to surmise that subjects would act as imperfect filters and some of the environmental noise would pass through to the subjective forecast. 4


2. The Philosophy of Economic Experimentation As currently practiced, neoclassical economic theory is ideally suited for experimental investigation. Neoclassical theories specify a model in order to make predictions about future behavior. Rational expectations theory, which is one of the most important neoclassical theories today, asserts that economic agents know the true model of the economy. By using all of the information available to them the economic agents solve the model appropriately to form expectations. In order to test such a theory one can, in principle, reproduce the theory in the laboratory. Economic data generated under the controlled conditions of an experiment, one which reproduces the theory the experiment is designed to test, appears tobe better suited to test the theory than data generated under other (i.e. natural or 'wild') conditions. One of the prime motivations for today's experimental economics research program is this ability to collect economic data under controlled conditions. This data can then be used to test the relevant economic theory under 5


the same conditions which the theory was formulated. These new theories can then be scrutinized under experimental testing (Davis, 1993). In economics, experimental testing was implemented long after the theoretical stage was established. Therefore, much of todays experimental testing is to validate already accepted economic theories. One needs to know that an economic theory works on its own, (i.e. logical, self consistent etc.) and that it survives the transition into the real world. As will be shown, using the three step process of the scientific method, experiments can help bridge the gap between the theory and the observation of the real world (Pheby, 1988). 2.1 standard Economic Experimentation Practices As a science matures, several standardized practices regarding methodology and technique begin to evolve. In the natural sciences this includes independent replication of the experiment before it becomes acceptable to the scientific community. 6


The importance of independent replication was recently reinforced in the 'cold fusion' hoax. As the heavy water 'cold fusion' experiment was not independently reproducible, it was declared a fraud and has been discredited in the literature. In experimental economics there are three basic ingredients of an experiment: 1. The environment; 2. The institution; 3. The observed behavior (Smith, 1991). 1. The Environment. This specifies the initial endowments, preferences, and costs which motivate exchange. It should always be controlled using monetary rewards to induce desired behavior. 2. The Institutions. This defines the language, messages, market communication, and rules which governs the exchanges in the experiment. 3. The Observed Behavior. The observation of the participants in the experiments as a function of the environment and institute. The data analysis resulting from the observed behavior will usually involve the well established techniques of econometrics. 7


2.2 Motivations to Perform Economic Experiments Using the framework of the environment, the institu-tion, and the observed behavior, Vernon Smith (1994) has established seven prominent reasons to perform economic experiments: 1. Test a theory or discriminate between theories. 2. Explore the causes of a theory's failure. 3. Establish regularities as a basis for a new theory. 4. Compare environments. 5. Compare institutions. 6. Evaluate Policy Proposals. 7. Testing ground for institutional design. Each of these seven reason has an independent justifica-tion: Test a theory or discriminate between theories. Fol-lowing Popper's hypothesis any good scientific theory when brought to the test, is vulnerable. It could be falsified. The greater the frequency with which the experimental observations match the predictions of the theory, the better the theory. ExPlore the causes of a theories failure. Well articulated theories, when faced with repeatable exper-imental falsification, can allow the experimentalist to discover the boundaries or constraints under which the theory does hold. After discovering the specific exper-imental conditions under which the theory properly per-8


forms, a new and more powerful theory can then be developed which can explain both the successes and failures of the old theory. Establish empirical regularities as a basis for a new theory. Historically, the discovery of empirical regularities preceded many new theories. Kepler's laws of planetary motion took place prior to Newton's law of gravity. An example of an empirical regularity was the discovery of the Phillips curve. The Phillips curve was not a theory but a statistical observation that low unemployment went hand in hand with high inflation. For government policy makers, who were talking about the trade off between inflation and unemployment, it all seemed to fit. Governments were told that maintaining a low inflationary economy would mean throwing people out of work. Compare environments or institutions. Leaving the institution (environment) fixed and varying the environment (institution) will allow the experimentalist to test the robustness of the institution (environment). This is important in discovering the domain of applicability for many theories. In our experiment, we fixed the institution and varied the environment. The institution included the use of an identical computer interface and 9


identical rules which governed the transactions in the experiment. The only environmental parameter which was varied was the size of the variance in the random walk. Evaluate policy proposals and as a testing ground for institutional design. Experiments offer an inexpensive, in both monetary and political costs, alternative to the full scale implementation of various uncertain policy proposals. 10


3. Literature survey -Expectation Formation At the very foundation of neoclassical economics stands an assumption regarding individual behavior: economic agents are presumed to act in a rational, selfinterested manner. This stems from Muth's well known definition of the rational expectation hypothesis: .. that expectations ... tend to be distributed, . about the predictions of the theory." For complex circumstances, this may require the agents to act as if they can solve a set of multidimensional simultaneous equations. The relevance of many neoclassical theories depends on whether agents behave as if they make such calculations. The question is then asked, "Do agents act as if they perform such calculations?" This question is particularly important with the rational expectation paradigm pre-eminent in the world of economic theory. The interpretation of the rational expectations hypothesis is that people make predictions about the real world 'as if' they knew the true model. Forecasts are rational if there is no systematic, correctable difference between the forecasts and the resulting actuals. 11


Due to extraneous background noise theresulting forecasts might still be wrong, however the errors would be random containing no extractable information. The critical focal point in rational expectations is that the forecast errors are. random, containing no extractable information. This leads to the interesting conclusion that announced changes in the money supply by the central bank will not cause any effect on output or employment. As soon as the central bank announces that the money supply is going to increase, people jump to the rational conclusion that inflation will rise and people therefore adjust their expectations to account for the increased price levels. As Messrs. Sargent and Wallace (1975) stated: "In order for the monetary authority to induce fluctuations in real output, it must induce unexpected movements in the price level But by the virtue of the assumption that expectations about the price level are rational, the unexpected part of price movements is independent of the systematic part of the money supply, as long as the authority and the public share the same information. There is no systematic rule that the authority can follow that permits it to affect the unexpected part of the price level." p 244 Clearly, if expectations are not purely rational, then some other outcome will probably occur. Formerly, the most used alternative model concerning expectation 12


formation was adaptive expectations. Adaptive expectations model the world by putting various weights upon past observations. Usually a higher weight is placed upon the more recent years with the weight trailing off as the data becomes older. In an interval where the economic variable of concern is constantly accelerating, the adaptive expectation model will consistently under predict the variable. The forecasting errors would be non-random. In this example, the adaptive expectation forecasting error would be persistently negative and increasing in size. Thus, the errors would contain information which could be exploited to make more accurate forecasts. 3.1 Bounded and Near Rational Behavior A new series of non-adaptive/non-rational expectation formation is now being developed in the literature. This behavior is close to, but not perfectly, rational. Known as 'near-rational' or 'bounded' rational behavior, studying these effects on economic systems is very prevalent in the literature. A definition of near-rational' or 'bounded' rational behavior can be found in Akerlof and Yellen (1985). They define near-rational behavior to be behavior that is per-13


haps suboptimal but nevertheless imposes very small individual losses on its practitioners relative to the consequences of their first best policy. Others, following Herbert Simon's inspiration, have stressed that rationality may be 'bounded'-that is, people try to make self-interested choices but are limited by lack of information and the cost of gathering and interpreting it. 3.1.1 The :Impacts of Near Rational Behavior There are several papers in the literature which discuss the impacts of 'bounded' rational behavior on various economic situations including financial markets and forecast planning for capital accumulation. Wang (1993) argues that despite the insignificance of the near-rational behavior from individual standpoints, their presence in financial market$ amplifies the effects of transient shocks on the market. The impact of nearrational agents on the market is found to be disproportionately large causing asset prices to become excessively volatile arid deviate significantly from their rational equilibrium levels. Sterman (1989) constructs a multiplier-accelerator interaction experiment where subjects must forecast demand in order to correctly plan their capital accumula-14


tion path. Forecast errors create the potential for chaotic behavior which emerges in 22% of subjects. Sterman concludes that chaos can result from cognitive limitations and a relatively simple dynamic economic model. 3.1.2 Effects on Experimental Design The design of economic experiments must account for the impacts _of 'bounded' rational behavior. Timmerman (1994) is concerned that the literature does not consider two questions in regards to rational expectations. First, can agents learn to become rational, and how long does this take? Second, are the forecasts agents make, using the rational hypothesis, stable? He illustrates his point by showing that most empirical analyses of economic time series do not test the rational expectation hypothesis directly and, if such an attempt is made, only do so by using the entire sample of data points. Using the entire data set amounts to assuming that agents form rational expectation forecasts instantaneously throughout the sample. This ignores the notion that they may only gradually learn to form rational expectation forecasts. Timmerman also details two conditions which should be satisfied to ensure that the 15


assumption of rational expectations is properly applied in empirical work: 1. Agents expectations must converge to the rational expectations equilibrium. and 2. Their expectations should not display large fluctuations away from rational expectations on their path towards equilibrium. The necessity of properly motivating the economic agents during the experiment is the subject of a paper by Harrison (1992). He argues that many experiments in economics are invalid, or at least questionable, due to a near'-rationality effect. Harrison calls this the "payoff dominance" critique. Dominance of the rewards over subjective costs (or benefits) to a subject from participating in the experimental task is a particularly important concept. In practice it requires that the rewards corresponding to the null hypothesis are appreciatively greater than the rewards corresponding to the alternative hypothesis. 3.1.3 Previously Performed Experiments The results of previously performed experiments have implied the existence of near rational behavior. Mark Pingle (1992) argues that bounded rationality may be modeled by adding decision costs to the optimization problem 16


and provides supporting experimental evidence. He finds that adding decision costs forces subjects to search for the optimum from below. Williams (1987), using the results of a double auction experiment, has determined that many forecasts are not efficient and may be adaptive. He then describes a potentially fruitful course of work: Is a necessary condition for the existence of strict Muthian rationality created when a market converges to the point where subjects are forecasting the mean of a stable, lowvariance price distribution? All these papers emphasize that pure rationality may be an unrealistically restricted view of human conduct. Therefore, the process people use to simplify and cope with reality is an area worthy of study. 3.2 Forecasting in an Asset Market context The use of experiments in economics to determine how agents make forecast decisions has been to date quite limited. Although the use of direct empirical investigation has been talked about by many economists, remarkably few such attempts have been made, and relatively little attention has been paid to their findings (Hey, 1991). 17


In one of the first experimental studies of expectation formation, (Williams, 1987) forecasts are collected from subjects engaged in computerized auction markets. Conventional regression tests of forecast bias indicate a very strong bias in the Williams data even though histograms of forecast errors are nearly symmetric and centered at zero. The apparent contradiction is explained by Williams's need to estimate the rational forecast leading to simultaneous equation bias (Beckman, 1992). A similar debate exists over the proper use of survey data. Keane and Runkle "(1990) are able to overturn previous rejections of the rational expectations hypothesis by applying more appropriate econometric procedures to the same survey data. Experimental economists have the great advantage that the econometric problems of survey data may be avoided by carefully constructed experiments. In a multiperiod asset market context (i.e. a stock market simulation experiment) (Smith et al. 1988) traders submitted forecasts of the mean price in a simulated securities market for the next trading period. Smith discovered that a market with shares having a well defined and commonly available information on fundamental dividend value can self generate price bubbles. This, of 18


course, is followed with an eventual (and wholly expected) 'crash'. The results indicate that: 1. Forecasters failed to provide an unbiased prediction; 2. Both the group average and separate individual forecasts shows a tendency to over predict the mean price, and consistently failed to predict turning points (i.e. Booms or Crashes) in the time series; 3. Better (i.e. more rational) forecasters earned more money; 4. Forecasts are highly adaptive, and the forecasting errors are autocorrelated; 5. Both the mean deviation and variance decline significantly with increasing subject experience. When the asset value "fundamentals" remain unchanged over the horizon of trading: the subjects were unable to forecast the turning point in the shares price immediately in advance of the trading period where the crash occurred. His results also support the view that expectations are initially adaptive with the speed of adaptation coefficient converging over time to a rational expectation equilibrium. These experiments indicate that the subjects are at worst adaptive, and at best 'near rational'. For example, consider the more recent experiment by Dwyer, Williams, Battalio and Mason (1993) in which sub-19


jects report one step ahead forecasts of a random walk. The rational expectation is both simple and clear: subjects should report the previous observation as the forecast. There is no need to estimate the rational expectation and therefore there is no possibility of rejecting rational expectations through a faulty esti-mate. The experimenters have deliberately constructed a stark environment to allow a clean test of the rational expectations hypothesis. The experimental results showed that the forecasting errors formed a near normal distri-bution closely centered on the rational expectation value. Their results indicate that the forecasts are reasonably well described as the rational expectation plus or minus a random error and therefore provide sup-port for rational expectations. However, the subjects failed to converge to the pure rational expected state of simply mirroring back the previous observation. This again showed a 'near rational' behavior which distressed Dwyer who states: It is possible that subjects would have smaller deviations from rational expectations if just given more observations. Nonetheless, further reductions in the variance around rational expectations may occur slowly, if at all. Suppose that learning occurs by continuing behavior that is followed by increased earnings and changing behavior that is followed by decreased earnings. If some constant is added onto the rational expectation in a period, say o.os, this 20


deviation decreases expected earnings. But ex-post, it decreases sometimes and increases earnings other times. Of the forecasts different from rational expectations in the last 30 periods, only 51.1% of them generated lower actual earnings than rational expectations. Not only may it be hard for subjects to detect deviations for the maximum expected earnings, but ex-post many forecasts different than rational expectations are being rewarded. This state of affairs may not be so different from market settings, where doing the wrong thing can turn out to be best. Such unpredictable deviations from rational expectations have non-trivial implications in many economic situations. [Italics Added] 21


4. The Experimental context This paper examines one step ahead forecasts of random walks because the rational expectation is so well defined. The main innovation in our experiment is contained in the data to be forecast. Let A(t) represent the actual value at time t and F(t) the one step ahead forecast constructed in t-1. The actuals used in our experiment were generated by a first order autoregressive function: A(t) = A(t-1) + v(t) The error term (v) is generated randomly and is uniformly distributed with a mean of zero. Thus, the expected value of the next actual is equal to the actual of the previous round: E(A(t)) = A(t-1). The Muth's rational expectations hypothesis assumes that the forecasters know the true model and are therefore able to form correct expectations. Similarly to the actual realizations, the forecasts (F) are therefore equal to the actuals expected value E(A) plus an error term (w) which is given by: F = E(A) + w. 22


Using the expected value of the actuals equal to the previous time step: E(A(t)) = A(t-1), the actuals and forecasts can be expressed as: A(t) = A(t-1) + v(t) F(t) = A(t-1) + w(t) Muth assumed that v was unbiased (having an expected value of 0) and uncorrelated with A. If these criteria are not met, then the-separation between present and future expectations has not been achieved. 4.1 The Experimental Design and Procedures our goal is to observe if there is some association between w and v. If human cognitive limits impose sys-tematic deviations from classical rationality, these deviations should be more pronounced and visible as the size of the random shock is increased. Subjects are placed in four different environments distinguished by the size of the random error. In treat-ment I, v is drawn from a uniform 3 distribution of whole 3 Dwyer et. al. (1993) run the random number generator and then select series that pass a battery of tests. We simply run the random number generator. We believe our procedure is more appropriate to our circumstances because subjects will see four series. If runs are selected to have no trend, for example, then subjects will rationally expect mean.reversion near the end of the experiment. Dwyer's subjects see only one series 23


numbers centered at zero and ranging from -5 to +5. Therefore, in treatment I, there are 11 equally likely possible outcomes. In treatments II, III, and IV the whole numbers comprising v range from =10,= 15 and = 20. Each subject received all four treatments but in a unique order. Exposing each subject to all four treat-ments allows us to measure a treatment effect uncontami-nated by subject to subject variation. Similarly, varying the order of the experiments avoids contaminating the treatment effect through a systematic association with order effects. Order effects may exist for a vari-ety of reasons such as learning, restlessness or past earnings. There are 24 different orders of 4 treatments (24 = 4!), accordingly, two sets of 24 student subjects were recruited from economics and political science classes in the summers of 1993 and 1995 and randomly matched to the 24 treatment orders. The experiment itself is conducted by a computer program4. The instruc-tionsS and an example of the computer screen are repro-duced in Appendix I. 4 Available on request from the authors. 5 The instructions between rows of asterisks appear on the computer screen at one time. The 1995 instructions may be constructed from the.1993 instructions by replacing material in{} with the contents of [ ]. 24


As the experiment begins, subjects know only the initial observation which is either 100 [1993 Experiment] or 500 [1995 experiment]. The subject is asked for a forecast, and then the next actual, A(t) = A(t-1) + v(t), is revealed. Forecasts are collected for 100 rounds in each treatment for a total of 400 forecasts per subject. In the 1993 experiments the subject is paid $11.00 less a penny for each unit of forecast error. If the subject does echo back the previous observation as the forecast then F(t) -A(t) = v{t) and the expected value of each individual error for treatment I is equal to: 2(5+4+3+2+1)/11 = 2.727 cents. over 100 observations the expected forecast errors under strictly rational behavior sum to $2.73 leaving an expected payment of $11 -$2.73 = $8.27. Similarly, the expected payment for the remaining6 treatments are $5.76, $3.26 and $0.76 for an expected total, assuming strict rationality, of $18.05. The experiments last roughly 1 to 1.5 hours so that the expected rational average hourly earnings exceed $12. 6 Given the fixed positive payment and potentially unlimited deductions, it is possible to end a treatment with zero pay. In treatment IV, 18 of 24 subjects were unpaid. 25


Preserving the base pay of $11.00 implied average pay declined perhaps contributing to deviations from strict rationality as noise increases. Therefore, in 1995 we conducted a second similar set of experiments with base pay of $8, $11, $14 and $17 for the four treatments. If we recalculate average pay in the 1993 experiments with this base pay, earnings would have been $4.77, $4.88, $5.10 and In this way we hoped to have two sets of experiments with rising variance, one that holds base pay constant and the other that prevents average pay from declining. Average pay in the 1995 experiments were $4.74, $5.15, $5.31 and $5.74 indicating? average pay did not decline. If subjects do not echo back the previous observation as the forecast then the expected monetary penalty for deviating from rationality is partially dependent on the variance of the random shock, v. For example, if the variance is zero, then a one unit forecast error will certainly cost a penny. But as the variance increases, the probability that the actual value will be different 7 Only subject 24 The next lowest pay with a base pay of $14 was unpaid. for any treatment was $3.43. 26


from the previous observation rises and forecast errors may actually be rewarded, creating a more difficult learning environment. 4.2 survey and Experimental Results Figure 4.1 presents histograms for the two experimental data sets and a survey of professional forecasters currently conducted by the Philadelphia Federal Reserve Bank. The survey was initiated by the American Statistical Association and the National Bureau of Economic Research in 1968 and questionnaires are mailed to broadly based and diverse group of professionals engaged in the analysis of current and prospective business conditions. Many topics are covered. We focus on forecasts of the implicit price deflator for different forecast horizons. For experimental data the histograms are of w, the deviation of forecasts from previous observations. The survey histograms represent the deviation of individual forecasts from the mean forecast. In all three data sets, as more time elapses or variance of v rises, there is greater dispersion about the mean. A more variable realization reduces consensus. 27


10 0 QJ ..... 0 u. 0 QJ N Cl 10 00 'E QJ 0 ..... QJ a. Distribution of Forecasts About the Rational Expectation Survey of Professional Forecasts 1993 Experiment 1995 Experiment of the Implicit Price Deflator 30 30 JO 25 2sJ I I 25 I 20J r-. Current Quarter 20 20 Low noise Low noise 15 15 15 10 High noise 10 High noise Quarters Ahead 5 .5 5 l -------;2. \.......:'--, __ 0 0 -........................ Or;, 1 1 1 1 1 -5 0 5 -5 0 5 -5 0 5 Deviation From the Rational Expectation Deviation from the Rational Expectation Deviation From Consensus Forecast Figure 4.1 -Distribution of Forecasts About the Rational Expectation


In the 1995 experiments, it is apparent that the forecasts are quite strongly centered at the rational expectation. Indeed these histograms are decidedly non-normal because they are too strongly peaked. In contrast the histograms for 1993 are bimodal about the rational expectation. This is the most unexpected and puzzling difference in the data sets. We do not know what accounts 8 for the difference. Curiously, the profes-sional forecasts for 3 quarters ahead are also bimodal, with the mean 9 forecast between the two modes, just as in the 1993 experiments. It is important to make the connection betweeri the experimental and survey data sets. In general a series may be represented as a function of the information at the time the forecast is made and information that will be released after the forecast date as At = f(It_1 ) + f(It -It_1). The forecast then is Ft = f(It_1 ) + u where u 8 We recruited from similar courses, some upper and some lower division, in and out of the department but we did switch from day to night students because recruiting was easier. The 1993 subjects may have observed recruiting difficulties creating a selection bias toward cooperative subjects. Reporting the previous observation as the forecast might appear lazy. 9 While it is purely conjecture, if forecasters are divided in their opinion of Fed behavior, some believing they will raise interest rates, and others that they will not, then two modes could emerge. 29


represents a processing error. If we assume that the consensus forecasts is the rational expectation f(It_1), we can decompose realizations and forecasts into their two components. Therefore v is analogous to the realization minus the consensus and w is analogous to the forecast minus the consensus. The basic structure of the experiment is maintained in that forecasts for various horizons are collected. The greater passage of time presumably creates higher variance in the release of information after the forecast is made. In principle, disagreement among forecasters could be due to private information or different methods of processing the same information. Private information in the experiments is limited to noise. The surveys are not as controlled. Figure 4.2 begins the search for the form of the relationship. We initially plotted standard errors of w and v but observed the scatters were strongly asymmetric. Using the log form allows the variable to range from positive to negative infinity therefore increasing symmetry. The standard deviations of v are calculated for each individual forecaster or subject according to the series they actually observe. Therefore the variation in the experiments is due partly to the four treatments and 30


partly due to different realizations of random draws. For the survey data set, the variation in v is due partly to differences in forecast horizon and partly to the fact that survey respondents drop in and out of the sample; sometimes the forecast is made in a time period that is highly uncertain and sometimes not. Table 4.1 presents regression results from a fixed effects model with a correction for heteroscedasticity. Given two sets of 24 subjects and four treatments, there are 192 experimental observations. We included all the professional forecasters that filled out at least 30 surveys between the first quarter of 1969 to the fourth quarter of 1993, whether or not the surveys are contiguous. Survey responses for the current and next three quarters provide 196 observations from 49 professionals. The principle variables are ln(a(w)) and ln(a(u)). A dichotomous variable, 1995, takes the value 1 if the experiments were conducted in 1995 and is used to check if the transmission of noise is affected by altering average pay. The coefficient of ln(a(w)) indicates that if noise increases 1%, processing errors increase the standard deviation about consensus by 0.9%. There is no significant difference between coefficients in the two 31


experiments. The somewhat higher rate of transmission for professional forecasters could reflect the importance of private information or a more complex forecasting environment. The double digit T-statistics indicate we have found a highly reliable relationship in three independent data sets. This is our main result. If forecasters have difficulty filtering out noise it is easy to see how noise might be transmitted to the forecast. 32


s:: 0 -1d t) 0 p,. 0 ca c:: 0 -ca ..... .._, 0 -5 c s w 0 w : c:: 0 -....... cd > 0 Q Experimental Subjects Survey of professional forecasts (based on last 50 obs. per treatment) ...... of the implicit price deflator 3 (.) 2 0 + ..... + t8 Ill + ::I 1 + .t Ill 2. """. .. c:: + fJ( 0 t' T +... 't Ill + ;:- "'.1 $ c:: +* ...... + ............ +t-0 0 + + ... t r :t.l(. +"+ u + jl" + + i'+ i: i ... ..,.. j: ++ + ++.,_ + .. !i't + + 0 1 + .... + .. + ..... ..c: ++ .............. .; ... 1:++ b i '; ... t{""-:.. ... i-tt ... ...... +t s -1. + u* + :t;t+ +Iff{ + {'t i .. ,. + 1't t t ._+ + / ... t )+ t + +-..... t ..... 0 t i-\-I 1 f t t : +t .... 4,. .. .. + t t t t tl"it. Q. :.... c:: \ .......... 0 -2 t-T t-1 i -t-t-+ ...... + ro + ;; 0 -1 Q -3 0.5 1.0 1.5 2.0 2.5 3.0 -1.5 -1.0 -0.5 0.0 ln(cr(v)) ln(cr(v)) Unknown infonnation Consensus error 4.2 -Association Between Forecast Dispersi6n and Environmental Uncertainty 0.5


Table 4.1 Relationship between consensus forecast error and deviations about consensus Dependent variable = ln(a(w)) (for last 50 observations in each treatment) Independent Experimental Data Professional Fore-Variable 192 observations casters 196 observations Coef. T-Stat. Coef. T-Stat. Constant -0.236 -1.564 0.269 6.425 ln(a(v)) 0.927 17.498 0.973 21.864 1995 ln(a(v)) -0.095 -1.246 first 0.034 0.502 second -0.030 -0.502 third -0.016 -0.225 R-2 0.844 0.894 F-Statistic 14.191 34.557 First, second and third are dummy variables indicat-ing whether the data is from the first, second or third set of 100 observations. We calculate the standard deviations of w and v based on only the last 50 observations in each treatment in-part because it eliminates order effects. Apparently learning and any other order effect occurs in the first 50 forecasts. Table 4.2 exploits the uninterrupted time series available in the experimental data to explore the propagation of noise across time. Only the 1995 experiments are used because they seem to be cleaner and better motivated (although the results for 1993 are similar). The dependent variable is the last 50 ws in each treat-34


ment and the independents are polynomial distributed lags of w(-1} and v(-1}. All regressions use 6 lags and a third order polynomial. Taking the sample as a whole, subjects apparently expect mean reversion. If there were 6 uninterrupted one unit increases in v, subjects, on average, report a forecast one or two tenths below the prior forecast. This is not a large effect but it is statistically significant. However the large and statistically significant effects of lagged values of w on w indicate that if a subject has been reporting forecasts above the prior observation it is quite likely he or she will continue to do so. 35


Table 4.2 Propagation of noise across time Distributed lag effects of w and v. Dependent variable = w (for last 50 observations in each treatment) 1200 observations per treatment. Treat. 1 Treat. 2 Treat. 3 Treat. 4 Base = $8 Base = $11 Base = $14 Base = $17 Coef T-St. coef T-St. Coef T-St. Coef T-St. v-1 0.01 0.62 -0.01 -0.59 0.06 3.89 0.01 0.96 v-2 -0.00 -0.35 -0.01 -1.60 0.00 0.02 -0.02 -1.87 v-3 -0.01 -1.56 -0.02 -2.33 -0.03 -3.52 -0.03 -3.21 v-4 -0.02 -2.36 -0.03 -3.40 -0.05 -5.85 -0.03 -3.79 v-5 -0.03 -3.11 -0.03 -3.30 -0.04 -4.85 -0.02 -2.68 v-6 -0.03 -3.32 -0.03 -3.00 -0.03 -3.07 -0.02 -2.30 v-7 -0.03 -2.14 -0.02 -1.29 -0.00 -0.33 -0.02 -1.87 Sum v -0.12 -2.46 -0.17 -3.86 -0.11 -2.62 -0.15 -3.82 w-1 0.13 4.47 0.17 4.84 0.18 5.51 0.16 4.18 w-2 0.09 6.06 o.os 4.02 0.10 5.58 0.08 4.18 w-3 0.06 4.14 0.04 2.29 0.06 3.21 0.05 2.38 w-4 0.04 2.61 0.04 2.89 0.04 2.91 0.04 2.65 w-5 0.04 2.71 0.06 3.00 0.04 2.19 0.05 2.49 w-6 0.05 3.37 0.06 3.13 0.05 2.77 0.05 2.72 w-7 0.07 2.54 0.02 0.87 0.07 2.42 0.05 1.69 Sum w 0.50 8.88 0.49 8.22 0.56 10.50 0.50 8.46 R2 0.09 0.10 0.14 0.09 F-St 19.91 16.72 27.02 16.92 It is interesting to speculate about what such a pattern might mean in a market. For example, it is Malkiel's thesis that the careers of technical analysts depend on their clients' beliefs that stocks follow patterns. As Malkiel (1990, pp. 14) puts it, "The persistence of this belief in repetitive patterns in the stock market is due to statistical illusion ... Chartists 36


believe there is momentum in the market. Supposedly, stocks that have been rising will continue to do so, and those that have been falling will go on sinking." Our experiment provides mixed support for technical analysts. There is no simple misperception of random walks that leads people to suspect momentum; indeed if anything, subjects are slightly contrarian. But, subjects do persist in reporting forecasts that deviate from the prior observation in a particular direction. It remains possible that the persistent errors on the part of individuals aggregate as momentum 10 in a market. These behavioral traits appear to be due to forecas-ting errors but could conceivably be due to non-randomness in the forecasted series. We did employ a random procedure but perhaps the particular series presented to a subject gave every appearance of being non-random. First we explore if the overall series v was indeed mean reverting. Polynomial distributed lags of 6 lagged values of v on v deliver F-statistics and proba10 There have been a few pioneering experiments on the effect of forecast formation on market dynamics. Smith, Shuchanek and Williams (1988) provide a market experiment that endogenously creates bubbles and crashes. Peterson (1993) argues the process reflects bounded rationality as subjects gradually learn to form expectations grounded on fundamentals. 37


bilities of a larger F (given no association) for the four treatments of 0.871 (.481), 0.170 (.954), 1.011 (.400) and 1.423 (.224). The sum of lags is never sig-nificant and there is no pattern to the lag distributions. The random number generator behaved well overall and delivered 11 series that appear random. Table 4.3 provides regression results based on data an individual subject has actually observed. The depen-dent variable is w and the independent variable, Vfit, is itself the fitted value of a regression. The regression runs v on the unrestricted 6 lags of v beginning with the first 50 observations in each treatment. The 51st value of v is forecast, the regression is rerun on the first 51 observations, the 52nd value is forecast and so on until the 100th forecast is constructed separately for each treatment and each subject. Table 3 shows no evidence that any significant part of w is explained by the predictable part of v. All R2 are below .001 indicating that less than 1/1000th of the variation of w is explained by the predictable component of v. Our use of the prior observation as the rational expectation intro-11 We also examined correllograms which show no pattern. 38


duces at most, a very small error and avoids debates about the proper adjustment based on the sample subjects observe. Table 4.3 Do subjective errors reflect the predict-able component of v? Dependent variable = w for the last 50 observations. 1200 observations per treatment. Treat. 1 Treat. 2 Treat. 3 Treat. 4 Base = $8 Base = $11 Base = $14 Base = $17 Intcp. -0.086 -0.141 0.048 -0.143 (-1.473) (-1.407) (0.323) (-0.824) Vfit 0.077 0.065 -0.003 0.044 (1.038) (1. 230) (-0.070) (0.809) R2 0.001 0.001 0.000 0.001 F-Stat. 1.077 1.513 0.005 0.419 T-statistics in parentheses. Table 4.4 explores whether the deviations from strict rationality cost our subjects money. The dependent vari-able is the sum lost by not reporting the prior observa-tion as the forecast. The independent variable is the standard deviation of w for the last 50 observations. In each treatment we observe a positive and statistically significant effect. Indeed, the marginal influence of higher variance on average pay is roughly constant across treatments. The value of the avoidable loss averaged $.80 per treatment or about $3.20 per subject. 39


Table 4.4 Do subjectively introduced errors cost subjects money? Dependent variable = losses above those given ratio-nal expectations. 24 observations per treatment. Treat. 1 Treat. 2 Treat. 3 Treat. 4 Base = $8 Base = $11 Base = $14 Base = $17 Intcpt. 4.805 -32.659 -48.263 -60.969 (0.326) (-1.582) (-1.193) (-1.029) a(w(SO)) 28.436 30.208 31.219 32.293 (3.645) (4.874) (3. 728) (3.085) R2 0.376 0.519 0.359 0.270 F-Stat. 13.282 23.576 8.496 9.517 I* T-statistics in parentheses. I 40


s. conclusion The debate between Simon and Muth over appropriate models of expectation formation centers over whether human cognitive limits need to be brought into the analysis. Simon's (1961} form of rationality argues that economic actors are: "intendedly rational, but only limitedly so". Muth {1961} argues than expectations are: "es:sentially the same as the predictions of the relevant economic theory." The distinction between the theories is determined by whether large and systematic deviations of forecasts from theoretically correct predictions can be traced to cognitive limitations. We approach the question of tracing the agents cognitive limitations experimentally by creating environments that reinforce rational behavior but to varying degrees; best behavior is unchanged but the smaller relative punishment for incorrect behavior creates a more difficult learning environment. The change in payment functions is achieved through increases in the variance of realizations which imply higher probabilities that theoretically incorrect forecasts will be rewarded by chance events. 41


Indeed, we find that increasing the variance of a random walk does create a more diffuse set of deviations from theoretically correct behavior. A 1% increase in the standard deviation of the random error generates about a 0.9% increase in the standard deviation of the forecast about the rational expectation. This transmission of errors between the environments occurs in two different experiments with differing payment schemes and in a survey of professional forecasters. Exploration of the time paths of deviations from rational expectations reveals positive serial correlation that is not due to systematic mis-perceptions of objective events, but occurs endogenously as an association between idiosyncratic processing errors across time. We believe the result is potentially quite important. If asset values are at least in part subjectively determined then a simple feed back loop may exist. An exogenous random shock to the asset's value may trigger a secondary wave of changes as market participants form increasingly discordant appraisals of the event's significance. Such a theory closely parallels the work of David Romer (1993) who explains changes in stock prices, despite the absence of any news, by arguing that changes 42


in stock prices convey information on the confidence of investors. Large stock price changes signal low confidence and hence generate additional large changes in stock prices. This could produce momentum in an otherwise random market. The transmission of errors between the actual and forecasting environments leads to forecasts that have systematic errors. This result questions the basis of Muthian rationality that expectations are essentially the same as the predictions of the relevant theory. If the standard rational expectations model is wrong, what is one to use? One cannot describe a theory simply based on 'cognitive limitations' without providing any additional predictive abilities. The second step of devising a theory based upon systematic errors must be developed. As Barre and Fisher (1976, p. 163) state: "A fundamental difficulty with theories of expectations that are not based on the predictions of the relevant economic model .. is that they require a theory of systematic mis takes". A theory based upon systematic mistakes characterizes how human judgment differs from the rational model. It is a theory based upon boundedly rational expectations. 43


As support to boundedly rational expectations, there is significant evidence that people do not make predictions as if they were using Bayes's rules. Forecasters tend to give too much weight to recent evidence and not enough weight to the long term trends. They overreact in their forecasts. This was also documented in the laboratory using monetary incentives (Grether, 1980) and in the field using published earnings forecasts by professional security analysts (De Bont and Thaler, 1990). De Bandt and Thaler (1985,1987) were also able to use this insight about human behavior to predict a stock market anomaly, namely mean reversion in long-term stock price movements (Medema and Samuels, 1996). In closing, our experimental results indicate that increasing the variance of a random walk does create a more diffuse set of forecasts with systematic deviations from rational behavior. This result questions the basis of Muthian rationality and lends support to theories which characterize how human judgement differs from the rational model such as theories based upon bounded rationality. 44


References Akerlof G. A. and Yellen J., 1985, A Near Rational Model of the Business Cycle, with Wage and Price Inertia, Quarterly Journal of Economics, vol. 100, pp 823-838. Barro, R. and Fisher, s., 1976, Recent Developments in Monetary Theory, Journal of Monetary Economics, 133-76. Beckman, s.R., 1992, Sources of Forecast Error: Experimental Evidence, The Journal of Economic Behavior and Organization, vol. 19, 237-244. Bulkley, and Tonks I., 1989, Are UK Stock Prices Excessively Volatile? Trading Rules and Variance Bounds Tests, The Economic Journal, vol. 99, pp. 1083-1098. Davis D. D. & Holt c. A., 1993, Experimental Economics, Princeton University Press. De Bont, W. F. M. and Thaler, R. H., 1985, Does the Stock Market Overreact?, Journal of Finance, 40, July, 793-805. De Bont, w. F. M. and Thaler, R. H., 1987, Further evidence on Investor overreaction and stock Market Seasonality, Journal of Finance, 42, July, 557-81. De Bont, w. F. M. and Thaler, R. H., 1990, Do Security Analysts overreact?, American Economic Review, 80, May, 52-7. Dwyer, G.P., Williams, A.W., Battalio, R.C., and Mason, T.I., 1993, Tests of Rational Expectations in a stark Setting, The Economic Journal, vol. 103, pp. 586-601. Grether, D. M., 1980, Bayes Rule as a Descriptive Model: The Representativeness Heuristic, Quartly Journal of Economics, 95, November, 537-57. Harrison G. W., 1992, Theory and Misbehavior of First-Price Auctions: Reply, American Economic Review, vol. 82 No. 5, Dec 1992, pp 1426-1443 Hey J. D., 1991, Experiments in Economics, Blackwell. Keane, M.P. and Runkle, o.E., 1990, Testing the Rationality of Price Forecasts: New Evidence from Panel Data, American Economic Review, vol. 80, pp. 714-35. Keynes J. M., 1936, The General Theory of Employment, Interest, and Money, HBJ Inc. 45


Malkiel B.G., 1990, A Random Walk Down Wall Street, New York: w.w. Norton & company. Medema, s. G. and Samuels W. J. eds., 1996, Foundations of Research in Economics: How Do Economists Do Economics?, Aldershot, U.K.: Edward Eigar Publishing. Muth, J. F., 1961, Rational Expectations and the Theory of Price Movements, Econometrica, val. 29, pp. 315-35. Peterson, s. P., 1993, Forecasting Dynamics and Convergence to Market Fundamentals, Journal of Economic Behavior and Organization, val. 22, 269-284. Pheby J., 1988, Methodology and Economics-A critical Introduction, Sharpe Inc. Pingle M., 1992, Costly Optimization: An Experiment, Journal of Economic Behavior and Organization, vol. 17, 3-30. Romer D., 1993, Rational Asset Price Movements Without News, .American Economic Review, vol. 83 No. 5, 1012-1130. Sargent T. J. and Wallace N., 1975, Rational Expectations and the Optimal Monetary Instrument, and the Optimal Money Supply Rule, Journal of Political Economy, April 75, 241-254. Simon, H. A., 1959, Theories of Decision-Making in Economics and Behavioral Science, American Economic Review, vol. 49, 253-283. Simon, H. A., 1961, Administrative Behavior, 2d ed., New York: Macmillian. Original Publication: 1947. Smith V. L., 1991, Papers in Experimental Economics, cambridge University Press. Smith v. L., 1994, Economics in the Laboratory, The Journal of Economic Perspectives, val. 8 No. 1, pages 113-131. Smith V. L., Suchanek G. L., Williams A. L., 1988, Bubbles, Crashes and Endogenous Expectations in Experimental Spot Asset Markets, Econometrica, val. 56, 119-1152. Sterman, J. D., 1989, Deterministic Chaos in an Experimental Economic system, Journal of Economic Behavior and Organization, vol. 12, 1-28. 46


Timmerman A., 1994, Can Agents Learn to Form Rational Expectations? Some Results on Convergence and Stability of Learning in the U.K. Stock Market, The Economic Journal, July 1994, pp 777. Wang Y., 1993, Near-Rational Behavior and Financial Market Fluctuations, The Economic Journal, November 1993, pp 1462-1478. Williams, A. w., 1987, The Formation of Price Forecasts in Experimental Markets, Journal of Money, Credit, and Banking, vol. 19, pp 1-18. 47


Appendix A -Instructions This is an experiment in the economics of forecast decisionmaking. The instructions are simple and if you follow them carefully and make good decisions you might earn a considerable amount of money that will be paid to you in cash. In any given period you must decide how much to produce before you know how much customers want. Your cash reward will be greater the closer you are able to match production to customer demand. The computer will keep a visual and numeric record of past production and demand to aid you in this task. Your cash payment for each of four experiments will be {$11} [$8, $11, $14 or $17] less one cent for every unit of difference between production and demand. That is, if you produce 700 units and 710 are demanded then 10 cents is subtracted from your reward. If you produce 700 units and 690 are demanded then again 10 cents is subtracted from your pay. Please press the c key to continue with this tutorial. Press the M key to display the Market Screen Press the B key to display the tutorial from the Beginning. (C)ontinue, (M)arket, (B)eginning? *** The four experiments are different only in that demand changes more in some experiments than others. When demand changes more it is more difficult to match production to demand and it is more difficult to retain your {$11} [ ] payment. {} [For this reason, your base pay will be $8, $11, $14 or $17 in the four different experiments. Athough you will probably do the experiments in some different order. ] {In fact, it is quite possible that for some experiments deductions may exceed $11.} [Even so, for some individuals it may be that deductions exceed pay.] If so, the deductions in excess of {$11} [base pay] are ignored. They will not be applied to the earnings in other experiments. Should this happen to you, please continue to the end of the experiment because we will pay only if all four experiments are complete. Each run of the experiment will last for 100 "days". Everyday, you will be told the previous day's demand and be asked to decide how much to produce today. The computer will keep a numerical record of demand and production for the most recent 10 days and will plot demand and 48


production for the entire 100 day period. In all the experiments, demand is preprogrammed and determined but is different for every subject. Demand does not change in response to production .. All of the book-keeping will automatically be taken care of by the computer. Your only effort is in deciding how much to produce. Please press (C)ontinue, (M)arket, (B)eginning? *** The market screen is the actual accounting information screen you will use during the simulation. Please look at this screen by pressing the "M" key now. To return to this tutorial from the market screen, press the return key. The graph in the upper left corner of the screen is the past market. The number of customers, demand, is plotted by the solid line. The number of units you produced is plotted by the squares. All the previous turns production and demand are displayed for you on the screen. The windows on the right side of the computer screen displays accounting information. The previous day's production, demand and the difference is displayed in the top half. The bottom half keeps a record of the running totals of {} [net] production, {} [net] demand {and the} [net} difference. The net difference is the number that will determine your pay. If at the end of the experiment the net difference is 456, {} (and your base pay is $11] then your pay is $11.00 -$4.56 = $6.44. Please press (C)ontinue, (M)arket, (B)eginning? *** You enter production by typing in the number you wish to produce. Should you make an error, press the key for any letter BEFORE you press the return key. Once the return key is pressed, no corrections can be made. The key items to remember from this tutorial are: 1. The closer production is to demand the higher your pay. 2. Your pay is {$11.00} [the base pay of $8, $11, $14 or $17] less one cent for every unit of difference between production and demand. 49


3. If you make an error while entering your production decision, press the key for any letter before you press the return key. This is the end of the tutorial; if you wish to view this tutorial again please press "B" for the Beginning of the tutorial. If you wish to view the Market screen again press the "M" key. If you are finished with the tutorial and wish to run the market press "C" for Continue. Please press (C)ontinue, (M)arket, (B)eginning? *** 50