SOURCES OF INACCURACIES AND IRRATIONALITIES
IN THE STATES' REVENUE FORECASTS:
PANEL DATA EVIDENCE
by
Shahriar Azadmanesh
B.A., University of Colorado at Denver, 1982
A thesis submitted to the
Faculty of the Graduate School of the
University of Colorado at Denver
in partial fulfillment
of the requirements for the degree of
Master of Arts
Economics
1994
i
This thesis for the Master of Arts
degree by
Shahriar Azadmanesh
has been approved for the
Department of
Economics
by
W. James Smith
Date
ii
Azadmanesh, Shahriar (M.A., Economics)
Sources of Inaccuracies and Irrationalities in the
States' Revenue Forecasts: Panel Data Evidence
Thesis directed by Assistant Professor Naci Mocan
ABSTRACT
Using a new panel data set, this paper employs a
random effects model to investigate the determinants of
General Fund Revenue forecast errors of state Legislative
Fiscal offices. The results show that employing
exclusively judgmental methods increases the forecast
error. The use of crosssectional data and the existence
of at least one other official forecast increases the
accuracy. The forecast error is also reduced if the
majority of the Senate, the House and the governor belong
to the same political party, and if the forecasts are
obtained less frequently than monthly or bimonthly.
Grants from the federal government have a small worsening
effect. If the predictions of the national economic
trends used in forecasting are obtained from executive
branch (legislative branch), forecasts become less (more)
accurate. The reverse is true for state economic trends.
The forecasts are free of systematic under/or over
prediction, but they can be improved by using available
information more efficiently.
iii
This abstract accurately represents the content of the
candidate's thesis. I recommend its publication.
iv
CONTENTS
CHAPTER
1. INTRODUCTION ...................................... 1
Accuracy of Revenue Forecasts ................ 2
Notes......................................... 7
2. DATA AND MEASUREMENT OF THE VARIABLES.............. 8
Data.......................................... 8
Measurement of the Variables................. 10
Notes........................................ 16
3. ESTIMATION METHODOLOGY AND RESULTS ............... 17
Panel Data: An Overview...................... 18
Fixed Effects Model.......................... 19
Random Effects Model ........................ 20
Notes....................................... 25
3. TESTS OF RATIONALITY ............................. 26
Notes ..................................... 30
4.. CONCLUSION...................................... 31
APPENDIX............................................. 38
BIBLIOGRAPHY ....................................... 48
V
TABLES
TABLE
1. DESCRIPTIVE STATISTICS .......................... 34
2. GENERAL FUND FORECAST ERROR REGRESSION RANDOM
EFFECTS MODEL ......................... 37
vi
CHAPTER I
INTRODUCTION
State governments are important elements of the
United States public finance system. In 1993, the total
General Fund Revenue collected by the 50 states of the
United States was $317.1 billion, with a median of $3.6
billion1. The magnitude of the state budgets, coupled
with the balanced budget requirements that apply to a
majority of state governments underline the importance of
obtaining accurate revenue forecasts. Revenue
forecasting has historically been an important task in
the state government budget process, which traditionally
has been part of the executive budget procedure.
However, because many states are facing fiscal crises,
legislators are concerned about their state's revenue
prospect. A recent survey (Summer 1992, see Appendix)
conducted by the author shows that many state
legislatures now consider forecasting to be important
enough to permit independent legislative capabilities or
at least significant legislative involvement.
This paper investigates the efficiency of various
means legislatures have developed to improve their
ability to forecast the general fund revenues.
1
Specifically, it deals with such issues as the types of
organizational structures legislatures have devised to
perform the forecast function and the level of
sophistication of those forecast organizations.
Additionally, the paper includes a discussion of the
range of political involvement in forecasting by
legislative and executive branches of state governments,
and types of methodologies employed. Above all this
paper deals with the issues of accuracy and objectivity
in revenue forecasts, and analysis of errors in the
application of forecasting techniques.
Accuracy of Revenue Forecasts
In order to effectively manage the state budget, the
legislature must have accurate revenue estimates. An
inaccurate forecast can seriously weaken the
legislature's ability to achieve the desired balance
between statefunded services and the level of taxes.
Revenue forecasting inaccuracies result in problems both
for those who manage state fiscal affairs and for the
public, regardless of whether the error is on the high
side or low side. For example:
Underestimating of revenues can result in 
underfunding of public services, which means that
the tax rates are being higher than they really need
2
to be. Feenberg, et al.,2 report that
...[The present and former state budget
officials] stated that unexpected surpluses are
just as bad as deficits from their point of
view. When there is an unexpected surplus,
much of the extra revenue goes to localities.
While the localities are happy to receive the
new money, they are irked that they have to re
do their planning, and resent the fact that
they were not given correct figure at the
outset.
Overestimating of revenues can result in (1) program
cutbacks and (2) tax increases, in order to avoid
budget deficits. The problems associated with
revenue overestimates can be especially serious when
the revenue shortfall is discovered after
implementation of the expenditure plan for fiscal
year has begun. Feenberg, et al.,3 also report that
...budget officials also emphasized
the fact that the newspapers point
out forecast errors very
aggressively, whether they are
negative or positive....
Shkurti and Winefordner4 analyze the revenue forecasting
process and the political consequences of forecast errors
3
in the State of Ohio. They underline the belief that the
Governor John Gilligan's handling of an unexpected budget
surplus in 1974 contributed to his defeat by Republican
James A. Rhodes. Eight years later, when Democrat
Richard F. Celeste was elected governor, feeling that
rosy revenue forecasts had contributed to the
debilitating budget crises that had plagued his
predecessor, he instructed his budget planners to be
conservative in their revenue and spending estimates for
the remainder of the Fiscal Year 1983. These arguments
indicate that it is to the best interest of the policy
makers to obtain the most accurate revenue forecast
possible.
Many factors can cause revenue forecasts to be
inaccurate, including: inaccurate forecasts of the level
of economic activity; inaccuracies in forecasting the
size of the state's tax base and the effective tax rates
that will be applied to the base; inaccurate forecasts of
the time lags between when tax liabilities are incurred
and when revenues are actually collected by the state;
unanticipated changes in state laws, court decisions, and
voter approved ballot initiatives which greatly affect
the amount of revenues collected. The level of economic
activity, however, is the primary determinant of how much
revenue the state will collect. Consequently,
4
forecasting the performance of state1s economy in the
future is the single most important task in preparing
revenue forecasts. Therefore, the immediate analysis of
forecast inaccuracies in this paper will be based on the
investigation of techniques and methodologies used to
forecast state economic activities.
Some recent studies investigated the determinants of
forecast errors in state government revenue forecasts.
Bretschneider, et al.,5 analyzed surveys of three state
governments and found that forecast accuracy increases
when there are independent forecasts from competing
agencies. Accuracy decreases when outside expert
advisors are used and when there is a dominant political
party or ideology in the state. Interestingly, they
also found that the accuracy of the forecast increases
when simple regression models and judgmental methods are
used as opposed to univariate time series methods or
econometric models.
Cassidy, Kamlet and Nagin6 examined the data from 23
states for the period 1978 to 1987. They found that the
forecast errors in economic variables, such as the
state's personal income, generated increases in the
forecast error in general fund budget, but there was no
link between political and institutional factors and the
revenue forecast accuracy. Bretschneider and Gorr7
5
analyzed the determinants of the errors in predicting the
sales tax revenues. They reported interactions between
economic uncertainty, political environment and bias in
forecasting.
In this paper, we use a new longitudinal data set
compiled through the National Conference of State
Legislatures. Using random effects models, we analyze
the determinants of the accuracy of the general fund
revenue forecast of 20 state legislative fiscal offices
over seven years. We also investigate if the forecasts
are rational (efficient); i.e. whether all the available
information is utilized by the forecasters and whether
the forecasters make systematic errors.
Chapter II describes the data used in the study and
the measurement of the variables. Chapter III presents
the results of the analysis of the forecast accuracy.
Chapter IV describes the methodology of the rationality
analysis and reports the corresponding results. Chapter
V is the conclusion.
6
Notes
1. The General Fund Revenue constitutes approximately 85% of
revenues from all sources.
2. Daniel R. Feenberg, William Gentry, David Gilroy, and
Harvey S. Rosen. 1989. "Testing the Rationality of State
Revenue Forecasts." Review of Economics and Statistics
49:301.
3.Ibid.
4.William J. Shkurti, and Darrell Winefordner. 1989. "The
Politics of State Revenue Forecasting in Ohio, 19841987:
A Case Study and Research Implications." International
Journal of Forecasting 5:364.
5.Stuart I. Bretschneider, Wilpen L. Gorr, Gloria Grizzle,
and Earle Klay. 1989. "Political and Organizational
Influences on the Accuracy of Forecasting State Government
Revenues." International Journal of Forecasting 5:307320.
6.G. Cassidy, M. S. Kamlet, and D. S. Nagin. 1989. "An
Empirical Examination of Bias in Revenue Forecasts by State
Governments." International Journal of Forecasting 5:321
331.
7.S. Bretschneider, and W. Gorr. 1992. "Economic,
Organizational, and 1 Political Influences on Biases in
Forecasting State Sales Tax Receipts." International
Journal of Forecasting 8:45766.
7
CHAPTER II
DATA AND MEASUREMENT OF THE VARIABLES
Data
Previous studies on the analysis and evaluation of
the forecasting process and forecast accuracy used data
obtained from the executive branch of the state
governments. In this study we analyze data obtained from
State Legislative Offices to investigate the accuracy and
rationality of their General Fund forecasts. The General
Fund is the largest source of appropriations in all
states, financing most staterun or statesubsidized
activities, such as education, welfare, environmental
protection and general operations.1 The fiscal year for
most states runs from July 1 through June 30. Generally,
the Legislative Office prepares its forecast for
submission to the General Assembly during December, and
the budget deliberations start in January which utilize
the forecast of the Legislative Office along with the
executive branch's forecast.
To obtain data on the details of the Legislative
Branch's forecasts, a questionnaire was mailed through
the National Conference of State Legislatures (NCSL) to
8
the Fiscal Analysis Offices of all 50 member states as
well as Washington D.C. and Puerto Rico. After followup
phone calls twenty states replied. They are Alabama,
Arizona, Colorado, Connecticut, Florida,. Illinois,
Indiana, Iowa, Louisiana, Maryland, Michigan,
Mississippi, Missouri, Nebraska, North Carolina,
Pennsylvania, Rhode Island, South Dakota, Vermont and
Wisconsin. The survey included questions regarding the
way in which the forecasts of the General Fund revenue
are obtained. More precisely, information about forecast
frequency, agents that are involved in the process other
than the legislative office, various data sources and
types, and estimation techniques that are used by the
legislative branch for the fiscal years 19851992 are
obtained. Also acquired is the magnitude of the final
forecast and the actual realization of the general fund
revenues for the same period. Since some states failed
to report their forecast values for earlier years,2 we
used the final general fund revenue forecasts obtained
from the records of the NCSL for all states. The values
reported by the states on the questionnaire and the ones
obtained from NCSL were extremely close to each other
(the zero order correlation between the two was 0.994).
The actual general fund revenues are also obtained from
NCSL records. These data are merged with the state
9
unemployment rate, per capita income, nonfarm
employment, population, the amount of federal grants
received by the state, the number of seats occupied by
the Republicans and Democrats in the Senate and House,
Governor's party affiliation and the national CPI based
annual inflation rate.
Measurement of the Variables
The dependent variable (FRCSTERR) is the absolute
percent forecast error of the general fund revenue
forecasts of state legislatures, which is calculated as
j10Ox(ACTUALFORECAST)/ACTUAL[, where ACTUAL is the
actual general fund revenue collected by the state for
that fiscal year and the FORECAST is the predicted value.
The forecast error can be a function of the quality of
the data employed in the analysis, the methodology used,
and the political environment surrounding the process.
To capture the impact of the data quality we employed
eight variables which are NFRLEGISL, NFRGOV, NFRCONSULT,
NFROTHERS, SFRLEGISL, SFRGOV, SFRCONSULT and SFROTHERS,
that demonstrate the group who was responsible for
forecasting the national (prefix N) and the state level
(prefix S) economic trends. For example, NFRLEGISL is a
dichotomous variable which takes the value one if the
legislative staff was responsible for forecasting the
10
national economic trends, and zero otherwise. NFRGOV,
NFRCONSULT and NFROTHERS are also dichotomous variables
which take the value of one if the state government,
outside consultants (including DRI, CHASE and WEFA), and
others were responsible for forecasting the national
economic trends, respectively.3 Similarly, SFRLEGISL,
SFRGOV, SFRCONSULT and SFROTHERS present the groups that
were responsible for predicting the state economic
trends. Note that these categories are not mutually
exclusive, and it is possible (in fact common) to have
the economic trend forecasts obtained by more than more
group in a given year for a given state.
An objective and rigorous analysis of the data using
statistical methods should increase forecasting accuracy.
However, it may be argued that econometric and time
series models may not produce precise predictions in the
event of structural changes, unless the predictions are
modified by those who have insights into the dynamics of
the state economy. Because of this, some states rely
heavily on the advice of "old hands" who have a good
sense of what is really going on in the state.4 There is
also evidence in the literature indicating the forecast
error reducing effect of the judgmental methods.5,6 Our
data sets enables us to investigate the relationship
between the judgmental forecasts and forecast accuracy
11
directly. The legislative fiscal officer is asked
whether qualitative (judgmental), quantitative, or
combination of qualitative and quantitative forecasting
methods were used. We employ the variable JUDGEMENTAL,
which takes the value 1 if judgmental methods were used
exclusively, and zero otherwise. To analyze the impact
of using quantitative techniques further, we included
another variable, CROSSDATA, which is one if the
forecasting process employed crosssection data and zero
otherwise. All of the respondents indicated that they
used timeseries data in all years. Thus, the use of
crosssection data in forecasting the General Fund
Revenue is an indication of the additional emphasis put
on the use on quantitative techniques.
Bretschneider and Gorr7 and Bretschneider, et al.,8
found that if both the executive and legislative branches
participate in the forecasting process, this reduces the
forecast error. To capture the impact of this
institutional structure, a dummy variable, LEGISPLUS, is
included which is equal to one if the executive branch
and/or a consensus group also produces a forecast in
addition to the Legislative Branch's forecast, and zero
if the legislative branch's forecast is the only one
supplied to the General Assembly for budget
deliberations. Some states are allowed to carry their
12
deficit to the next fiscal year, while some others are
required to finish the year with a balanced budget. The
possible impact of this difference is captured by a dummy
variable NODEFCARRY, which takes the value of one if the
state is not allowed to carry its deficit to the next
fiscal year, and zero otherwise.
Shkurti and Winefordner9 emphasize the need for the
investigation of the relationship between the frequency
of the forecast revisions and the accuracy. They mention
the case where frequent revision of the forecasts
produced a forecast error because of the
misinterpretation of few additional monthly observations
as a trend. Three dummy variables are included to
capture the impact of forecast frequency on accuracy.
ANNUAL, BIANNUAL, and QUARTERLY are variables that take
the value of one if the General Fund revenue forecast is
performed once a year, twice a year, and four times a
year, respectively. The omitted category is the group
with more frequent forecast, such as monthly and bi
monthly .
Partisan politics may also influence the forecasts.
Cassidy, Kamlet, and Nagin10 could not find evidence that
partisanship affects federal economic forecasts.
Bretschneider, et al.,11 on the other hand, found
evidence which indicates that states with a dominant
13
political party or ideology will create less accurate
forecasts. We include a dummy variable MAJORITY, which
takes the value of one if the majority of the house, the
majority of the senate and the governor belong to the
same political party, and zero otherwise. Two additional
variables that are included in the analysis of forecast
accuracy are GRANTPERPOP and URATE. GRANTPERPOP is the
ratio of federal grants coming in to. the state divided by
state population. URATE is the unemployment rate of the
state.
Table 1 presents the variables, their definitions
and the descriptive statistics. The mean percent error
(not reported in Table 1) is 0.58 with a standard
deviation of 7.58. The mean value of the absolute
percent error, which is the dependent variable, is 4.49,
with a standard deviation of 6.12. Note that 16 percent
of the sample used strictly judgmental (gualitative)
forecasts.12 The majority of the sample obtains
forecasts on a quarterly basis, followed by biannual and
annual forecasts. In 55 percent of the sample the
executive branch and/or a consensus group was responsible
to generate an official General Fund revenue forecast in
addition to the Legislative Branch (LEGISPLUS=0.55). The
mean of MAJORITY is 0.359, indicating that in 36 percent
14
of the sample the majority of the House and the Senate
belonged to the same party as the governor of the state.
Both the Legislative Office and the Executive Branch
are involved more heavily in the prediction of the state
economic trends as opposed to national trends. For
example, in 48 percent of the sample the Legislative
Office is involved in forecasting the national economic
trends (NFRLEGISL=0.48), whereas its involvement in
forecasting the state economic trends is 63 percent.
Similarly, the Executive Branch provided projections for
national trends in 15 percent of the sample
(NFRGOV=0.15), whereas in 19 percent of the sample it
provided projections for state economic trends. On the
other hand, the mean value of NFRCONSULT is 0.52, and the
mean value of SFRCONSULT is 0.38, demonstrating that 52
percent of the sample used outside consultants to predict
the national economic trends, but only 38 percent
employed outside consultants, which include DRI, CHASE
and WEFA, to predict state economic trends. Others, such
as the economic advisory groups, are being employed more
frequently to provide predictions about the state
economic trends than national trends (the mean of
SFROTHER is 0.38, whereas the mean of NFROTHER is 0.12).
15
Notes
1. G. Cassidy, M. S. Kamlet, and D. S. Nagin. 1989. "An
Empirical Examination of Bias in Revenue Forecasts by State
Governments." International Journal of Forecasting 5:321
331.
2. This was generally the case when the financial officer
who filled out the questionnaire was hired after 1985,
which was the first year covered by the survey. Thus
he/she was not knowledgeable about the details of the
process before his/her time.
3.OTHERS includes economic advisory groups.
4. Daniel R. Feenberg, William Gentry, David Gilroy, and
Harvey S. Rosen. 1989. "Testing the Rationality of State
Revenue Forecasts." Review of Economics and Statistics
42:302.
5. J. Scott Armstrong. 1983. "Relative Accuracy of
Judgmental and Extrapolative Methods in Forecasting Annual
Earning." Journal of Forecasting 2:437447.
6.Stuart I. Bretschneider, Wilpen L. Gorr, Gloria Grizzle,
and Earle Klay. 1989. "Political and Organizational
Influences on the Accuracy of Forecasting State Government
Revenues." International Journal of Forecasting 5:307320.
7.Stuart I. Bretschneider, and Wilpen L. Gorr. 1987. "State
and Local Government Revenue Forecasting." The Handbook of
Forecasting: A Manager's Guide New York: Wiley.
8.Stuart I. Bretschneider, Wilpen L. Gorr, Gloria Grizzle,
and Earle Klay. Ibid.
9. William J. Shkurti, and Darrell Winefordner. 1989. "The
Politics of State Revenue Forecasting in Ohio, 19841987:
A Case Study and Research Implications." International
Journal of Forecasting 5:361373.
10. G. Cassidy, M. S. Kamlet, and D. S. Nagin. Ibid.
11.Stuart I. Bretschneider, Wilpen L. Gorr, Gloria Grizzle,
and Earle Klay. Ibid.
12.Note that "sample" refers to the 140 observations which
are obtained from 20 states over seven years. Thus, 16% of
the sample, for example, does not imply 16% of the states
16
CHAPTER III
ESTIMATION METHODOLOGY AND RESULTS
The empirical analysis is carried out using the
following model:
Y,t=a+Z/JjXJ1.,+u1.+6tt, (1)
where Yjt represents the absolute percent forecast error
for state i in year t, Xjit is the value of the jth
explanatory variable for state i in year t, and a and jS^
are the coefficients. eft is an iid error term with usual
properties and ui captures statespecific unobservables.
u} is also assumed iid with mean zero and variance cru2.
Recent studies which employed panel data to
investigate the determinants of the biases in state
governments' revenue forecasts used ordinary least
squares (OLS) to estimate the models.1,2 OLS is not an
appropriate method in the presence of acrossstate
variations. More precisely, pooling all the observations
and estimating the model using OLS would yield biased and
inconsistent estimates if there are differing structures
across states represented by ui in equation (1) above.
17
Panel Data: An Overview
When observations are available for several
individual units (households, firms, states, etc.) over a
period of times, some method of combining the data should
be used. In our case a single year crosssection model
might include such crosssection explanatory variables as
attributes of the state's sources of data, forecast
software package, and political party affiliation. If
data were available for each state over a series of
years, we might question whether crosssection parameters
remain constant over time. If they do, we can consider
the possibility of combining the data to obtain more
efficient parameter estimates. The process of combining
crosssection and timeseries data is called panel or
pooling.
Consider the following model:
Yit=ai+pzit+eit (2)
Where i = 1,... . .N
t = 1,...
N is the number of crosssection units (states) and T is
the number of time periods. Xit is a vector of
explanatory variables. If both a and /3 are constant over
time and over crosssection units, more efficient
parameter estimates can be obtained if all the data are
18
combined so that one large pooled regression is run with
NT observations. a. allows for the flexibility of
changing intercepts across states.
Fixed Effects Model
The effects of unobserved heterogeneity (a,.) can be
controlled for by including (Nl) dummy variables for i,
if this is not practical (if N is large) we can transform
the original variables such that
Xlt=XitXd.
where
is the mean of Y for individual i averaged across T, and
is the mean of X for individual i averaged across T.
Then, regress Y*jt on X*it. The dummy variable
coefficients (ai' s) can be recovered by
However, this algorithm does not work if the variables
are constant over time. For example, consider the
following two individuals where the values of the
variable X observed over three time periods.
19
If X = 10
In this case all X*t = 0, because there is no variation
in Xi over time. Thus, fixed effect models will not work
if there is no variation of X over time.
Random Effects Model
It may be more appropriate to view the individual
specific constants (ai) as randomly distributed across
crosssectional units. Consider the model:
E(eit)=E[Mi)=0, E[e2it]=S2e, E[m2,]=52u for all i, t and j.
Yit=a+p*it+lAi+eit
The variance of eit is equal to
The numerator is the sum of squared residuals from the
fixed effects model and K is the number of regressors.
Consider the regression:
=EiEt (eicei.)2/ (NTNIQ
(4)
(5)
The estimated errors are
20
and their variance is
o*=Â£ (et)2/(NK)
which yields equation (6).
(6)
(4) and (6) are used to create
=i [de (rdu+dg) V2]
and use 0 to transform the original variables, such that:
and
X^X^QX,.
regress Y*jt on X*jt.
As explained above, the heterogeneity due to state
specific unobservables can be controlled for by using
standard longitudinal methods. An easy way is to use a
fixed effects model, which can be implemented by
including state dummies in the regression equation. If
this is impractical, (due to a large number of states but
relatively small number of time periods), one can apply
the transformation in which the timemeans of the
variables are subtracted out from the original ones for
every state. In our case, this procedure is not feasible
because many variables do not exhibit time variation,
although they differ across states. Therefore, we employ
21
a random effects model to investigate the determinants of
forecast accuracy. The random effects model views the
individual specific intercepts (a+u^ as randomly
distributed across the states.
The results are reported in Table 2. The dependent
variable is the absolute percent forecast error. Thus, a
negative coefficient indicates an increase in forecast
accuracy. The coefficients of ANNUAL, BIANNUAL and
QUARTERLY are negative and significant, demonstrating
that if forecasts are obtained less frequently (once,
twice or four times a year, instead of monthly or bi
monthly) this helps to decrease the forecast error.
LEGISPLUS has a coefficient of 1.54, which is
significant at the 1% level, which indicates that if
there exists at least one other official forecast, either
from the executive branch or from a consensus group, the
absolute value of the forecast error would go down by 1.5
percent. If the majority of the House, the majority of
the Senate and the governor belong to the same party
(MAJ0RITY=1), this lowers the forecast error by 2.56
percent. The ability of the state to carry the deficit
over to the next fiscal year has no statistically
significant impact on the forecast accuracy.
Another interesting result pertains to the use of
national and state economic trend forecasts. NFRCONSULT,
22
SFRCONSULT, NFROTHERS and SFROTHERS are not statistically
different from zero. The insignificance of these
variables may be due to possible collinearity among them.
Further tests, however, revealed that they were not
jointly significant either.3 This means that the use of
projections of the national and state economic trends
that are supplied by outside consultants or economic
advisory groups has no effect on the accuracy of the
General Fund revenue forecasts. However, the coefficient
of NFRGOV is positive and significant, and the
coefficient of NFRLEGISL is negative and significant.
This implies that if the predictions of the national
economic trends that are used in General Fund revenue
forecasting are obtained from the state government, the
General Fund revenue forecast becomes less accurate, but
if they are supplied by the Legislative Office, the
forecast error will be smaller. The reverse is true for
the source of the prediction of state economic trends.
SFRGOV is negative, SFRLEGISL is positive, and both are
significant. These results indicate that the state
(national) economic forecasts to be used in the
predicting the General Fund revenue should be obtained
from the executive branch (legislative branch) in order
to reduce the prediction error of state General Fund
revenue forecasts.
23
Table 2 also reveals that the estimated coefficient
of JUDGMENTAL is 3.53 with a tratio of 2.85, indicating
that using exclusively judgmental methods increases the
forecast error by 3.5 percentage points. On the other
hand, the use of cross section data reduces the absolute
forecast error by almost seven percentage points. All
states in all years indicated their use of timeseries
data, while only 16 percent of the sample declared the
use of crosssection data in addition to timeseries.
While timeseries data can be used for relatively simple
procedures such as extrapolation and trend forecasting,
crosssectional data necessitates the use of econometric
methods. Thus, the use of crosssection data is an
indication of additional emphasis put on quantitative
methods. These results underscore the importance of
using quantitative methods in increasing the forecast
accuracy. Lastly, changes in unemployment rate have no
impact on prediction error, whereas an increase in the
federal grants the state receives per capita has a small
aggravating impact.
24
Notes
l.G. Cassidy, M. S. Kamlet, and D. S. Nagin. 1989. "An
Empirical Examination of Bias in Revenue Forecasts by State
Governments." International Journal of Forecasting 5:321
331.
2.Stuart I. Bretschneider, Wilpen L. Gorr, Gloria Grizzle,
and Earle Kay. 1989. "Political and Organizational
Influences on the Accuarcy of Forecasting State Government
Revenues." International Journal of Forecasting 5:307320.
3.We tried various combinations, as well as using all four
as a group. In no combination we found significance.
25
CHAPTER IV
TESTS OF RATIONALITY
The previous section demonstrated the ways in which
the errors in state General Fund forecasts can be
reduced. In this section we analyze the rationality of
the forecasts. Let Rt stand for the actual revenue at
time t and tR*t_,, be the forecast of Rt formed at time t1,
based on the information set It.1 that is available to the
forecaster at time t1. A test for weak rationality can
be performed by estimating the regression
Rt=a+/3 (1)
and testing the null hypothesis H0: (a,/?) = (0,l) where et
is the white noise error term with usual properties. If
H0 can not be rejected, this implies that E[Rt tR*t_
1]=E[et]=0. In other words, the failure to reject the
null hypothesis of weak rationality implies that the
expected value of the forecast error is zero; i.e., the
forecaster does not make systematic errors, and predicts
the correct revenue on the average. Thus the test for
weak rationality is also a test for unbiaseness.
If the forecaster utilizes all the available
information contained in the information set It.1
efficiently to obtain J.R*,..,,, the forecast error Rt tR* ^
26
should not depend on It_,,. Using this notion, strong
rationality implies running the regression
the variables in the information set It.1. Failure to
reject the null hypothesis gives support to strong
rationality and implies that all information is being
utilized efficiently to create forecasts.1,2,3
Fund revenue as the dependent variable, and the forecast
of the General Fund revenue (FORECAST) as the independent
variable. The estimated constant was 74.017 and
statistically not different from zero. The coefficient
of FORECAST was 0.987 and highly significant. The joint
test of the constant being zero and the coefficient of
FORECAST being one could not be rejected (the pvalue of
the joint hypothesis is 0.31). Thus, we could not reject
the hypothesis that the Legislative forecasts of the
State General Fund Revenues were weakly rational. This
implies the absence of systematic errors and indicates
that the forecasts hit the actual values on the average.
The execution of the strong rationality test
requires the knowledge about the variables in the
information set 11. First, we used the same variables
(2)
and testing the null hypothesis
H0: ( / 5 1^2, * *  0
0), where X
We estimated equation (1) using the actual General
27
that were used in the analysis of the percent absolute
forecast error (explanatory variables of Table 2) .
Following Feenberg, et al.,4 and Gentry5, we entered the
variables with one lag based on the presumption that the
forecasters have reliable information on the variables at
the time of the forecasts. After estimating equation (2)
the joint hypothesis of H0: (<$0,
strongly rejected with a pvalue 0.00.6 Next, we
included the following additional variables that were
also employed by Feenberg, et al.,7 and Gentry8: nominal
state per capita personal income, state nonagricultural
employment, the national inflation rate, state population
and previous year's revenue. Reliable personal income
data are generally not available until after the
completion of the budget process. Therefore, income was
entered with two lags.9 Again, we easily rejected the
null hypothesis of strong rationality with a pvalue of
0.00. This implies that there is information in the
explanatory variables which can be extracted to alter the
forecast error.
Feenberg, et al.,10 employed the framework described
above to test the rationality of revenue and grant
forecasts for New Jersey, Massachusetts and Maryland
using time series data from the late 1940s to 1987. They
provided strong evidence rejecting the rationality of
28
both short and longterm forecasts. Gentry11
investigates the rationality of the forecasts for
individual taxes for the state of New Jersey using time
series data. With few exceptions, he also rejects the
rationality of the forecasts and reports a downward bias
in forecasts. Instead of using the levels, these studies
employed the percent changes of the variables to test the
rationality hypotheses. We also performed the test using
percent changes. In this specification the dependent
variable is the difference between actual percent change
in General Fund revenue and the predicted percent change
from one year to the next; i.e. [ ] [ (tR*t_.,Rt_
.jJ/R^,,]. All explanatory variables are entered in percent
change form, except for dichotomous variables. The
results did not support the strong rationality
assumption. The hypothesis that the coefficients are
jointly zero is rejected at the 1 percent level.
29
Notes
1. Byron W. Brown, and Shlomo Maital. 1981. "What do
Economists Know? An Empirical Study of Experts'
Expectations." Econometrica 49(March):491504.
2. Daniel R. Feenberg, William Gentry, David Gilroy, and
Harvey S. Rosen. 1989. "Testing the Rationality of State
Revenue Forecasts." Review of Economics and Statistics
42:301302, 429440.
3. Donald J. Mullineaux. 1978. "On Testing for Rationality:
Another Look at the Livingston Price Expectations Data."
Journal of Political Economy 86:329336.
4. Daniel R. Feenberg, William Gentry, David Gilroy, and
Harvey S. Rosen. Ibid.
5. Gentry, William M. 1989. "Do State Revenue Forecasters
Utilize Avalable Information." National Tax Journal
6. Exclusion of the intercept did not have any impact on the
results.
7. Daniel R. Feenberg, William Gentry, David Gilroy, and
Harvey S. Rosen. Ibid.
8. William M. Gentry. Ibid.
9. We also used income with one lag. The results did not
change in any meaningful way. To be comparable to earlier
studies, we report the results with twolag specification.
10. Daniel R. Feenberg, William Gentry, David Gilroy, and
Harvey S. Rosen. Ibid. 11
11. William M. Gentry. Ibid.
30
CHAPTER V
CONCLUSION
Using a new panel data set that covers the years
19861992, this paper investigates the determinants of
forecast errors in state General Fund Revenue forecasts
of 20 state Legislative Fiscal offices. To control for
unobservable state characteristics, we estimate a random
effects model. Clearcut results emerge. If the states
exclusively employ qualitative (judgmental) methods for
forecasting, this increases the absolute forecast error
by 3.5 percentage points. The use of crosssectional
data which indicates emphasis on quantitative methods
reduces the forecast error by 6.8 percentage points.
These results underscore the importance of employing
quantitative techniques in forecasting the General Fund
revenue. The existence of at least one other official
forecast, either from the executive branch or from a
consensus group increases the accuracy of the Legislative
Office's forecasts. The forecast error is also reduced
by almost 3 percentage points if the majority of the
Senate, the House and the governor belong to the same
political party. If the forecasts are obtained once,
twice, or four times a year, as opposed to monthly or bi
31
monthly, forecast accuracy increase. Per capita grants
from federal government have a small worsening effect on
forecast accuracy.
The accuracy of the forecasts also depends on the
source which provides projections of the state and
national economic trends. If the predictions of the
national economic trends that are used in General Fund
revenue forecasting are obtained from state government,
the forecasts become less accurate, but if they are
supplied by the legislative branch, the forecast error
will be smaller. The reverse is true for the data on
state economic trends. If the data on state economic
trends are obtained from the executive branch, this
reduces the forecast error; if they are obtained from the
legislative branch, the forecast errors increases.
We also investigated the rationality of the General
Fund Revenue forecasts. The forecasts are called
strongly rational if all available information is used
efficiently to obtain the forecasts, and if the forecast
error can not be influenced by the information that was
available at the time of the forecast. The forecasts are
weakly rational if a subset of the information set is
used efficiently, which implies that the projected value
is equal to the actual value on the average. We found
that the forecasts are weakly rational; i.e they are free
32
of systematic under or overprediction. However,
are not strongly rational, which implies that the
forecasts can be improved by using the available
information more efficiently.
they
33
TABLE 1
DESCRIPTIVE STATISTICS
. Variable Obs Definition Mean Std. Dev.
FRCSTERR 138 Absolute percent forecast error 4.49 6.117
NFRLEGISL 121 Dichotomous variable (=1) if the legislative staff is responsible for forecasting national economic trends,(=0) otherwise. 0.48 0.502
NEKGOV 121 Dichotomous variable (=1) if the state government is responsible for forecasting national economic trends,(=0) otherwise. 0.15 0.357
NFRCONSUIT 121 Dichotomous variable (=1) if outside consultants (including DRI, CHASE and WEFA)are responsible for forecasting national economic trends,(=0) otherwise. 0.521 0.502
NFROTHER 121 Dichotomous variable (=1) if other groups are responsible for forecasting national economic trends, (=0) otherwise. 0.124 0.331
SERIEGISL 128 Dichotomous variable (=1) if the legislative staff is responsible for forecasting state economic trends, (=0) otherwise. 0.633 0.484
SFRGOV 128 Dichotomous variable (=1) if the state government is responsible for forecasting state economic trends, (=0) otherwise. 0.195 0.398
34
Variable Obs Definition Mean Std. Dev.
SEROONSUIT 128 Dichotomous variable (=1) if outside consultants (including DRI, CHASE and WEFA) are responsible for forecasting state economic trends, (=0) otherwise. 0.383 0.488
SFROTHER 128 Dichotomous variable (=1) if other groups are responsible for forecasting state economic trends, (=0) otherwise. 0.281 0.451
CROSSDATA 127 Dichotomous variable (=1) if cross sectional data are used in General Fund revenue forecasting in addition to timeseries data, (=0) otherwise. 0.126 0.333
URATE 140 State unemployment rate 5.901 1.920
JUDGEMENTAL 127 Dichotomous variable (=1) if only judgmental (qualitative) methods are used in General Fund revenue forecasting, (=0) otherwise. 0.165 0.373
QUARTERLY 140 Dichotomous variable (=1) if the General Fund revenue is obtained quarterly, (=0) otherwise. 0.464 0.501
ANNUAL 140 Dichotomous variable (=1) if the General Fund revenue is obtained annually, (=0) otherwise. 0.150 0.358
BIANNUAL 140 Dichotomous variable (=1) if the General Fund revenue is obtained biannually, (=0) otherwise. 0.329 0.471
35
Variable Obs : Definition j/lieiah Std. Dev.
IEGISPLUS 140 Dichotomous variable (=1) if in addition to the Legislative Office, the Executive Branch and/or a consensus group is also responsible for obtaining General Fund revenue forecast, (=0) otherwise. 0.550 0.499
MAJORITY 131 Dichotomous variable (=1) if the Majority of the House, the majority of the Senate and the Governor belong to the same political party, (=0) otherwise. 0.359 0.482
NODEFCARRY 140 Dichotomous variable (=1) if the state can not carry the deficit over to the next fiscal year, (=0) otherwise. 0.650 0.479
GRANTPERPOP 140 Per capita grants the state receives from the Federal Government. 466.904 136.140
36
TABLE 2
GENERAL FUND FORECAST ERROR REGRESSION
RANDOM EFFECTS MODEL
Explanatory Variable Coefficient tstatistic pvalue
CONSTANT 1.136 0.172 0.863
ANNUAL 12.296 7.193 0.000
BIANNUAL 4.898 5.117 0.000
QUARTERLY 7.633 6.404 0.000
LEGISPHJS 1.543 2.606 0.011
MAJORITY 2.559 2.573 0.012
NODEFCARRY 0.150 0.147 0.884
NFRSTGOV 8.674 3.923 0.000
NFRLEGISL 9.731 2.423 0.017
NFRCONSULT 4.183 1.532 0.129
NFROTHERS 2.939 1.355 0.179
SFRSTGOV 6.227 4.963 0.000
SFRLEGISL 12.340 3.924 0.000
SFRCONSUIT 3.715 1.203 0.232
SFROTHERS 0.675 0.334 0.739
JUDGMENTAL 3.530 2.849 0.005
CROSSDATA 6.813 2.348 0.021
GRANTPERPOP 0.020 2.662 0.009
URATE 0.140 0.365. 0.716
F 194.684
Adjusted R2 0.970
N 107
37
APPENDIX
The Survey
1. Is there an "official" General Fund revenue forecast
in your state preformed by legislative staff (i.e a
revenue forecast that is required by law or
statute)? Please circle one.
a. YES b. NO
2. Which branches of government were responsible for
generating an official revenue forecast for the
general fund? (Please mark all the appropriate
cells).
FY 1986 FY 1987 FY 1988 FY 1989 FY 1990 FY 1991 FY 1992
executive branch
legislative branch
consensus group
others (please identify)
3a. Does your office prepare a revenue forecast whether
it is "official" or not? Please circle one.
a. YES b. NO
38
3b. If no, does your office perform a review and comment
function of the executive revenue forecast? Please
describe this function.
IF YOUR ANSWER IS NO TO 3a, DISREGARD THE REST OF THIS
QUESTIONNAIRE AND RETURN IT TO US. THANK YOU FOR YOUR
COOPERATION.
3c. If your answer is yes to 3a, please provide us with
your original (the first revenue forecast, normally
done before general assembly goes into the session)
and final (revenue forecast after the general
assembly appropriates the budget for the next fiscal
year) revenue forecast of the general fund
(legislative forecast exclusively) for the past
_____seven fiscal years. _________
General Fund Legislative Revenue Forecast (original) Years General Fund Legislative Revenue Forecast (final)
FY 1986
FY 1987
FY 1988
FY 1989
FY 1990
FY 1991
FY 1992
39
4. How frequently the general fund forecast were made
_____by the legislative staff?
FY 1986 FY 1987 FY 1988 FY 1989 FY 1990 FY 1991 FY 1992
monthly
quarterly
semi annually
annually
biennially
5a. Which sources supplied the national data that were
used to create the general fund revenue forecast?
(Piease mark all those that apply).
FY 1986 FY 1987 FY 1988 FY 1989 FY 1990 FY 1991 FY 1992
federal government
legislative staff
state government
outside consultants; e. g., DRI
CHASE
WEFA
economic advisory group
others (please identify)
40
5b. Which sources supplied the state data that were
used to create the general fund revenue forecast?
(Please mark all those that apply).
FY 1986 FY 1987 FY 1988 FY 1989 FY 1990 FY 1991 FY 1992
federal government
legislative staff
state government
outside consultants; e.g., DRI
CHASE
WEFA
economic advisory group
others (please identify)
6. Do you employ national and state indicators to
forecast the revenue for the general fund? Please
circle one.
a. YES b. NO
If yes, then:
41
7a. Who was responsible for forecasting national
_____economic trends in years indicated below?
FY 1986 FY 1987 FY 1988 FY 1989 FY 1990 FY 1991 FY 1992
legislative staff
state government
outside consultants
economic advisory group
others (please specify)
7b. Who was responsible for forecasting state economic
trends in years indicated below?
FY 1986 FY 1987 FY 1988 FY 1989 FY 1990 FY 1991 FY 1992
legislative staff
state government
outside consultants
economic advisory group
others (please specify)
42
8.
What kind of data set have you been using for your
general fund forecast?______________________
FY 1986 FY 1987 FY 1988 FY 1989 FY 1990 FY 1991 FY 1992
cross sectional (set of data for one year only)
time series (set of data going back in time)
other (please identify)
9. What method(s) have you used to forecast the general
fund revenue?
FY 1986 FY 1987 FY 1988 FY 1989 FY 1990 FY 1991 FY 1992
qualitative methods (i.e. judgmental forecast)
quantitative techniques
combination of qualitative and quantitative forecasts
10. IF YOU HAVE USED QUANTITATIVE TECHNIQUES DURING LAST
SEVEN FISCAL YEARS, PLEASE ANSWER QUESTIONS 1013.
43
10a. Please put an "x" in appropriate cells if you have
used a timeseries technique like ARIMA (BOX
JENKINS), EXPONENTIAL SMOOTHING or the like to
forecast the general fund. Note that these
techniques use only the past values of the general
fund revenue. They do not involve other
variables.
FY 1986 FY 1987 FY 1988 FY 1989 FY 1990 FY 1991 FY 1992
ARIMA
exponential smoothing
OTHER TECHNIQUES that use only past values of the general fund (please be specific)
10b. If you have employed one of the methods in question
10a. to forecast the general fund revenue, how many
of the past general fund revenue data were
_____available to analyze in the following years?________
FY 1986 FY 1987 FY 1988 FY 1989 FY 1990 FY 1991 FY 1992
44
11a. If you have used a single equation regression model
(where the general fund revenue depends on a number
of other variables), how many variables were
included to forecast the general fund revenue in
_____following years?_________________________
FY 1986 FY 1987 FY 1988 FY 1989 FY 1990 FY 1991 FY 1992
lib. If you have used a single equation regression model
as in question 11a., how many years of data did you
use to estimate the equation in the following years?
FY 1986 FY 1987 FY 1988 FY 1989 FY 1990 FY 1991 FY 1992
11c. If you have used a single equation regression model
to forecast the general fund revenue, you probably
needed to forecast the values of the variables that
explain the general fund revenue (the independent or
explanatory variables). How have you obtained the
_____projections of these variables?______________________
FY 1986 FY 1987 FY 1988 FY 1989 FY 1990 FY 1991 FY 1992
using timeseries extrapolation
using expert opinion
other (please specify)
45
12a. If you have used a multiequation regression model,
(where the general fund depends on its determinants,
and some of the determinants of the general fund
revenue are in turn explained by other variables),
how many equations did your model include in
_____following years?____________________________________
FY 86 FY 87 FY 88 FY 89 FY 90 FY 91 FY 92
27
815
1625
26 and over
12b. How many variables did you have in each year?
FY 1986 FY 1987 FY 1988 FY 1989 FY 1990 FY 1991 FY 1992
27
815
1625
26 and over
. 46
13. Please indicate the forecasting software that you
_____have used to forecast your general fund revenue.
FY 1986 FY 1987 FY 1988 FY 1989 FY 1990 FY 1991 FY 1992
AREMOS
RATS
TSP
SAS
SPSS
INHOUSE
OTHER (please be specific)
NONE
14. Please indicate the actual (realized) general fund
______revenue for each of the years below._______
Actual (realized) General Fund Revenue
FY 1986
FY 1987
FY 1988
FY 1989
FY 1990
FY 1991
15. Please write in any other comment you may have
regarding budget revenue forecasting or this survey.
Thank you very much for taking the time to complete
this survey.
47
BIBLIOGRAPHY
Books
Hsiao, Cheng. 1992. Analysis of Panel Data. New York:
Cambridge University Press.
Eckl, Corina, et al. 1993. State Budget Action. 1985
1993. Denver: National Conference of State Legislatures
Press.
Periodicals
Armstrong, J. Scott. 1983. "Relative Accuracy of
Judgmental and Extrapolative Methods in Forecasting
Annual Earnings." Journal of Forecasting 2:437447.
Bretschneider, S., and W. Gorr. 1992. "Economic,
Organizational, and Political Influences on Biases
in Forecasting State Sales Tax Receipts."
International Journal of Forecasting 8:457466.
Bretschneider, Stuart I., Wilpen L. Gorr, Gloria Grizzle,
and Earle Klay. 1989. "Political and Organizational
Influences on the Accuracy of Forecasting State
Government Revenues." International Journal of
Forecasting 5:307320.
Bretschneider, Stuart I., and Wilpen L. Gorr. 1987.
"State and Local Government Revenue Forecasting." In
The Handbook of Forecasting: A Manager's Guide,
edited by S. Makriadis and S. C. Wheelwright. New
York: Wiley.
Brown, Byron W., and Shlomo Maital. 1981. "What Do
Economists Know? An Empirical Study of Experts'
Expectations," Econometrica 49(March):491504.
48
Cassidy, G., M.S. Kamlet, and D. S. Nagin. 1989. "An
Empirical Examination of Bias in Revenue Forecasts
by State Governments." International Journal of
Forecasting 5:321331.
Feenberg, Daniel R., William Gentry, David Gilroy, and
Harvey S. Rosen. 1989. "Testing the Rationality of
State Revenue Forecasts." Review of Economics and
Statistics 42:301302, 429440.
Gentry, William M. 1989. "Do State Revenue Forecasters
Utilize Available Information." National Tax Journal
4:163198.
Mullineaux, Donald, J. 1978. "On Testing for Rationality:
Another Look at the Livingston Price Expectations
Data." Journal of Political Economy 86:329336.
Shkurti, William J., and Darrell Winefordner. 1989. "The
Politics of State Revenue Forecasting in Ohio, 1984
1987: A Case Study and Research Implications."
International Journal of Forecasting 5:361373.
49
