Citation
Regulatory policy analysis and discourse

Material Information

Title:
Regulatory policy analysis and discourse assessing the need for knowledge organization, management, and transer
Creator:
Altman, Tracy Allison
Publication Date:
Language:
English
Physical Description:
228 leaves : ; 28 cm

Subjects

Subjects / Keywords:
Policy sciences ( lcsh )
Public administration -- Decision making -- United States ( lcsh )
Environmental policy -- Decision making -- United States ( lcsh )
Air -- Pollution -- Law and legislation -- United States ( lcsh )
Air -- Pollution -- Law and legislation ( fast )
Environmental policy -- Decision making ( fast )
Policy sciences ( fast )
Public administration -- Decision making ( fast )
United States ( fast )
Genre:
bibliography ( marcgt )
theses ( marcgt )
non-fiction ( marcgt )

Notes

Bibliography:
Includes bibliographical references (leaves 210-228).
General Note:
School of Public Affairs
Statement of Responsibility:
by Tracy Allison Altman.

Record Information

Source Institution:
|University of Colorado Denver
Holding Location:
Auraria Library
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
40326467 ( OCLC )
ocm40326467
Classification:
LD1190.P86 1998d .A58 ( lcc )

Full Text
Regulatory Policy Analysis and Discourse:
Assessing the Need for Knowledge
Organization. Management, and Transfer
by
Tracy Allison Altman
B.S.. Oklahoma State University. 1980
M.B.S.. University of Colorado at Denver, 1991
A thesis submitted to the
University of Colorado at Denver
in partial fulfillment of the requirements for the
Doctor of Philosophy
Public Administration
1998


SI998 Tracy Allison Altman
All rights reserved.


This thesis for the Doctor of Philosophy
degree by
Tracy Allison Altman
has been approved
bv
TtyuA 3 mg
Date
Robert Amott


Altman. Tracy Allison (Ph.D.. Public Administration)
Regulatory Policy Analysis and Discourse: Assessing the Need for Knowledge Organization.
Management, and Transfer
Thesis directed by Professor Peter deLeon
Abstract
This dissertation examines the role of policy analysis in the debate over federal
regulatory policy: How do analysts participate in discourse and knowledge transfer? In what
forums do they introduce, and how do they represent, their arguments? Concern over the
costs and efficacy of U.S. environmental laws has triggered numerous reform efforts,
offering substantial opportunity for analysts to investigate policy questions. However, the
appropriate scope and the likely influence of policy analysis are continually reexamined in
the literature.
Among the theoretical underpinnings of public administration are fundamental
questions about purpose: investigators have not agreed on analytical benchmarks nor
resolved questions about the profession. Recognition of the role of argumentation and
ideological frameworks only blurs any previously drawn distinctions. Current processes do
not encourage knowledge transfer between researchers and practitioners.
To assess trends in environmental policy analysis over the past 25 years, Clean Air Act-
related discourse was examined. Characteristics of 400 documents pertaining to EPA
permitting programs were coded in a database; contributions to both the peer-reviewed and
the "gray" literature were included in this content analysis. Bibliometric analysis was
performed to determine the extent of co-citation among the various contributions.
Five hypotheses were tested: descriptive statistics were developed using database
IV


queries. Results show that participants contribute in many forums, invoking a variety of
threats, but citing specific theories or empirical data less frequently. No consistent analytical
methodology emerged. However, the nature of the contributions has been relatively constant:
reference to economic criteria has been consistent over time. Likewise, bibliometric analy sis
reveals that co-citation among contributors to this discourse has also remained steady .
Results underscore the lack of sophisticated policy analysis in the formal discourse
they also reveal the need for better synthesis of the substance of policy analysis and the
management of program implementation. A more systematic approach to managing
environmental programs would call for better knowledge organization and coordination
between decision-makers, practitioners, and researchers: between public, private, and non-
profit entities: and among analysts working in different disciplines or organizations.
This abstract accurately represents the content of the candidate's thesis. I recommend
its publication.
Peter deLeon
v


Dedication
This dissertation is dedicated to my husband, Jeff.
Without him. none of this would have been possible,
nor would it matter.


Acknowledgments
Many thanks to all the members of my committee
for their leadership, insight and support especially to
Dr. Peter deLeon. for years of valuable guidance.
Thanks also to my parents, who always encouraged me.
and to Ray Bohn, for his valuable technical assistance.


Contents
Chapter
1. Introduction ..................................................
1.1 Unanswered Questions........................................
1.2 Environmental Policy: Ripe for Review ......................
1.3 Focus of This Study.........................................
2. Development of U.S. Environmental Regulatory Policy............
2.1 Origins of the EPA Framework................................
2.2 Air Pollution Control ......................................
2.2.1 CAA Round Two...............................................
2.2.2 EPA's Permitting Programs...................................
2.3 Quantitative Decision-Making Methods .......................
2.3.1 Cost-Benefit Analysis ......................................
2.3.2 The Risk Assessment Business ...............................
2.4 Designing Pollution Policy .................................
2.4.1 Variables in the Environmental Equation ....................
2.4.2 Regulatory Reinvention......................................
2.5 Where Does EPA Lie on the Information Technology Curve? .
3. Policy Literature: The State of the Art.......................
3.1 Policy Tools and Analysis of Alternatives...................
3.1.1 Traditional Techniques Give Way.............................
3.1.2 Argumentation and Languages of Discourse....................
3.1.3 The Search for Identity ....................................
3.2 Supply and Demand: Is There a Knowledge Utilization Deficit?
3.2.1 Schools of Thought: An Interdisciplinary Challenge .........
3.2.2 Complexity..................................................
3.2.3 Bridging Between Disciplines................................
3.3 Summary of the Policy Literature............................
4. Methodology...................................................
4.1 Scope of the Study..........................................
4.2 The Approach................................................
4.2.1 Statement of Hypotheses ....................................
4.2.2 Data Collection and Document Review.........................
4.2.3 Database Design.............................................
4.2.4 Validity and Reliability ...................................
viii
. 1
. I
. j
. 5
. 8
. 8
. 9
12
15
20
20
26
29
32
36
41
43
43
44
48
51
53
58
63
64
68
70
70
71
74
75
76
77


Contents (Coot.)
5. Findings ..................................................................80
5.1 Key Results ...............................................................80
5.1.1 Bibliometrics ............................................................81
5.1.2 Content Analysis .........................................................84
5.1.3 Hypothesis Testing........................................................99
5.1.4 Standards for Analysis...................................................103
5.1.5 Informational and Organizational Issues..................................104
5.2 Assessment................................................................105
5.2.1 Lessons Learned..........................................................107
5.2.2 Validity, Limitations, and Constraints...................................114
6. Conclusions ...............................................................116
6.1 Opportunities for Analysts................................................116
6.1.1 Teach Analytical Thinking................................................118
6.1.2 Be a Bridge..............................................................118
6.1.3 Manage the Knowledge Base................................................120
6.2 Avenues for Future Research...............................................123
Appendix
A. Key Decision Points for Air Quality Permitting Programs..................126
B. Documents Included in Study...............................................128
C. Data Base Design .........................................................197
Glossary .......................................................................208
References..........................................................................210
IX


KJ\ U\
Figures
Figure
2.1 The Clean Air Decision-Making Pipeline ...................................11
2.2 Trends in U.S. Emissions of Major Pollutants..............................13
2.3 Trends in Ambient Concentrations of Major Pollutants .....................13
2.4 Evolution of Federal Air Quality Permitting Programs......................14
4.1 Bibliometric Analysis to Identify Common Citations........................73
5.1 Documents with Common Citations ..........................................83
5.2 Document Orientation .....................................................86
5.3 Authors'Institutional Affiliations .......................................89
5.1 Document Types by Profession .............................................94
5.5 EPA Programs Discussed....................................................97
.6 Examination of Policy Alternatives ........................................99
.7 Discourse Space for Environmental Regulatory Policy Decisions ............Ill
6.1 Placement of Analysis within the Universe of Policy Discourse............117
6.2 Learning in an Information-Rich Environment .............................120
C.l Sample Citation Analysis Data Base Entry Form ...........................197
C.2 Sample Content Analysis Data Base Coding Form ...........................198
C.3 Sample Data Base Query ..................................................205
x


VI U\
Tables
Table
2.1 Summary of Key Federal Air Quality Permitting Programs .....................19
5.1 Most Commonly Cited Documents ..............................................82
5.2 Audience Addressed by Authors ..............................................88
5.3 Discourse Channels .........................................................88
5.4 Distribution of Documents According to Institutional Affiliation............90
5.5 Key Characteristics of Study Documents According to Institutional Affiliation 91
5.6 Stated Position on Regulatory Policy .......................................92
5.7 Threats Invoked According to Institutional Affiliation......................93
.8 Document Purpose According to Author's Profession ...........................96
.9 Analytical Focus According to Institutional Affiliation .....................98
5.10 Trends in Use of Quantitative Decision-Making Criteria ....................101
5.11 Diversity of Theoretical Perspectives......................................102
5.12 Examination of Alternatives................................................104
A. l Key Air Quality Permitting Program Decisions by Program .................126
B. I Documents Included in Content Analysis ....................................128
B.2 Documents Included in Citation Analysis ...................................160
B.3 References Cited Multiple Times in Study Documents ........................172
XI


1. Introduction
Do not ask What is policy analysis?" ... Ask rather "What can we make analysis become?"
Aaron Wildavsky
How do regulatory policy analysts participate in discourse and knowledge transfer? In
what forums do they introduce, and how do they represent, their arguments? With many
national-scale policies now at critical stages of development, these experts have an
opportunity to make significant contributions. Regulatory agencies are being tasked to solve
important problems. Calls for workfare' programs, cost-effective air quality controls,
deregulation of the electric power industry, health care reforms, and equitable
telecommunications policies all involve key public interests and require crucial choices. By
raising questions such as How clean is clean?, federal environmental policy poses
especially tough challenges and offers significant analytical opportunities.
1.1 Unanswered Questions
This dissertation examines to what extent given current developments and associated
opportunities analysts are contributing to regulatory policy discourse. If the goal is to
grasp or clarify important, contemporary problems in hopes of guiding policy choices, are
analysts making meaningful contributions? Are they repeatedly asking the same questions to
little apparent purpose, or investigating new and increasingly thoughtful ones?
These questions can be more generally posed with respect to knowledge utilization: Are
academics and practitioners sharing information, or is knowledge transfer restricted along
institutional, professional, and/or academic boundaries? Do practitioners and decision-
makers communicate their needs to scholars by indicating which types of research are most
1


useful to them? With respect to work flows and processes, are consistent standards employed
in representing analytical results or advocating alternatives? How are new technologies and
management strategies being incorporated into this process?
Many would agree that policy analysis has been transformed from its traditional,
objectivist roots into a more diverse set of methodologies. A substantial portion of recent
research addresses issues such as advocacy, citizen participation, and argumentation.
Investigators are exploring how various languages or modes of representation either enable
or constrain analysts' efforts.
The focus on discourse could ultimately provide clues as to how the analytic communities
are resolving their seemingly opposing views on evaluating policy problems. However, we
need to learn more about how scholars and practitioners in private and public institutions are
sharing or could share knowledge and how analysts are communicating their results.
The literature provides only minimal guidance as to how real-world policies should be
designed, or how rational problem-solving methods can best be applied in an institutional
setting. Generally, neither scholars nor practitioners have adopted reliable, useable standards
for applying interdisciplinary analytical techniques or for sharing new knowledge that is.
for talking to one another despite claims to the contrary.
The importance of relationships between various constituencies, including public and
private organizations, is evident in current regulatory research priorities and in the
management literature. Some analysts have identified interesting alternatives to traditional
regulatory strategies, such as optional institutional designs or voluntary and incentives
programs. Still, it is unclear which, if any, institutions are facilitating discourse or promoting
learning across institutional, disciplinary, or other battle lines, and to what ends.
2


1.2 Environmental Policy: Ripe for Review
Interventions by federal, state, and local governments are imposing record costs on the
private sector; significant public expenditures are also associated with these activities. Going
back as far as 1982, early in the life of many federal environmental statutes, roughly 1.5% of
the U.S. gross national product was being spent on pollution control programs (Harrison,
1982). Moreover, the effectiveness of many regulatory programs is hotly disputed. Concern
over the costs, efficacy, and/or administrative components of environmental laws and
regulations has led a variety of groups to prepare numerous studies and recommend a range
of statutory or program changes.
One thing that many stakeholders seem to agree on, however, is that U.S. environmental
regulatory policy requires reform. In recent years, the Environmental Protection Agency
(EPA) has promoted program reinventions, including the Common Sense Initiative and
Project XL (National Academy of Public Administration, 1997). This state of affairs was
suitably described by Davies and Mazurek in their recent report (1997):
If the showcase project of a pro-environmental administration is designed to help get
around the environmental laws, then something is wrong with these laws and the
programs that implement them (p. 1).
Several other federal programs are also experiencing reinvention and the challenges in
one area often mirror those in another. As individual programs are reevaluated, a myriad of
difficult contingent questions are being addressed concerning knowledge utilization,
federalism, administrative law, technology diffusion, government efficiency, global
competition, public involvement, and priority-setting. In addition, the traditional approach to
regulating U.S. industry, which relies on individual laws designed to correct isolated
problems, appears to need adjustment as does the widespread reliance on conventional
command-and-control methodology.
Current political emphasis on policy realignments such as regulatory reinvention,
3


negotiated rulemaking, risk-based decision making, and cost-benefit analysis guides
much of the discourse in this important area. Decision makers at the state and federal levels
have considered a number of requirements imposing efficiency- or risk-based criteria on
policy choices, while other events are forcing agencies to embrace more democratic
decision-making styles or consider the effects of their rules on specific groups. At the same
time, policies ^re being implemented under increasing public scrutiny. Analysis is requiring
significant resources and a more open, multidisciplinary approach. The proliferation of
nonprofit institutes and "think tanks'' seems to be adding more to the noise than the light. In
short, the volume of information and associated discussion of various policy components
and the speed with which it occurs is escalating almost to the point of analytical
intractability.
Environmental health policy is an area where analysis is particularly necessary. Roughly
25 years old, the U.S. pollution control programs have matured and are now at a critical
stage of development agencies are tinkering with new tools, enhancing enforcement
strategies, revamping state/federal relationships, and considering other possibilities as
political priorities change. For example, with respect to the electric utility industry,
conventional command-and-control strategies have given way to a more market-oriented
system of emission allowance trading. EPA is beginning to employ performance measures
that tie more closely to high-priority environmental improvements as compared to
previous evaluation techniques, which focused on metrics that did not adequately reflect
trends in environmental quality.
Federal regulations affect many diverse groups and therefore represent an ideal
opportunity to study how those stakeholders participate in wide-ranging discourse.
Consequently, the wave of environmental legislation in the 1970s and 1980s was one of the
most important developments for the fields of policy analysis: Design and implementation of
pollution control programs have pushed the evaluation envelope, requiring simultaneous
resolution of numerous scientific, economic, political, legal, ethical, and administrative
issues. A number of propositions have been transformed into real-world solutions, although
not always as smoothly or easily as originally anticipated.
4


1J Focus of This Study
Development, administration, and subsequent restructuring of several controversial
programs has resulted in accumulation of a vast body of environmental and regulatory
knowledge. As in other social science areas, observers of environmental decision-making
processes often express concern over whether analytical results are appropriately utilized.
To be of benefit to decision makers and others, policy information ideally should be
accessible, useful, and high in quality (i.e., available, relevant, and consistent with accepted
analytical and scientific standards). This study looks at formal discourse in an effort to
examine some of the factors that might affect knowledge dissemination or utilization in the
environmental policy arena. Discovering how to improve the quality or the utilization of
policy-relevant analysis is an important objective, and could potentially avoid a significant
waste of resources. For instance, resources may be wasted when important research results
fail to become woven into the policy-making fabric, or when an analyst reinvents the
wheel by recreating information that had already been generated, but was undiscovered.
One aspect of knowledge use in particular the positioning of analytical results within
the continuum of discourse on policy decisions has gone relatively unexplored. How do
regulatory analysts participate in the discursive process? How well are their contributions
organized and made available for knowledge dissemination and transfer?
Consequently, this study focuses on an important subset of ongoing policy discourse:
Contributors modes of participation as they seek to influence decisions concerning
environmental policy. The results should provide tentative clues as to how analysts use
language to construct problems, include or exclude particular alternatives, and estimate
potential policy effects. At the same time, the concomitant state of knowledge organization,
transfer, and utilization are examined.
This research is intended to provide insight into where analysis fits in the argumentation
process and how various participants might become more effective. Toward that end, this
investigation included a survey of the policy literature, a systematic review of 25 years of


discourse on air quality permitting requirements, and a bibliometric analysis of contributions
to that same formal discourse. The research reported in this dissertation is as follows:
In Chapter 2, U.S. environmental policy is briefly reviewed, particularly EPA's air
quality programs. Also included is a summary of regulatory policy concepts and the
various ways in which they are being applied to environmental decision-making.
Chapter 3 presents a review of the public policy literature, focusing on the relevance and
role of policy analysis in regulatory applications.
The methodology employed in this research study is outlined in Chapter 4.
Chapter 5 provides an overview of the results and assesses their implications for policy
analysts working in the environmental field or other regulatory arenas.
Final conclusions and avenues for additional research are presented in Chapter 6.
The results of this effort reflect many of the current attempts to revamp some of EPAs
original strategies now that the federal programs have matured, a significant amount of
data on outcomes and administrative processes is available. The findings also illustrate the
noticeable lack of a consistent form or repository for this information. No clear role for
policy analysts is evident within the context of argumentation over regulatory policy this
may present an opportunity for those willing to take the lead in defining how analysis can be
better incorporated into routine processes.
Today, the management of environmental information is a key compliance-related issue
within the regulated community; streamlining of information requirements is closely
associated with many of the current regulatory reform efforts. However, advances in
management of environmental or policy knowledge across organizational boundaries are
lagging far behind.
6


2. Development of U.S. Environmental Regulatory Policy
Considering the world all round,
A thorough study has recently found:
To save the place for the human race
Is not economically sound.
Heinz Kallman
The discussion in this chapter draws a picture of the current state of federal
environmental regulatory policy. In addition to key statutes and administrative programs,
analytical considerations and management strategies are also examined. Topics include:
The framework for the U.S. environmental programs;
Overview of EPAs air pollution control efforts;
Trends in regulatory policy;
Challenges to environmental program design, evaluation, and implementation; and
Opportunities presented by new technologies and management strategies.
2.1 Origins of the EPA Framework
Several pieces of landmark environmental legislation were passed during the 1970s and
early 1980s, including the Clean Air Act (CAA); the Environmental Protection Agency
(EPA) was established to administer those statutes. Needless to say, the agency has had its
hands full attempting to implement the more controversial mandates, including measures
restricting hazardous waste disposal (under the Resource Conservation and Recovery Act),
remediating contaminated Superfund" sites (per the Comprehensive Environmental
7


Response, Compensation, and Liability Act), limiting effluent discharges to U.S. waters (the
CWA), and controlling air pollution (the CAA).
While each of these programs possesses some unique characteristics, they also have
drawn attention to several significant environmental policy questions of more general
applicability. For instance, the tension between agency discretion and legislative authority
appears in sharp relief across many of the EPA programs.
2.2 Air Pollution Control
Air pollution became a major issue in the United States during the 1960s. Since then,
concern over human health and the environment led Congress to pass a series of bills,
collectively known as the Clean Air Act, or CAA. The first, passed in 1967, provided some
authority for setting air quality standards; however, the program did not have real regulatory
teeth until the CAA of 1970. Significant revisions were adopted in 1977, followed by an
even more significant revamp in 1990 (EPA. 1993).
The long and complicated history of EPAs clean air programs is well documented and
will not be repeated here. Suffice to say that this legislation has triggered an enormous
amount of program development and compliance-related activity within federal, state, and
local governments; private industry; and other interested organizations. The complexity and
magnitude of the associated administrative and technical requirements are often
overwhelming.
A majority of EPAs requirements pertain directly to protection of ambient air quality
this is achieved by enforcing compliance with air quality targets known as the national
ambient air quality standards (NAAQS). The NAAQS establish maximum allowable
concentrations for six major or criteria pollutants: lead, carbon monoxide, nitrogen oxides,
sulfur oxides, particulate matter, and secondary ozone (which is controlled by limiting
emissions of volatile organic compounds, or VOCs). Several types of control requirements
8


are imposed on polluters to ensure that all areas of the nation attain (i.e., satisfy) the
NAAQS; so-called nonattainment areas face additional source control requirements and
possible sanctions such as withdrawal of highway funding.
When EPA first began adopting and administering clean air regulations, the most obvious
way to achieve reductions was to control the larger stationary sources (industrial pollutants)
and mobile sources (automobile and truck emissions). On the mobile-source side,
implementing the new CAA policies involved primarily the automobile manufacturers and
the fuel processors (e.g., limits on lead in gasoline); the political tradeoffs were not as clearly
in the foreground for many Americans nor for many organizations.
Imposing new requirements on industrial facilities was much more visible, however, and
directly involved thousands of individual entities in significant compliance activities. This
"low-hanging fruit was targeted primarily by requiring emission permits that set specific
limits on pollutant releases from the specified facility. Industrial sources are now subject to a
variety of site-specific or industry-specific emission limits depending upon whether they are
new or existing facilities (and certain other factors). Debate over decisions concerning how
to design these control requirements is the focus of this dissertation.
The traditional method of addressing pollution problems is referred to as command-and-
control regulation because it often mandates use of specific technologies or compliance
techniques, rather than allowing flexibility for the individual facilities. Most of EPAs
permitting rules are administered and enforced by state agencies, adding a whole new layer
of complexity to the design of command-and-control schemes. Furthermore, certain air
quality issues are handled on a regional basis (e.g., transport of atmospheric ozone).
As shown in Figure 2.1, policy affecting stationary source control typically takes shape
through a complicated series of decisions at the federal, state, and (possibly) local levels
both at the state house and in the courthouse. Core requirements are driven by the CAA and
by EPAs implementing regulations. However, details as to how individual state
governments will ensure compliance with those requirements are spelled out in complicated
documents known as state implementation plans (SIPs). States also develop their own sets of
air quality regulations that may or may not mirror the EPA requirements, depending on the
9


circumstances (at a minimum, the state programs must impose levels of control equivalent to
the CAA requirements).
U.S. Office of
Congress President
Federal statutes
(e.g.. Clean Air Act
and amendments dated
19?0. 197?. and 1990)
T
State statutes
(typically confirm or
expand on requirements in
Federal laws)
EPA , OMB
Federal regulations
and policy statements
(e.g.. EPA!s PSD rules,
operating permit program
requirements, or
modeling guidelines)
State regulations,
policies, and procedures
(e.g.. SIPs. permitting
regulations)
Pollution control
agencies and
regulated entities
Implementation
and compliance
{e.g., decisions
affecting permits
for individual
regulated
facilities;
T
SUte State
legislatures governors
State regulatory agencies,
administrative pollution
control boards
A
Key influences
Policy
outcomes
~T~
Program
evaluation
(feedback)
Public comment
on program
proposals and
implementation
Reform efforts Court
(e.g.. Common Sense decisions
Initiative, cost-benefit
requirements. Project XL) Enforcement :
_ actions
FIGURE 2.1
The Clean Air Decision-Making Pipeline
10


For purposes of this discussion, those statutes and regulatory programs that establish new
control requirements affecting a broad segment of economic activity are referred to as big
decisions (e.g.. the acid rain program formed by the 1990 CAA amendments). Others, such
as decisions on whether to approve a particular permit application, will be referred to as
"small decisions (this is not intended to imply that they would have a small impact on
that particular permittee, necessarily). Oppenheimer (1983) notes the potential for certain
administrative alternatives to result in a "tyranny of small decisions. This has implications
for the role of policy analysis, as discussed later.
2.2.1 CAA Round Two
Judging from EPA reports on pollutant levels (1995 and 1996), early CAA efforts were
successful some of the agency's data are illustrated in Figures 2.2 and 2.3. Elimination of
leaded gasoline had the most dramatic impact. Several program adjustments were made
during the 1970s, at both the statutory and administrative levels. After 1980, however,
intense debate stalled efforts to further amend the statute. Meanwhile, widespread study and
analysis began to form a substantial body of knowledge on policy implementation with
respect to social regulation.
The 1980s became a time for doing reality checks on the CAA programs. The focus of
these activities included fine-tuning requirements and administrative strategies, in addition to
laying the groundwork for new initiatives. For instance, it became apparent that the
legislative mandate to control toxic pollutants through the NESHAP (national emission
standards for hazardous air pollutants) program was generally considered a failure. Congress
had authorized EPA to set standards for toxic compounds, providing an ample margin of
safety this led to intractable debate over complex scientific issues such as exposure
thresholds, and only seven NESHAP were ever promulgated.
11


Change(%) Change(%)
60
40
20 -
0 -
-20
-40 -
-60
-80 -
-100
60
40
20
0
-20
-40
-60
-80
-100
Sulfur
PartiaJate Oxides
Cartoon Nitrogen
Monoxide Oxides
Compounds Lead
FIGURE 2.2
Changes in U.S. Emissions of Major Pollutants
r p
Sulfur
Particulate Oxides
r"
Zero
Change
Carbon Mtrcgen
Monoxide Oxides
FIGURE 2.3
Compounds Lead
Changes in Ambient Concentrations of Major Pollutants
1970-80
1980-91
1970-80
1980-91
12


When it revamped the CAA in 1990, Congress found a foolproof way of eliminating this
problem with administrative discretion by targeting hundreds of specific HAPs for pollution
controls. Problems also developed within the fast-growing bureaucracy that followed in the
wake of the 1970 and 1977 CAA legislation. Natural science research and social science
theory had collided with administrative reality.
In 1981, Johnson considered the challenges, costs, and delays associated with
bureaucratic governance:
Processes are not only important, they are critical to producing desired results. Since
the economic and political landscape is littered with the wreckage of well-intentioned
but disappointing programs, the thoughtful activist cannot ignore the economists'
warnings. Government environmental protection programs have not fulfilled their
positive promises on the one hand and have led to unanticipated negative
consequences on the other.... [t]he resulting frustration cannot be eliminated by
means ofbetter programs with better people running them.
Instead, it is obvious that more attention should be devoted to the institutions and processes
that led to the original, undesirable outcome as well as to the processes set in motion when
we adopt new programs, (p. 218)
The legislative logjam of the 1980s was broken with passage of the latest CAA
amendments in November 1990. With many of the fundamental questions about major
pollutants seemingly resolved, Congress began targeting more special cases.
A progressive approach to addressing acid rain was adopted: Title IV of the amendments
established a market-based methodology for sulfur dioxide emission allowance trading by
electric utilities. Title III mandated maximum achievable control technology (MACT)
controls for sources of specified HAPs, at least temporarily avoiding the scientific logjam
within the NESHAP program.
Generally speaking, EPA u'as given relatively little discretion in choosing how to
implement many of the new measures. In fact, the new Title V provisions required the
agency to establish an additional administrative set of federal permitting requirements that
mandate no new pollution controls (and therefore could potentially result in no additional
policy benefits, as discussed further below).
13


2.2.2 EPAs Permitting Programs
The CAA permitting programs demonstrate that both the big and small decisions can
haunt administrators, analysts, the regulated community, and even legislators for years to
come. As shown in Figure 2.4, EPA is still attempting to redesign its controversial new
source review (NSR) program, first established during the 1970s.
Phase i: Creation of EPA, development of early programs
New source review
(c-C- PSD);
ambient standards
Early 1970's
Late 1980's
_________________________I
Learning, feedback, change
Phase II: Revamp of early programs, adoption of new policy tools
Acid rain and Revision of PSD;
Title V permit new ambient
programs standards
Early 1990's
FIGURE 2.4
Evolution of Federal Air Quality Permitting Programs
14


Many core elements of EPAs air quality control program are in the second phase of
development; this time, the political environment, the available information base, and the
stakeholders sophistication are substantially different, however. The agencys three primary
permitting programs the focus of this research are described briefly below.
2.2.2.1 New Source Review: Preconstruction Permits
As the name implies, the NSR program mandates pollution controls for new (or
reconstructed) facilities. EPA's requirements apply differently to sources being located in
nonattainment areas versus those in so-called pristine areas. In the latter case, the regulations
implement a concept known as prevention of significant deterioration (PSD); a proposed
facility that will emit one or more pollutants over certain threshold levels must obtain a PSD
permit prior to beginning construction. Sources seeking emission permits in nonattainment
areas are restricted even further generally, they must obtain offsets to ensure no
additional reduction in air quality (i.e., they are responsible for demonstrating that their
pollutant releases are being offset by emission reductions elsewhere within that geographical
area). The NSR program currently being redesigned is characterized by:
Extreme complexity.
Confusion over applicability, and
Legal challenges forcing EPA to revise its strategies.
Oren (1988) and Meiburg (1991) have examined the intricacies of the PSD program from
political science and legal perspectives. Technical complexity, the role of the courts, and
transfer of policy discretion from Congress to EPA are long-standing (and apparently
unresolved) issues associated with the PSD program.
15


2.22.2 Acid Rain Permits
Based on the results so far, this permitting program will likely go in the win column for
economists and policy analysts. The acid rain program, instituted by Title IV of the 1990
amendments, requires a new type of permits for coal-fired electric utilities. However, rather
than mandate specific control technologies in the conventional command-and-control format,
facilities are allowed to trade allowances for sulfur dioxide emissions. This market-based
system is implemented on the Chicago Board of Trade. Pleased with the acid-rain program
performance, in 1998 EPA proposed a second emission-trading scheme for nitrogen-oxide
pollution, which contributes to smog.
Under these programs, the details of who can emit what, and how trades are
accomplished, are all extremely complex. The general concept, however, is not: A facility
may choose to incur the expense associated with newer control technologies, or alternatively
purchase additional emission allowances (i.e., rights) on the open market. This innovative
policy design has returned a significant degree of operating flexibility to the regulated
community.
2.2.2.3 The Operating Permits Program
Operating permits are frequently referred to as Title V permits because the program
was established by Title V of the 1990 CAA amendments. Like the acid rain initiative, this
one is also unique: The statute imposes no additional control requirements on polluters.
Instead, the operating permit regulations serve as an administrative or enforcement device.
From an efficiency standpoint, the policy approach being applied in Title V is 180
degrees from that embodied in the acid rain program. Allowance trading allows facilities to
select the compliance strategy they believe is most efficient, while still achieving the
16


required environmental results. The operating permit program offers no defined
environmental benefits, yet imposes regulatory burdens on federal, state, and local agencies
and private industry. EPAs regulatory impact analysis for the Title V program offered legal
and economic arguments for the operating permit rules (which the agency was required to
develop under the law). EPA stated the benefits of the program in qualitative terms,
including more efficient enforcement, fewer legal actions, and administrative savings by
consolidating source requirements into one document; the cost of the operating permits rules
were estimated at more than $500 million (EPA. 1992).
An operating permit is like a regulatory "bucket into which EPA places all the diverse
CAA-related requirements applicable to a given facility. While the water and hazardous
waste programs have long included comprehensive permits spelling out all applicable
regulatory requirements, the air pollution program has not. This new layer of permitting is
intended to establish a means of accounting for all the emissions from a facility, which
ideally could support a more market-based system. In addition, the permits will facilitate the
multimedia investigations and enforcement being adopted by EPA.
This program represents somewhat of a missed opportunity for policy analysts, since it
garnered little attention or debate while Congress was considering whether to adopt these
provisions. However, since then. Title V has been extremely controversial because of the
potentially burdensome procedural issues associated with obtaining and operating under one
of these permits.
The central controversy involves determining in which cases a permitted facility must
obtain agency approval before instituting a physical or process change that might affect
emission rates. The environmental community has argued for the right of the public to
comment on virtually all substantive permit revisions, which would slow the process of
reviewing modification requests (e.g., a potentially large emission increase could be
achieved through a series of smaller permit revisions that escaped the public comment
process). However, the associated delays pose a significant problem for facilities by limiting
their operational flexibility to respond quickly to customer, supplier, or market changes, or
correct emergency equipment problems.
17


Litigation has been extensive, and EPA has struggled to satisfy the many stakeholders
(Clean Air Implementation Project v. EPA. 1996). The agency was late in promulgating its
original regulations; since then, it has proposed significant program revisions on three
occasions, although none of those were ever finalized. The third re-proposal was withdrawn
within a month of publication in the Federal Register.
Another unique policy feature added by the 1990 amendments was the notion of
application and permit shields. In essence, these shields protect a permit applicant or
permit holder from enforcement action in certain circumstances once a variety of procedural
requirements have been satisfied. For example, the permit shield represents agreement
between the agency and the facility that all required elements are included in the plant's
operating permit, and that no enforcement action will be taken so long as the facility
complies with all provisions of that permit. Like many other administrative activities under
the CAA, however, this one can become quite complex in a given application.
The Title V program has highlighted some interesting information management and
institutional memory issues as well. For instance, because of the nature of the operating
permit system, members of state agencies and the regulated community have found
themselves in the position of needing access to various states SIPs to review specific
provisions. This has proved to be challenging indeed, since a SIP represents a lengthy series
of approvals and detailed program changes (i.e., small decisions); many apparently were
not well documented or managed, complicating subsequent information retrieval.
Table 2.1 outlines key features of the three air quality permitting programs examined in
this dissertation. The summary underscores the diversity of these regulatory efforts.
18


TABLE 2.1 Summary of Key Federal Air Quality Permitting Programs
Program CAA citation Description Comments
New source review Title I Requires preconstruction permits and emission limits for many new' facilities in areas: - In violation of the NAAQS (nonattainment areas), or - With pristine air (the PSD program) Complex Undergoing program redesign
Acid rain Title IV* Permits are required for many coal-fired electric utilities Limits sulfur dioxide emissions; compliance with permitted emission levels is achieved by installing control equipment and purchasing allowances Allows trading of emission allowances as an alternative to installing additional control equipment
Operating permits Title V* Umbrella" permit required for most emitting facilities Imposes no additional emission limits Sources may qualify for permit or application shields Quantifies site-wide emissions as possible currency for future market-based program Public involvement is somewhat at odds with operational flexibility
Citation to 1990 CAA amendments.
19


2.3 Quantitative Decision-Making Methods
Considering just the "hard science of air pollution control without factoring in the
economic impacts or political processes poses difficult challenges with respect to policy
analysis, public management, and program evaluation (Lefohn, 1997; Liu, 1998). In such a
complicated technical environment, its not just a question of rulers (i.e., decision-makers)
and how they respond to information from their helpers (analysts).
Regulatory decision-making, particularly where environmental and health policy is
concerned, now revolves largely around scientific and quantitative techniques; as with policy
analysis in general, there exists a strong temptation to apply a rigid methodology. While the
two most prevalent types of quantitative approaches cost-benefit/effectiveness analysis
and risk assessment offer advantages, they do not fully resolve our questions. These two
prevalent methodologies are reviewed below.
2.3.1 Cost-Benefit Analysis
Cost-benefit analysis and similar techniques rooted in welfare economics have become
institutionalized in many regulatory settings, partly because they lend themselves to
widespread application more easily than other tools (Fuchs, 1987; Weimer, 1991). These
analyses involve a formal weighing of the estimated costs and benefits associated with a
particular policy action. Often, analysts attempt to convert the non-monetary benefits (e.g.,
reduced health risk) to units representing their economic value (e.g., dollars) to allow a direct
comparison to the estimated costs.
No matter what theories are used to justify a program, and no matter which groups are
identified as targets, key regulatory policy questions are often sketched within a welfare
economics frame cost-benefit analysis represents one component of policy analysis that
20


has become embedded in agency activities. The term usually refers to studies in which both
the benefits and costs can readily be quantified and compared based on values in the
marketplace (Weimer & Vining, 1992). In other cases, where an assumption or economic
technique is used to establish a surrogate value for nonquantifiable benefits such as peace of
mind, human lives, or clean ground water, authors may refer to cost-effectiveness analysis
(Patton & Sawicki. 1986). These evaluations are intended to compare the costs of a program
(which are usually quantifiable) with the results as a more abstract gauge of the program's
effectiveness. Certainly, this approach is more valuable when alternatives are being
compared (Downs & Larkey, 1986). Weimer and Vining (1992) go further and identify
conditions under which various types of analyses can and should be performed.
Quantitative comparison of the costs and benefits of federal programs first gained
official sanction with the U.S. Flood Control Act of 1939 (Gramlich, 1981). A key factor at
that time was impact on national income (Wildavsky, 1969). The cost-benefit approach to
regulatory analysis became somewhat institutionalized over the next 35 years, enjoying
nonpartisan support. During the Ford administration, the notion of comparing costs to
benefits was more rigorously applied (Campen, 1986); much of the concern at that time was
the potential impact of federal policies on inflation. President Carter was also a proponent of
economic analysis; the development of more refined cost-benefit techniques coincided with
the general tendency at that time toward quantitative decision analysis (Quade, 1975). Early
on. Miller and Yandle (1979) evaluated the application of cost-benefit analysis to social
regulations, using case studies from the Council on Wage and Price Stability.
In 1981, the cost-benefit analysis became a requirement for major federal regulations,
i.c., those with an expected impact of $100 million or more on the domestic economy
(other circumstances also trigger the requirement). Although some might say that requiring
cost-benefit techniques is misguided, the intent of at least some proponents was to de-
politicize decision-making processes (Fuchs, 1987; Luken & Fraas, 1993). Reagans
Executive Order 12291 outlined the means by which the Office of Information and
Regulatory Affairs (OIRA) would review regulations proposed by federal agencies.
21


Economic review of major regulations continues today; in 1993, President Clinton
established a similar plan with another executive order.
Currently, under Executive Order 12866, federal rulemaking actions must be evaluated
using a cost-benefit scheme, although subsequent decisions are not necessarily based on the
results of that economic analysis. EPA. among others, has developed specific guidelines for
performing a required regulatory impact analysis, or RIA (1991). Likewise, the Office of
Management and Budget (1992) has developed some financial ground rules for preparing
economic estimates.
Critics note that the cost-benefit approach illustrates the difficulty of incorporating
values into analysis; it emphasizes computation of expected policy effects, not identification
of alternative strategies or values such as equity or equality (Schmid, 1989; Swartzman,
1982). Such an exercise can answer questions about impacts and government efficiency;
however, total reliance on welfare economics would certainly not advance the cause of
policy analysis or promote other societal goals (e.g., environmental protection).
The realities of performing complex analysis and communicating it to decision-makers
can also limit the usefulness of the studies (Morgenstem, 1997). Besides fundamental
theoretical and analytical challenges, the representativeness of a given analysis can also be
hindered by the obvious difficulty of measuring the 'real' costs of regulatory compliance.
For example, many federal regulatory programs have grown so complex that they impose a
significant burden of "knowing'1 numerous professional firms have formed simply to help
public and private entities track requirements and compliance efforts. Nevertheless, one
would be hard pressed to actually document all the associated costs. GAO discovered that
firms have difficulty itemizing all costs imposed by regulatory requirements (1996b).
Mishan (1976) delved deeply into the complications of cost-benefit analysis. Among
other things, the issue of standing has been debated. For instance, the analyst eventually must
draw a line and stop weighing the impacts of sequences of events. As pointed out by Stone
(1988), items typically counted as costs (e.g., expenditures made by a government agency)
also will serve as income to the individuals who perform the service on behalf of the
government; thus, to neglect these benefits denies standing to these individuals and also
22


might distort the results of the analysis. This could explain how analysts sometimes fail to
take into account political forces: for example, an agency defending its own program might
be seen by critics as a beneficiary of a set of proposed regulations, attempting to preserve
organizational longevity' (Kaufman, 1976) although the cost-benefit analysis would not
"count this as a consideration.
Economists continue to refine sophisticated methods for situations where impacts are not
easily computed using a similar unit of measure (Crasswell, 1989). Moore (1990) analyzed
the benefits of the 1990 Clean Air Act amendments using hedonic pricing, attempting to tie
clean air to higher housing costs. Others have examined the problem of benefits that shift
with time using "unstable analyses (Baumol & Oates, 1975; Cinti, 1990). Shadow pricing is
a widely studied technique (Crew, 1985; Dasgupta & Pearce, 1972), as is existence value
(Kopp, 1992); contingent valuation/willingness to pay is another (Carson & Mitchell, 1993;
Kahneman & Knetsch. 1992). The usefulness and applicability of approaches such as these
continue to be debated. Meanwhile, federal agencies seeking to defend their actions are
embracing sophisticated techniques for calculating benefits and costs; EPA has sponsored
the development of a contingent valuation methodology for ground-water cleanups as a
means of valuing project benefits (McClelland, Schulze, Lazo, Waldman, Doyle, Elliott, &
Irwin, 1992).
Many of the strengths of cost-benefit analysis parallel the strengths of other quantitative
techniques (Quade, 1975). They can make the analyst or decision-maker more aware of what
he knows or does not know (Dewhurst, 1972). Another strength is recognized even by
critics: Deliberate weighing of the costs and benefits associated with a proposed action aids
in identifying alternatives, allowing the elimination of unworkable ones.
In addition to the standing issue, policy analysts cite the promotion of efficiency at the
possible cost of equity and equality. Some contend that the effort to attach dollar figures to
such squishy benefits such as human lives, environmental protection, and avoidance of
false advertising is simply not worthwhile (Hoehn, 1983). Others note the lack of useful data
even in many cases when the benefits and costs are seemingly quantifiable (Bobrow &
Dryzek, 1987). Lavey (1993) uncovered significant inconsistencies in one agencys use of
23


economics and cost-benefit analysis in the rulemaking process (e.g., conflicting assumptions
between different rulemaking projects). Still others believe that attaching a price to a human
life is unethical and inappropriate (Self, 1977). Furthermore, complex issues may be quickly
oversimplified in an effort to select an alternative. As discussed by Gramlich (1981), simple
ranking of alternatives by cost-benefit ratios hides many of the pertinent facts associated
with various choices. However, when results are viewed as merely one indication of whether
or not a particular alternative is worthwhile, the economic analysis is not dangerous.
According to Downs and Larkey (1986):
If benefit-cost analysis is ever to be become more useful, it will have to be viewed as
an exercise in discovering values and alternatives, and not as a sufficient decision
procedure. Few benefit-cost analyses appear to have had much impact on decisions,
but many of those that have had an impact have apparently considered real
alternative programs and not just one real program and straw-men programs.
(p. 118)
Unfortunately, the literature does not currently offer a practical, comprehensive
methodology for practitioners within agencies to follow in developing individual rules.
These agencies already are required to do more than simply perform cost-benefit studies. For
example, the Small Business Regulatory Enforcement Fairness Act (and the Regulatory
Flexibility Act before it) requires that the impacts of new regulations on small businesses be
considered. These efforts respond to evidence that the federal regulatory programs impose a
disproportionate cumulative burden on small businesses (Small Business Administration,
1995).
Agencies have had difficulty interpreting and complying with these statutory
requirements, yet again illustrating the difficulty of prescribing an analytical approach that
can be applied in multiple settings (GAO, 1995). In a sense, by considering one constituent
group small business separately, this represents an attempt to introduce values other
than efficiency into the analytical process. However, it also might serve only to boost the
level of pointless quantification. This attempt to make allowances for particular groups is
supported in the literature. Ukeles (1977) called for tools that recognize group impact
differentiation; he recommended client analysis, which differentiates groups with relatively
24


homogeneous interests or needs and orders them by the degree to which they are likely to be
affected by a decision. Trade-offs then would be constructed on the basis of the net benefits
and losses to the most highly impacted groups; one benefit of this approach is that it could
allow an analyst to focus attention on groups that are not vocal in the political arena.
Schneider & Ingram (1997) also support this view.
Numerous researchers have investigated various aspects of placing administrative
controls on agency rulemaking, while considering the potential impacts on policy-making:
Boyd (1991), DeWitt (1993), and Howard and Benfield (1991) studied the influence of OMB
or other aspects of regulatory review. Fuchs and Anderson (1987) attempted to document the
number of instances in which OMB review resulted in a rule being returned to the
responsible agency for a redraft. Tracking what actually happened (and why) within a
political process is seldom easy: Investigators with the General Accounting Office ([GAO],
1996) found it virtually impossible to pinpoint in which instances an OMB review had led to
changes in a proposed regulatory requirement.
Nevertheless, use of cost-benefit analysis is often mandated by Congress as the
appropriate means of achieving efficiency objectives; some federal statutes require agencies
such as EPA to weigh the costs and benefits of their intended actions. Examining the role of
economic criteria in EPA's establishment of best available technology economically
achievable (BAT) under the Clean Water Act (CWA), Fraas and Munley (1989) determined
that "the evidence also reveals that a strict adherence to a cost-effective decision rule could
have substantially improved the efficiency of the BAT program. This failure suggests the
difficulty of achieving an efficient outcome through a bureaucratic regulatory process
subject to a variety of statutory criteria (p. 35).
Although blanket adoption of cost-benefit analysis as an administrative decision-making
tool poses key policy questions, it appears to be in no danger of extinction. In some cases,
these tools are used as weapons: Besides offering an opportunity (at least in theory) to de-
politicize decision making processes, cost-benefit analysis might also be used to delay
rulemaking through paralysis by analysis or strategic use of the legal system.
25


Both the 104th and 105th Congresses have considered a variety of proposals for
legislation that would somehow limit the scope of regulatory agencies. H.R. 450 (1995)
would have imposed a moratorium on new rulemaking projects. The Regulatory
Accountability Act, H.R. 3277 (1996), attempted to impose spending limits on agencies by
providing that new regulations imposing private-sector costs exceeding Congressional
authorization would not take effect.
Some proposals would impose statutory requirements for cost-benefit studies, giving the
regulated community leeway to sue over economic analyses of particular regulations.
However, opponents caution that an unintended consequence of such an effort at reducing
government intervention might be the creation of more bureaucracy, formed to perform the
required analyses and administer the activities of such a program. This has already happened
to a certain extent under the Executive Orders (Hams & Milkis, 1989).
Taking the cost-benefit requirement one step further, the Regulatory Improvement Act
(S. 981. 1997), currently being considered, would impose both cost-benefit and risk-
assessment restrictions on a broad range of agency actions, and also would require peer
review of those studies. In essence, this action would mandate the use of professional
procedures, although those procedures would be unspecified. Opponents note that such far-
reaching legislation uses a broad-brush to affect a wide variety of rulemaking activities.
Litigation, delays, and red tape are cited as possible outcomes, with little benefit to policy
analysis in some program areas.
Still, others continue to pursue similar, even more stringent requirements; the Federal
Regulatory Risk Assessment Act (S. 1728, 1998) places greater emphasis on quantifiable
risk assessment than previous proposals. That bill requires a risk assessment at the proposed
and final rulemaking stages for all major regulations dealing with health, safety, or
environmental risk; the assessment would become part of the agency record. The Mandates
Information Act (S. 389, 1998) would allow members of Congress to raise a point of order
against considering bills that would impose costs exceeding $100 million on the private
sector such a measure could pose analytical difficulties, since many regulatory cost issues
are not resolved until the agency makes a final determination. At the same time, because this
26


approach would apply an arbitrary limit based on overall economic impact, it would not
necessarily protect individual stakeholder groups equitably or equally.
2.3.2 The Risk Assessment Business
Quantitative risk assessment is a tool being applied widely in the health and
environmental segments of regulatory policy. In essence, it involves measuring (and possibly
comparing) risk levels associated with various actions, with risk representing possible
adverse and uncertain consequences. Incorporating complicated, scientifically based efforts
into policy decision-making processes is a massive undertaking; the Presidential/
Congressional Commission on Risk Assessment and Risk Management (1997) recently
released a comprehensive framework for environmental health risk management that
demonstrates the challenges of this policy approach.
By the end of the 1980s, EPA relied heavily on risk-based decision making when
selecting particular chemicals as regulatory targets, establishing acceptable contaminant
levels, or otherwise setting policy agendas. Andrews (1994) notes that:
The environmental control agenda was broadened and redirected, therefore, to
address the far larger domain of toxic substances, pesticides, drinking water
contaminants, hazardous wastes, and others.... To deal with toxic chemicals,
therefore. Congress enacted risk-based and risk-balancing statutes that required
the EPA to assess the risks of each substance it proposed to regulate, and then either
to protect the public with margins of safety against unreasonable risks or to
make choices that would balance those risks against the substances economic
benefits. In turn, EPA and other agencies (such as OSHA and the Consumer Product
Safety Commission) had to develop methods for setting risk priorities among many
possible candidates for regulation; for justifying particular regulatory decisions,
balancing risks against benefits', and for approving site decisions, based on an
acceptable risk ... (p. 210, emphasis in original).
27


However, in the rulemaking environment, the discourse centers more around cost-benefit
analysis than risk assessment at that point in the decision-making pipeline, EPA has
usually already applied risk management principles to establish a health-based standard or
target a particular pollutant for regulation. Once the rulemaking process begins, cost-benefit
analysis as required by the White House moves into the forefront.
The results of a quantitative risk assessment can be used in conjunction with a cost-
benefit analysis: After using risk assessment to estimate the risk to human lives posed by a
particular action, a cost-benefit analyst might compare the economic benefit of saving
those lives through regulation to the costs imposed by that regulation. Such an evaluation
would represent substantial reliance on two quantitative techniques, both with important
strengths and weaknesses. (Whereas the calculation of economic benefits for lives saved or
extended is considered inappropriate or impossible, lives saved can also be compared to
costs in a less rigorous approach known as cost-effectiveness analysis.)
Some investigators have adopted risk assessment wholeheartedly, seeing few alternative
decision-making or benefit-calculating strategies. A recent report by Resources for the
Future (1994) stated that
There are several reasons for the sudden interest in risk assessment but the major
underlying reason is the general recognition that government and private sector
resources are scarce and that it is therefore necessary to understand what society
gains from environmental laws and regulations. The only analytical method for
determining this is risk assessment. Once the premise of scarce resources is
accepted, the need to set priorities is unavoidable (p. 1).
However, a methodology of assessing only the health-risk aspect of environmental
regulations might ignore other policy considerations, such as distributional effects. Andrews
(1994) notes also that the U.S. environmental risk assessment framework in particular does
not specify how EPA should assign weights in order to establish relative risk importance:
In practice, risk assessment has dealt with these issues by oversimplifying them,
focusing on only a few human health effects. This simplification may, however,
28


obscure the more diverse considerations required by more complex decisions, (p. 217)
Thus, this technique might be subject to many of the criticisms lobbed at cost-benefit
analysis. As discussed in the previous section, recent legislative attempts to impose cost-
benefit analysis requirements on new regulatory actions have often included parallel
requirements that agencies perform (and defend) quantitative risk assessments. Again, this
attempt to restrict the growth big government could result in creation of a significant risk-
assessment bureau (Harris & Milkis, 1989).
Quantitative assessments focusing on only one (or a limited number) of issues (e.g., a
study of a certain group of chemicals or health effects) might appear to maximize risk
reduction, while actually ignoring greater risks not included in the scope of the evaluation.
Clearly, risk assessment can serve as a tool for priority setting, but decision makers will not
reach their policy objectives if their use of study results is itself not properly prioritized and
adequately incorporated into a policy design framework. Furthermore, like modeling and its
many quantitative cousins, risk assessment is subject to errors, unarticulated value
judgments, communication barriers, and other abuses.
2.4 Designing Pollution Policy
What is regulation? Francis (1993) defines it simply as "state intervention in private
spheres of activity to realize public purposes (p. 5). Regulatory issues are enormously
complex and difficult to grasp without a great deal of knowledge and effort (Wilson, 1980)
hence, they offer significant analytical opportunities. Noll (1985) explains that:
All levels of government attempt to control some private-sector economic decisions
to which the government is not a party. One such method of control is to assign to a
government agency the responsibility of writing rules constraining certain kinds of
private economic decisions, using a quasi-judicial administrative process to develop
these rules... Included in this classification are agencies that control aspects of
29


transactions such as the price or the quality of the good transacted, that mandate
certain features of the production process such as emissions-control methods and
worker-safety requirements, or that control entry, as by licensing, (p. 10)
From a policy analysis perspective, regulation is unique because it is cross-cutting,
affecting most private-sector industries and public-sector activities. Noll (1985) points out
that scholars" current focus on regulation is more practical than theoretical:
It may well be the case that the theory of regulation is very close to the general
theory of government policy, but regulation is a distinct kind of policy that has
spawned a distinct theoretical and empirical literature: indeed, in economics and
political science its study has been elevated to the status of a subdiscipline, (p. 10)
Development of any regulatory program has numerous legal, administrative, ethical,
political, and technical components. Kerwin (1994) provides an overview of the stages in the
rulemaking process, noting trends since implementation of the Administrative Procedure Act
(APA), identifying opportunities for agency exemptions or evasions, and evaluating the role
of constituent participation. Discussing accountability, he notes that rulemaking agencies are
expected to respond to multiple authorities (e.g.. the Constitution or Congress), each of
which yields considerable but different power over the agency furthermore, "there are
profound differences in the priorities and objectives of each of these authorities, (p. 71)
McCraw (1984) provides an account of how many of the "classic regulatory agencies
came about in the United States. Traditionally, regulation was justified along economic
grounds. Kahn (1971) and Sharp. Register, and Leftwich (1990) review the most common
"market failures that, according to economic theory, justify such intervention. Beyond the
classic ones externalities, imperfect information, and public goods several other
circumstances also are generally considered to justify government interference in the private
market. Weimer and Vining (1992) provide examples of nontraditional market failures (e.g.,
where a natural resource will be lost to future generations in the absence of government
regulation). Overall, economists, policy analysts, and other students of regulation tend to
30


agree on the overall philosophy behind government intervention in private transactions; the
disagreements lie in the degree to which this intervention should occur.
During the 1970s, federal policy expanded beyond that traditional economic regulation
of railroads, utilities, telecommunications concerns, and other concentrated industries or
monopolies to widespread social regulation with creation of EPA and the Occupational
Safety and Health Administration (OSHA) (Noble, 1986). Since that time, regulation has
been based on a new philosophy that affects the everyday lives of ordinary citizens; the
number of pages being published in the Federal Register has increased exponentially
(Reagan, 1987). A risk-avoidance strategy dominates much of the current regulatory effort,
particularly where health and safety programs are concerned.
The justifications for social regulation are not as clear-cut nor as universally accepted.
Francis (1993) explains that what is sometimes regarded as market failure is really
politically significant dismay with market outcomes (p. 10). He contends that, besides
being based on market failure, regulation is also justified by judgments that certain products
or services are hazardous to the health, safety, or morals of the population; especially in
recent decades, regulation is intended to reduce risks, protect morals, provide a middle
ground between opposing groups, or ensure stability when unsettling developments unfold.
Regulations are frequently used to implement redistributive policies; some programs are
designed to internalize externalities (Lowi, 1968: MacAvoy, 1992; Wilson, 1980). Because
of the diverse goals of different programs, regulatory policies are administered through many
different regimes. While conventional regulatory programs have relied on a command-and-
control approach, agencies are now experimenting with incentive programs, negotiated
rulemaking, and streamlined compliance procedures to reduce regulatory burdens (Meiners
& Yandle, 1989; NEPI, 1996).
Besides administrative procedures, political behavior also differs depending on the type
of regulatory strategy being employed. Wilson (1973) identifies categories of regulatory
politics based on the relative distribution or concentration of the policy beneficiaries and
targets. However, this taxonomy does not fully explain constituent behavior because it fails
to consider the degree to which the different categories have actual publics (May, 1991).
31


Effective environmental policy must balance several competing priorities. Based on
policy learning and experience during the environmental decades, a variety of researchers
have constructed conceptual models to explain certain phenomena. For example, Hahn
(1989) developed a scheme for designing environmental policy and applying it in an
institutional context, acknowledging the role of interest groups in decisions such as
instrument selection he sets up a framework for evaluating alternatives for achievement
of EPA's national ambient air quality standards (NAAQS) in so-called nonattainment areas.
Based on topics being addressed in the current literature, investigators will continue to
formulate numerous theories and designs to explain or prescribe policy choices.
2.4.1 Variables in the Environmental Equation
The decision-making environment has changed since the CAA and other environmental
laws were first adopted. Close public scrutiny of significant rulemaking decisions virtually
assures that EPA will be roundly criticized and frequently sued by either the
environmentalist groups or the regulated community, or both.
Mobilization of local activists by various interest groups has generated a cottage industry
of how best to manage environmental disputes, including strategies for communicating with
the public about risks, distinguishing between the needs and interests of local citizens and
those who claim to have standing to represent them, and negotiating for disposal site
approvals. For instance, Gorczynski (1992) examines in detail several models for negotiating
environmental disputes. Bacow & Wheeler (1984), Carpenter & Kennedy (1988), and
Crowfoot & Wondolleck (1990), also have addressed various aspects of community
involvement, risk communication, and negotiation in the public arena.
Now, with the advent of a market-based system for utility permitting, that environment
may shift once more. When momentum for the acid rain allowances program was first
gaining momentum, Oppenheimer (1983) predicted that
32


while regulatory decisions and actions will continue to be made under a permit
system, the matters decided will... tend to be of a more technical and less
obviously important nature, so that environmental groups will have difficulty
mobilizing support on any one of them and may find themselves on the wrong side
of a tyranny-of-small-decisions problem, (p. 143).
This raises an interesting point: Applying a governing set of rules to permitting
decisions, rather than handling each case individually, can add more consistency to the
effort, and does in fact, as indicated by Oppenheimer, make small decisions out of what
might have been big ones. This facilitates more systematic, comprehensive use of broad-
based policy analysis at the grass-roots permitting level: making the decision to issue a
certain number of allowances to facilities nationwide eliminated some of the need to
reevaluate each utility.
Environmental policy is subject to many influences. First, decisions are made within the
context of the political processes and thus encounter the complexities of interest group
politics, in which stakeholders attempt to influence decision makers to select their preferred
policy choice. For example, which type of emissions should be reduced, automobile or
smokestack? Oversight by OMB is another factor (GAO, 1993).
Authors have produced numerous examinations of the political processes affecting
environmental policy choices and implementation (Bryner, 1995). Agency discretion has
been an issue; in Pluralism by Design, Hoberg (1992) considers whether EPA has been given
sufficient authority along with its supposed autonomy; interest groups politics have played a
role in this tug-of-war. Greenwood (1984) spells out four types of agency discretion:
interpreting statutory language, balancing conflicting values, determining priorities, and
answering scientific and engineering questions. He notes that in exercising each type of
discretion, whether on procedural or substantive matters, an agency explicitly or implicitly
both embodies values and distributes costs and benefits among sectors of society.... [T]he
act of exercising discretion of any type always generates controversy, (p. 4).
Greenwood also offers an insightful analysis of the role of knowledge in government
regulation he contends that knowledge must be supplemented with discretion because at
33


any given time, agency staff will need to respond in the absence of complete information.
For example, when the results of contradictory studies have been presented, science and
engineering alone cannot provide the answer. Several different types of information that
might be useful in making regulatory choices are: information that is available, but
uncertain: information that is obtainable with modest effort; information that is obtainable,
but only with substantial effort: and information that is unobtainable in principle (e.g.,
knowledge about geologic stability at the proposed location of a nuclear waste site).
International activity is another important variable (e.g., the Montreal Protocol, to which
the United States is a party, mandates reduced emissions of certain ozone-depleting
chemicals meaning that EPA's regulatory policy must comply with both the CAA and
such international agreements).
Administrative and environmental law are also important to policy making. The
applicability of the CAA to many agency and private-sector actions has been examined by
the courts (Melnick, 1983). Overall, court decisions have been a significant factor in how
EPA has shaped some of its programs.
Notions about federalism have placed state and federal governments in opposition,
particularly with respect to environmental regulatory policy. In some cases, states and EPA
have sued one another over interpretation and/or implementation of certain CAA provisions
{Virginia v. Browner. 1996).
Experience continues to contribute to policy learning: The path from theory to successful
program results is a difficult one, indeed. Witness the statement of one analyst:
I hope to have persuaded the reader that creating marketable permits for air pollution
discharges is a difficult problem. As a long-time advocate of [a market-based]
approach, I have been taken aback by the many complexities I have uncovered. I
remain convinced, however, that marketable rights schemes are promising, provided
we view them cautiously and realistically. (Roberts, 1982, p. 108)
Eads (1981) called for abandonment of a legal model of regulation in which
individual statutes are intended to solve individual problems and adoption of a planning
model, in which regulation is viewed more as an alteration of a central decision-making
34


framework. The significance of smarter statutes (as a function of specifically targeted
populations) to successful policy outcomes has been examined by Ingram and Schneider
(1990). Environmental regulation provides a good example of the need for better statutes.
EPA has canvassed for regulatory reinvention alternatives that promote multimedia
enforcement strategies, increase agency accountability, provide operational flexibility to
industry, streamline agency rules, and reduce staffing requirements (NEPI, 1996). However,
analysts are discovering that overly prescriptive statutes hinder these initiatives (e.g., where
actual reporting deadlines are specified by law). As the major environmental statutes are
reauthorized over the coming years, researchers have an opportunity to steer legislators
toward statutory constructs that support regulatory policy improvements.
One of the characteristics of policy-making is an emphasis on individual decisions,
which results in an incremental focus. Wildavsky (1987) and Aaron (1978) discuss the
inherently conservative nature of policy analysis; Williams and Matheny (1995) note that
dialogue and environmental decision-making are significantly hampered by policy-makers'
focus on discrete issues. Federal rulemaking policy illustrates how this tendency can lead to
unintended consequences. While the current emphasis on cost-benefit analysis of new rules
supposedly raises the level of rational decision-making, it is done on a piecemeal basis.
Nowhere is an agency required to compare the effectiveness of a particular rule with others.
Thus, even if the analysis of a particular problem did represent the perfect embodiment
of all pertinent policy issues, the subsequent regulatory choice would be made in isolation,
not in the context of comprehensive policy goals, such as cost-effective improvement in air
quality (Lave, 1996). In his discussion of quantitative decision-making techniques, Quade
(1975) describes this situation as one in which individual components of a solution are
optimized, but the overall result is not; although it might be easy to break a problem into
discrete parts for analysis, recombining them is not so simple.
Several researchers have compared the cost-benefit ratios or risk-reduction
improvements promised by selected federal rules, noting significant discrepancies and
questioning whether resources were being committed to the right efforts (Bolch, 1993;
Smith, 1984). This problem has become part of the reform debate, and speaks to the need for
35


revised planning or possible multimedia, multi-program strategies. Breyer (1993) has
considered the problems posed by a piecemeal approach to rulemaking; he explores the
advantages of a multidisciplinary "umbrella group that would review proposals, consider
competing risk assessments, and make cross-agency or cross-program determinations
intended to maximize overall regulatory results.
Finally, more collaborative regulatory schemes, basing decisions more on information
and less on authority, are being investigated (Kettl, 1997). These include the use of voluntary
standards such as the International Organization for Standardization's ISO 14000 system as
an adjunct to EPA and state environmental rules.
Environmental policy is still evolving. A fundamental and continuing challenge involves
incorporating the results of natural science research into the policy framework. Calls for
reform abound, many of which are based on sensitivity to cost-effectiveness. A variety of
legal and administrative mandates are also driving agency action, along with court decisions
and international treaties. Furthermore, Congress has become much more specific over the
past 25 years about spelling out precisely what EPA must do to implement CAA policy; this
has generated additional questions about how environmental policy should be designed.
2.4.2 Regulatory Rcinvention
Efforts to reform regulatory programs are at an all time high (e.g., regulatory negotiation
initiatives and Vice President Gore's National Performance Review 1997 report on
Businesslike Government: Lessons Learned from America's Best Companies).
EPA has instituted its own programs to work more closely with the regulated community
to eliminate compliance requirements that produce no environmental benefits (i.e., to trim
paperwork requirements). These include Project XL (EPA, 1997) and the Common Sense
Initiative (EPA, 1997). Not surprisingly, some analysts have identified ways to reform and
improve the reforms (National Academy of Public Administration [NAPA], 1997).
36


Reform efforts abound, partly because the scope of regulation has expanded significantly
during recent years, and also because new technologies have recast the need for controls.
More than 15 years ago, Eads (1981) wrote:
The amount and scope of regulation applicable to industry in general has increased
to the point where the distinction between regulated and unregulated industries has
become meaningless. We are now living in a regulated society. Regulation affects
almost every decision made by businesses, both firms with market power and those
with a host of competitors, (p. 2)
The situation today has only worsened. While social regulation has changed and
expanded over the past 20 years, many economic-based regulatory restrictions have been
loosened, including controls on the natural gas, trucking, and air transportation industries
(Horowitz, 1989). Quirk (1981) has written extensively on the topic of deregulation and the
effects of industry influence over agencies. The concepts underlying cost-benefit analysis are
often employed to justify deregulation (Eads. 1984; Durant, 1992). Academics and
practitioners alike have written extensively on regulatory reform; many arguments are made
on the basis of economic impacts. French (1992) has examined the economic and other
consequences of pharmaceutical regulations. Smith (1994) has examined recent attempts to
quantify benefits of environmental protection. Barkovich (1989), Johansson (1993), Smith
(1976), and Swartzman. Liroff, and Croke (1982) have written detailed accounts of the
economic impacts of environmental statutes and regulations. Luken (1990) has attempted to
evaluate the problems with establishing efficiency in regulatory programs. Others have
addressed the difficulties faced by an administrator or agency attempting to ensure that
procedures and requirements are efficient (Bozeman, 1979; Gramlich, 1981; Hall, 1978).
These efforts raise many issues related to policy analysis.
Goodman and Wrightson (1987) assess several ways in which policy-makers might
achieve regulatory reform, particularly in environmental policy. During the 1980s and 1990s,
strategies have been grounded on legislative reforms, economic incentives, and management
changes (e.g.. revised enforcement procedures). However, Harris and Milkis (1989) note that
management strategies imposing cost-benefit and related economic analysis requirements
37


in the name of reducing burdens (government downsizing) can backfire, resulting in
substantial administrative organizations overseeing the regulatory review process.
Horowitz (1989) discusses other ironies of recent attempts at regulatory reform and the
more significant changes being made to economic, rather than social, regulation (e.g.,
telecommunications deregulation). Yandle (1989) notes that like new regulatory
initiatives reforms often alter redistributive effects, possibly relieving some burdens, but
raising new issues as well. He has found that regulatory programs are similar to other
government actions in that they frequently transfer wealth from politically weak to stronger
groups. In certain cases, critics of regulatory programs seek to reform the ways in which they
are structured or enforced, rather than the underlying program goals. For instance, some
groups want the states, not EPA, to have more responsibility for air pollution controls (Vig &
Kraft, 1993); environmental policies in particular pose important inter-governmental
questions, frequently pitting state and local agencies against federal bureaus.
Finally, like all discourse, discussion of possible reforms is not immune to the abuses of
language for political gain. For rhetorical purposes, Wilson (1980) notes that regulatory
issues are often framed using different terms depending upon whether that particular author
approves of the described activity. For example, the term regulatory capture is applied to
interest group-agency relations of which one does not approve, while more satisfactory
interest group-agency relations are frequently described as "citizen participation.
Assuming that the quest to find the "optimal means of regulating industry is a science,
then this effort has reached the "bureaucratic stage: McNamee (1994) explains that
bureaucratization of science involves the extent to which the production of knowledge is
accomplished on a large scale. Such big science production of knowledge can be
characterized as capital intensive, specialized, and having a complex division of labor (p.
409). With the health, safety, and environmental regulatory programs reaching maturity,
their analysis has indeed become big science (Noble, 1986; Price, 1986).
Regulatory reforms offer numerous analytical opportunities. In assessing information
requirements in regulatory policy, Bartlett and Apte (1981) concluded that
38


One area of practical uncertainty is apparent to all who deal with the problems of
regulation uncertainty as to exactly what the effects of any given set of rules will
be. There is an unacceptably high degree of trial and error in overcoming this, and as
a result far too much error. We must go beyond identification of areas where
regulation might be effective to the design of regulatory schemes that will be
effective, (p. 36)
The authors call for better understanding of the social nature of information, and contend
that regulatory objectives will fall short until policy variables are adjusted to reflect that
reality. In 1985, Wilson identified four research strategies that could contribute to
understanding of the nature of private-sector regulation: determining when and why
governments interfere in the market: estimating the effects of regulation; evaluating alternate
ways of intervening; and understanding alternative organizational structures and deciding
what difference the choices make. He found that up until that time, economists and others
had devoted significant energy to the first two avenues, while neglecting the latter two.
The evidence supports Wilsons conclusion: In outlining a research agenda for attacking
regulatory problems in the 1980s, key scholars focused more on determining costs and
benefits than on any other aspect of regulation (Ferguson, 1981). Other topics addressed at
length include political processes, market failure, and agency behavior evaluating
alternatives is not identified as a research priority (Eads, 1981).
This author reviewed doctoral dissertations completed nationwide over the past 15 years,
finding that hundreds of students have addressed the economics or the costs of regulation in
one way or another, while only a few dozen have elected to research regulatory policy along
other lines. To paraphrase Simon (1981), research bounds the space within which decisions
are made; we may be inadvertently limiting the boundaries for meaningful regulatory
decision-making. Virtually all constructs of policy analysis include not only careful
estimation of policy effects, but also identification and assessment of alternatives (Brewer &
deLeon, 1983; Patton & Sawicki, 1986); hence, the literature suggests that, at least through
the 1980s, comprehensive policy analysis was not being applied to regulatory problems.
While the importance of alternative administrative procedures to regulatory policy
success has not been fully investigated, it is now gamering attention in some circles. Jasanoff
39


(1993) contrasted four established approaches to incorporating science into regulatory
decision making (e.g., a science court), concluding that the most effective processes are
those most sensitive to the distinct characteristics of regulatory science. Breyer (1993) has
suggested that an umbrella group consider risk reductions across agency programs rather
than simply within one institution or program as an organizational alternative to policy
implementation. Bobrow & Dryzek (1987) offer a means of considering problems across
multiple (possibly conflicting) frameworks. The work of these investigators provides a
platform for valuable further research.
2.5 Where Does EPA Lie on the Information Technology Curve?
EPA has employed technology to solve problems of a scientific nature (e.g., dispersion
modeling and distribution of emission factor software). How successfully has technology
been applied to management of policy-related information? Some of the advantages of a
learning organization one that routinely incorporates new information into its activities
clearly could apply to an organization as expansive and diverse as EPA (see Senge,
1997). The extent to which the agency has had to revamp its CAA programs illustrates the
learning opportunities available.
Knowledge management is also becoming an important concern for those interested in
the management and performance of complex enterprises. A variety of secondary issues
and some possible management tools are just being examined. Stewart (1997) provides an
excellent overview of knowledge management concepts. Sharma (1997) and Albert &
Bradley (1997) address the placement of expert knowledge within an organization, or within
a group of organizations.
Although the formal recognition or quantification of so-called intellectual capital does
not appear likely (or even meaningful) for a governmental agency, EPA does possess
tremendous amounts of valuable knowledge in the form of subject-matter experts (SMEs). In
40


addition, the agency sponsors significant research outside its own boundaries. Moreover,
vast bodies of research are sponsored by other entities in the hopes of influencing EPA
policy. This burgeoning field of knowledge management might have important implications
for synthesis or sharing of knowledge "owned by the agency and other stakeholder
organizations.
The social science literature says very little about how government agencies, decision
makers, policy analysts, and associated stakeholders could or should take advantage of
available managerial and information science tools. More important are the ethical issues
that are becoming prevalent in this information age: Does EPA or some other entity or
collection of entities have an obligation to ensure knowledge accessibility? Bok (1983)
provides an excellent assessment of secrecy in the context of moral choice. Certainly, it has
serious implications for policy analysts; dissemination of policy-relevant knowledge (or the
lack thereof) also raises issues of elitism within governmental agencies or analytical circles.
Several federal agencies have made dramatic strides because of the Internet. For
example, the daily Federal Register is now available at no charge on the date of publication,
whereas only a few years ago it was available only as a subscription service, with several
days' delay due to mailing requirements. Virtually all agencies and most state governments
have world wide web sites for delivering the text of reports and other documents; many have
list servers to deliver press releases and other agency news or document information. EPA,
for example, offers an Online Library System (OLS) via world wide web or Telnet, on which
listings of many agency reports are available. The Food and Drug Administration (FDA) is
currently converting portions of its docket to electronic format.
However, it is unclear to what extent those tools are used to organize, collect, and
maintain a body of knowledge pertaining to policy analysis. Such a system could support
decision making on an ongoing basis, and reduce the extent to which agency staff must
reinvent the wheel with each new regulation.
EPAs regulatory docket is not yet available online, although the agency has developed
two systems for publishing public information on pollutant levels via the Internet. The first
provides water pollution indices for watersheds across the United States, and the second
41


summarizes the environmental impacts of individual industrial facilities according to an EPA
coding scheme. This coding program, the Facility Sector Indexing Project, demonstrates
both the power of new technologies and the danger of applying rigid, objective criteria
across a broad set of data critics (state agencies and regulated entities alike) claim that the
system overstates the risks posed by certain 'clean sites and understates those posed by
other, dirtier facilities (EPA Pollution-Grading Plan, 1998).
42


3. Policy Literature: The State of the Art
Knowledge of physical science will not console me for ignorance of morality in time of
affliction, but knowledge of morality will always console me for ignorance of physical science.
Blaise Pascal
Several schools of thought play a role in how analysis is applied to regulatory policy,
including economics, law, political science, policy studies, planning, natural science, and
information science. To shed the most light on the many aspects of analysis and its
application by decision makers, the literature is reviewed here along several lines:
Development of analytical methods and policy tools;
The roles of argumentation and professional standards: and
Factors affecting knowledge dissemination and utilization.
3.1 Policy Tools and Analysis of Alternatives
Policy analysis represents an application of the policy sciences methodologies.
According to Dunn (1981), policy analysis is "an applied social science discipline which
uses multiple methods of inquiry and argument to produce and transform policy-relevant
information that may be utilized in political settings to resolve policy problems (p. 33).
Ideally, the focus is on fundamental, important social issues; Lemer and Lasswell called for
the policy sciences to answer questions of significance to society (1951).
Identifying ways to transfer social science knowledge for eventual application in policy-
making circles is quite problematic. As discussed below, concerns over the relevance and
43


impact of policy analysis have inspired a number of explorations of new frameworks for
investigating public choices. Theory has shifted from a focus on positivist, rational methods
('traditional policy analysis) to a more comprehensive means of considering alternatives (a
post-modern approach). Rather than being treated as a separate component, the
representation and discussion of public choices policy discourse has been incorporated
into analytical and planning processes.
However, quantitative analytical techniques are still important. Lave (1981) discussed
various challenges of quantitative analysis, noting that dealing explicitly with uncertainty is
one of the most difficult aspects.
One possible implication of these problems is that quantitative analysis is
impossible, or at least useless. The analyses do provide insights into decision
making, however. Although analysis is difficult, it is necessary to impose order on
splintered, multifaceted problems. ... Analysis is not an end in itself. This has been
demonstrated by environmental and inflationary impact statements, which have
served mainly to delay implementation of new programs. The difficulties and
complexities of these problems underline the need for sensible analysis; at the same
time they show how easily the analysis can become useless or pernicious, (p. 127)
3.1.1 Traditional Techniques Give Way
Today, many methods employed by policy analysts (e.g., cost-benefit analysis) continue
to rely on quantitative, objective techniques (Patton & Sawicki, 1986; Stokey &
Zeckhauser, 1978; Weimer & Vining, 1992). Organizations have attempted to embrace
systematic, rigorous, rational approaches at the institutional level (Schick, 1966); however,
this sometimes led to well-documented failure, such as the program planning and budgeting
system (PPBS). Among other things, Wildavsky (1969) noted that this approach to budgeting
was too expensive, too involved, and that the system encouraged mindless quantification for
its own sake in gathering vast amounts of data. Nevertheless, some of these approaches
44


offered the advantage at least in theory of forcing decision makers to go back to
ground zero, rather than looking at the effects of incremental changes.
Criticism of so-called rational techniques stem from a belief that they do not properly
represent actual behaviors or incorporate societal values and goals (Dryzek, 1990), or from a
belief that we are incapable of properly designing and implementing analytical methods
grounded solely on rationality. Stone (1988) argues that the social science reliance on the
concept of the rational actor (i.e., the "rationality project) is unjustified, and that the use of
the society-as-market model is inappropriate in policy analysis. She criticizes economics for.
among other things, rejecting as "irrational any consideration of the obvious fact that
expenditures are income to someone else.
On the other hand, Lindblom (1990) argues that our failure to solve many social
problems is explained partly by our impaired ability to grasp all aspects of complex
processes. Who is to say that social interaction, rather than intellectual pursuit, cannot get us
where we want to go (Dryzek, 1990; Wildavsky, 1987; Williams & Matheny, 1995)? We
may arrive at an acceptable solution to a policy problem more quickly through discourse
among stakeholders than through calculation and theoretical analysis.
Some of this tension results from inherent differences between hard science and social
(soft) science, with a majority claiming superiority for the natural and physical sciences. But
even the notion and objectives of "value-free (hard) science have been successfully
challenged, sometimes by natural scientists themselves (Jasanoff, 1993; Proctor, 1991).
Differences between quantitative and qualitative empirical data are also cited to illustrate the
seeming "weaknesses of the social sciences (Bobrow & Dryzek, 1987). Numerous
assumptions underlie economic and other social science theory, even when supported by
quantitative research (Press, 1995).
Adherence to the notion of rationally calculated self-interest was challenged by Etzioni
(1988). who, when wondering how to define "rationality, pointed out that "obviously, if one
defines rationality in a way that makes it easy to meet the criterion, many more people will
be found to make rational decisions than if one uses more stringent criteria. Thus, if one
45


defines rationality as using whatever facts one commands, by whatever rules of inference
one believes in, even heeding the counsel of astrologists turns rational (p. 136).
Currently accepted wisdom says that policy studies are explicitly normative in nature,
intended to achieve certain public goals or values (Brewer & deLeon, 1983). While
traditional policy analysis has sometimes involved ill-advised attempts to impose a rational
actor framework, most agree this should not characterize policy analysis when properly
performed (Bobrow & Dryzek, 1987).
Concerns over the relevance and utilization of policy analysis are partly responsible for
shifts in the methods being employed by analysts. A substantial part of the long-standing
debate over appropriate techniques for policy analysis concerns the validity of so-called
rational approaches versus methods employing less objective criteria. MacRae (1993)
describes the two dominant models in the field: the rationalistic problem-solving model,
which concerns the internal qualitites of analysis, and the politicized context-determinant
model, which deals more with external contingencies (p. 291). Current research attempts to
synthesize these two models into an argumentative approach, not entirely methodological or
political, but avoiding radical separation of epistemological concerns (Fischer & Forester,
1993).
In regulatory policy analysis, pinpointing tractable problems demands simplification
frequently introduced by way of instrumentally rational, objectivist methods. However, at
the same time that many economists, analysts, and decision makers are fine-tuning cost-
benefit and other objective policy tools, others are rejecting that approach although not
without re-introducing more complexity into analytic processes. Many of the shifts in policy
theory represent rejection of rationality or attempts to combine it with other processes.
Dryzek (1990) describes instrumental rationality as the capacity to devise, select, and
effect good means to clarified ends; objectivism is rational choice concerning theories and
beliefs about matters of fact, values, and morals, made in reference to a set of objective
standards that are equally applicable. Flatly rejecting these two schools, he seeks policy-
making alternatives to objectivist and instrumental rationality, advocating a discursive or
communicative rationality, which restricts instrumental rationality to a more limited domain
46


and focuses on Aristotelian practical reason (i.e., persuasion, reflection on values, judgment
and free disclosure of ideas).
This forms the core of discursive democracy, hinging on the idea of competent
individuals who are free of domination, strategizing by the actors involved, as well as self-
deception (Habermas, 1979). The goals of discursive democracy are attractive: Each gets to
voice an opinion, and decisions are based on a deliberate consensus of those affected.
However, Dryzek emphasizes that the conditions of consensus and compromise are crucial
and prevent communicative rationality's degeneration into hopeless pluralism (p. 17). One
challenge is that this approach requires more active citizenship; the lack of citizen
participation is a widely recognized problem with current policy making (Fischer, 1993).
Other difficulties include the likelihood of self-deception (Lasswell, 1971)) and procedural
complexity (deLeon, 1992; Mansbridge. 1980).
Another focus of current research is on how policies are constructed (Weimer, 1993).
Bobrow and Dryzek (1987) examine methods of improving the outcome of analytical efforts
in the policy process. They caution against endless prediction of the effects of various policy
alternatives, and suggest better analysis through design, which involves (among other things)
more sophisticated definition of problems and consideration of theoretical frameworks.
Tension between rationality and alternative approaches tends to influence particular
researchers view's on the relevance of different analytical methods. However, while some
scholars lament the triumph of politics over reason, others believe this is a false dichotomy.
Albaek (1995) notes that most conceptualizations of the linkage between science and
politics have traditionally been informed by rationalist concepts of science and decision-
making. The result has been a false dichotomy between (legitimate) rational research use and
(illegitimate) political research utilization. This dichotomy must be overcome, on normative
as well as empirical grounds (p. 79). Supporting the idea that rational/political dichotomies
have been constructed in error, May (1991) reconciles policy analysis with consideration of
political environments, exploring the differences in policy design on issues both with and
without publics. Furthermore. (Weimer, 1993) points out that analysts (would-be
philosopher-kings) and decision-makers (irrational politicians) often have interchanging
47


responsibilities. In practice, they claim, the two schools are far from separate. Yet they are
still treated by many analysts as warring theologies.
Government reform offers important opportunities for analysts. Federal environmental
policies are under attack and require restructuring; information on relative risks and
competing priorities demands that the overall scope and effectiveness of many initiatives be
reexamined. However, despite recent theoretical trends, federal programs continue to apply
cost-benefit analysis, risk assessment, and related quantitative techniques to the review of
individual decision points, rather than to support more comprehensive ends.
The importance of dialogue and the extent to which it influences the analyst are now
widely recognized. However, the literature fails to fully answer how analysts in public or
private institutions should conduct dialogues amongst themselves or otherwise share policy-
related knowledge. Generally, scholars and practitioners have not developed a set of
standards for applying interdisciplinary techniques or communicating results during the
policy-making process.
3.1.2 Argumentation and Languages of Discourse
As noted earlier, policy theory now considers more eclectic, comprehensive approaches
to evaluating policy problems (Weimer, 1998). In this post-modem period, observers place a
great deal of attention on discourse and use of language (Williams and Matheny, 1995). In
fact, to many, discourse is viewed as intrinsic to analysis, rather than as a separate piece of
the policy process (Fischer & Forester, 1993). Majone (1989) stresses the fundamental role
of language: ... public policy is made of language. [Argument is central in all stages of the
policy process (p. 1).
To arrive at a fully realized, comprehensive model for policy analysis, rationalistic,
problem-solving models need to be synthesized with more politicized, context-determinant
ones. Current attempts to achieve this involve argumentative, or dialogue-based, models
48


(Dunn, 1990, 1993; Fischer & Forester. 1993). Analysts' use of language is critical to their
success the words used to express policy knowledge must convey the essence of the
underlying analysis. Lowi (1979) found that a significant problem with debate over public
policy is that the available 'languages are inadequate as Majone (1989) noted, policy
analysis is a rhetorical practice. Baumgartner and Jones (1993) and Kingdon (1984) have
found that language plays a key role in agenda-setting by various groups in the policy
process.
Hajer (1993) defines discourse as "an ensemble of ideas, concepts, and categories
through which meaning is given to phenomena (p. 45). Fischer and Forester seek to
synthesize discourse and policy, noting the "struggle over the criteria of social classification,
the boundaries of problem categories, the intersubjective interpretation of common
experiences, the conceptual framing of problems, and the definitions of ideas that guide the
ways people create the shared meanings which motivate them to act (p. 2).
Meltsner (1980. 1986. 1990) notes the importance of communicating with clients and the
extent to which many analysts have neglected this important aspect of analysis. However, he
stops short of evaluating whether analysts use a particular vocabulary that is either beneficial
or detrimental to their success, or whether potential language barriers separate scholars,
practitioners, decision makers, or the public. The language of expertise is a key part of policy
discourse today (Fischer, 1993). However, there is no agreement on a set of norms for
communicating analytical results.
Cobb and Elder (1983) examined the different types of speech that mobilize different
constituencies; easily understood themes are used to mobilize broad groups, while
complicated explanations are used to restrict a policy debate; does this imply that analysts
sometimes use off-putting or inconsistent language to shape policy argumentation?
With respect to regulatory policy, Williams and Matheny (1995) point out that the
phrases employed by regulatory agencies are typically grounded in a managerial language
incorporating a belief that analysis performed by neutral experts can remedy the
inadequacies of participatory democracy; value conflicts are reduced to an objective search
for the best policy. This managerial language dates back to the early development of
49


federal regulatory policy and is embedded in traditional analysis. The authors contend that
problems in social regulation would be better resolved if agencies employed language
embodying ideological frameworks beyond this traditional one. Besides the managerial'
dialogue embedded in traditional policy analysis, the authors also discuss a pluralist
language employed in regulatory debates, which assumes that conflicting interests are the
essence of politics and that they therefore could never be resolved by neutral experts. In that
sense, this pluralist language is less traditional than the managerial one; it assumes that the
public interest is served by creating an open political process allowing organized interests an
opportunity to affect public policy. These same authors discuss a third approach, the
communitarian language of challenge, which advocates an engaged citizenry. To
compensate for the weaknesses of these three means of debating regulatory issues, they have
developed a "dialogic model of social regulation applicable to localized policy debates. It
requires an ongoing democratic dialogue over difficult public problems rather than focus on
discrete decisions or issues.
Public discourse came to the forefront during the Reagan years (Pearce & Weiler, 1992).
Yanow (1996) explains some of the ways in which public policies which never fail to be
ambiguous come to have particular meanings for diverse audiences. Besides the delivery
of messages with certain terms or "languages, she also focuses on factors affecting
interpretation. Rieke and Sillars (1997) develop a complete model for critical appraisal of
argumentation, including the nature of arguments and the use of evidence, values, credibility
for support. These examinations offer insights to the casual observer of argumentation, but
may be difficult to apply in everyday situations without considerable study.
Finally, other scholars are researching the ways in which policy analysts frame issues
and alternatives during policy discourse (Rein & Schon, 1993). Hajer (1993) has studied the
function of discourse coalitions and the institutionalization of this practice in debate over
acid rain regulations. Analysts' use of norms of argument in recent health-care debates have
been evaluated by Jennings (1993).
50


3.1.3 The Search for Identity
Generally speaking, the term policy analysis" does not reflect a particular standard for
conducting or framing a study. Bardach (1996) has developed an eight-step process for
performing such an analysis, while acknowledging that what is required is as much an art as
it is a science. His framework establishes ground rules for defining problems, assembling
evidence, constructing alternatives, projecting outcomes, and making a decision. Although
following this path will not free anyone from the many recognized pitfalls of analysis, it
presents a usable synopsis of what a sound policy analysis might look like. Of particular
interest is the emphasis on audiences, telling the story', and formatting reports, memos, or
even press releases. In a sense, Bardach offers a practical surrogate for an acceptable
analytical standard or process.
Why have so few scholars attempted to set standards for how analysis should be
conducted? The types of questions asked and the individual decisions that are eventually
made can appear to be unique, at least on the surface. How an analyst should go about
researching alternatives, for example, would depend on available resources, delivery
mechanisms for the required information, and the nature of the subject matter involved
this makes it difficult to prescribe how much investigation, modeling, or analysis is enough.
In most cases, private organizations and public institutions lack a policy for how policy
should be analyzed. Rogers (1988) found that national policy for policy analysis funding is
at best fragmented, often ambiguous, and. even in some quarters where it should be
otherwise, nonexistent (p. 148).
Another difficulty is determining just who is an analyst. The status of analysts as bona
fide professionals is a popular and controversial topic, as is the appropriate scope of policy
analysis itself. What is a profession? Generally, investigators have agreed that a profession
is an occupation with particular prestige; it requires esoteric knowledge that is linked to
central needs and values (Larson, 1977; Sharma, 1997).
51


Bobrow and Dryzek (1987) have described analysis as evaluating the attractiveness of
policy options, or canvassing means to an end. This raises the question of whether policy
analysis should be distinguished from other forms of knowledge. Wildavsky (1987) believed
that policy analysis involves
Creating problems that are solvable by specific organizations in a particular arena of
action. A problem in policy analysis, then, cannot exist apart from a proposed
solution, and its solution is part of an organization, a structure of incentives without
which there can be no will to act. (p. 26)
Applying this definition, a great deal of social research on the supply side is set apart
from so-called policy analysis (although, ideally, the functions of a policy elite and an
analyst would both have problem-finding components). However, Mead (1985) sees a false
dichotomy between the use of science and other research tools and conducting a policy
analysis. In some respects, distinguishing between what is or is not a policy analysis is
important for instance, where development of analytical standards is concerned. But in
terms of examining what knowledge is available to support decision making, what matters
most is the quality or usefulness of that knowledge, not the labels applied to it.
The line between who is advising and who is consenting is also blurred. Jennings (1993)
notes that the analyst's knowledge may be more reflective than that of the average citizen,
but it is not qualitatively distinct from everyday repertoire. Weimer (1993) notes that
administrators sometimes act as consumers of advice offered by policy analysis specialists,
and at other times produce analysis themselves. Analysts relying on previous research to
make new contributions are functioning both as knowledge consumers and producers,
severely complicating any attempt to assess analytical impacts. MacRae (1991) describes
this intermediary role: Public policy analysis is a major link in the network of knowledge
use. Without analysts intermediary role, research would be used less for policy choice and
relatively more for pre-decision enlightenment (p. 27).
The relationship of the policy scientist to the world around her provides more
ammunition for criticizing her endeavors. Meltsner (1986) addressed the difficulty of
reconciling political feasibility with policy analysis; some contend that the need to tailor
52


recommendations to bureaucratic or institutional demands is at the heart of policy analysts
failure to develop a recognized profession. The ultimate fate of a particular study might
depend on the bureaucratic position of the analyst who prepared it (or hired an outside expert
to prepare it). These concerns can be extrapolated to analysts working in other institutions.
With respect to development of a professional identity for policy analysts, Torgenson
(1985) cautions:
This identity currently bears the distinct mark of institutional allegiances -
particularly, of involvements with large private corporations and agencies of the
administrative state. These allegiances serve to reinforce the tendencies against
which Lasswell warned toward oligarchy and bureaucratism. Typically, moreover,
the conduct of the policy-analytic professional in these arenas of power is guided by
a technocratic consciousness, generally amenable to these institutions, which is
resistant to that critical insight and moral reflection which Lasswell places at the
heart of a project of contextual orientation, (p. 254)
It seems unlikely that scholars will agree on a single approach to analyzing policy
problems. Even if that did occur, the effect would be only temporary, because subsequent
political developments would recast policy analysis (Fischer, 1993). As deLeon (1988) stated
earlier, the policy sciences are jointly defined by intellectual contributions and political
events,... it is the combination of these elements that provides grist for the successful
analytic mill (p. 95). The key is to recognize the significance of various impairments and
frames of reference when selecting or designing policy tools.
The lack of a clear identity for policy analysts is evident even within EPA. In recent
years, the design, implementation, and administration of environmental programs has been
an important area of study within public policy circles. However, the agencys own
fellowship program for graduate students does not include policy analysis, public policy,
public administration, or political science among the fields of specialization that are
recognized by the program but does specify categories for other social-science areas, such
as risk communication (EPA Office of Research and Development, 1997).
53


3.2 Supply and Demand: b There a Knowledge Utilization Deficit?
Although there is a blurry line between who demands analysis (consents) and who
supplies it (advises), both steps are necessary for the results to have any real impact. If
knowledge is not transferred and utilized, then there must be a weak link somewhere along
the analytical chain.
Expert advice is known to face unique challenges when it must navigate the political
process (MacRae & Whittington, 1997). Uncertainty about analytical and perhaps
organizational objectives are part of the problem. Stillman (1991) and Wamsley (1990)
find that the study of public institutions continues to search for a clear definition of purpose.
As noted by Ventriss (1991), It has become customary for social science disciplines to
engage in periodic soul-searching, but perhaps more than any other field of inquiry, public
administration has done more than its fair share (p. 4). Policy analysis is no exception;
questions about the relevance and/or utilization of analysts efforts and means of
improving them are asked repeatedly. To use a market analogy, analysts should be
motivated to deliver a product that is of high quality, is in demand, is delivered on time, and
is readily available. In today's political environment, the stakes may be high, particularly
where the survival of a nonprofit research institute or a major policy initiative is concerned.
Perceptions of the quality and/or relevance of available policy analysis and associated
social-science knowledge clearly could affect its eventual utilization. Generally speaking,
analysts early on relied heavily on objective criteria, believing that proper use of science and
technology could establish an optimal answer to policy questions (deLeon, 1988; Quade,
1975; Williams & Matheny, 1995). This approach did not go un-criticized, however. Simon
(1945) explored the idea of objective rationality, concluding that, when mere mortals
consider administrative choices, only a few of all possible alternatives ever come to mind.
He also noted that rationality requires complete knowledge and anticipation of the
consequences of each choice: ReaI behavior, even that which is ordinarily thought of as
54


"rational, possesses many elements of disconnectedness not present in this idealized picture
(P- 80).
To assess the impact of science and/or analysis on public policy, many have examined
certain aspects of knowledge use (Caplan. Morrison, & Stambaugh, 1975; Lester, 1993;
Nilsson, 1993). Trying to pinpoint the value or influence of research and knowledge on
particular decision-making processes can be as rewarding as herding cats. But accepted
wisdom suggests that social science knowledge typically enjoys more of an enlightenment
function than direct application in particular policy decisions (Weiss & Bucuvalas, 1977:
Weiss 1980). Lindblom and Cohen (1979) discuss the need for usable knowledge,
arguing that so-called ordinary knowledge" is often more effective than professional
knowledge from social inquiry. One explanation for the apparent underutilization of policy
analysis is that formally trained analysts make up a narrow part of the spectrum. Wildavsky
(1987) observed evaluators trying to become utilizers so that their work would be rejected
less frequently; this led him to wonder if more decision makers should be trained in analysis:
all else failing, the next move may be to train politicians to be analysts (p. xxxi). While
many support the conclusion that policy studies often go untapped, there is at least some
argument to the contrary. Caplan, Morrison, & Stambaugh (1975) concluded that:
Present knowledge utilization theories are often limited by their over-reliance on an
assumed pattern of knowledge use involving hard information . this type of
social science knowledge is used in a vast number of different contexts ... but only
rarely is policy formulation determined by a concrete, point-by-point reliance on
empirically grounded data. Although the impact of soft (i.e., nonempirical)
information on government function is extremely difficult to assess, our data suggest
that there is widespread use of soft information . . (p. 47)
Academic theories have also been shown to influence policy decisions; among the social
sciences, economics is frequently cited (Mead, 1985). The sulfur dioxide (SO,) emissions
credit trading system (which is a component of this study) clearly represents the
incorporation of theoretical explorations into an administrative program.
Jenkins-Smith (1990) found that mobilization of information and other products of
analysis does shape policy outcomes, that the interaction of analysis with interests and
55


ideologies does have an impact. He found that policy analysis tends to be conservative,
inhibiting policy initiatives and reinforcing the status quo. This is consistent with other
research (Aaron, 1978; Wildavsky, 1987), and could partly explain why some observers
believe analysis has little direct influence. The difficulty of assessing the outcomes of policy
analysis leads some to assign it less value. However, Albsek (1995) suggests that:
Scientifically generated knowledge constitutes an important, but on the whole
unquantifiable, part of the enormous store of knowledge which participants in the
politico-administrative decision-making process apply to their practical tasks. To
understand the complex interfaces between social science research and the decision-
making process, it is necessary to be aware that research is transferred to, and
becomes part of, a discourse of action, in the philosophical as well as the everyday
practical sense a discourse in which (self)reflecting participants deliberate on and
debate norms and alternatives with a view to concrete action. This makes the
contribution of science to policymaking both less tangible and potentially more
influential than is usually assumed, (p. 79)
Numerous factors could affect why a particular research result might (or might not) be
translated into useable knowledge, communicated or otherwise made available to an
appropriate audience, and/or eventually applied to a policy issue (Lindblom and Cohen,
1979). To cut to the chase, this problem might be collapsed into two components: First, is the
knowledge being generated of adequate quality, and second, is the audience listening? In
assessing the future of the policy studies. deLeon (1988) considers the supply, or giving of
policy advice, and the demand for these skills, or consent to advice. Possible reasons for
underutilization of analysis include supplying inadequate, ill-timed, or inappropriate
information. Dunn (1993) describes three types of threats to usable knowledge: threats to
cogency, to relevance, and to adequacy: these might include availability of otherwise
valuable knowledge too late in the decision-making process.
To make things better (the purpose of analysis, according to Wildavsky [1987]),
knowledge must be consumed. Investigators have made some headway in identifying factors
that might determine whether policy-related knowledge is actually utilized; however, others
have failed to follow these theoretical constructs with examinations of some possibly
significant determinants. For example, what is the role of presentation style in policy
56


discourse are analyses communicated adequately (Dunn, 1990)? Is the use of terminology
consistent with accepted practice (Hayek. 1968)? Majone (1989) addressed the fundamental
role of language: As social scientists too often forget, public policy is made of language.
Whether in written or oral form, argument is central in all stages of the policy process
(P- D-
The literature on social science utilization grew rapidly after scholars first suggested a
science of knowledge utilization (Havelock. 1969: Lester, 1993). According to Dunn and
Holzner (1988), a primary focus was understanding and improving the utilization of
scientific and professional knowledge in settings of public policy and professional practice.
After examining the transfer of social science knowledge from a variety of perspectives
(Caplan, Morrison, & Stambaugh, 1975: MacRae, 1991), investigators found that social
science research is used more frequently for enlightenment than for problem-solving (Weiss
& Bucuvalas, 1977; Weiss 1980).
Based on more recent utilization research. Dunn and Holzner (1988) have found little
evidence that specific variables can decisively predict knowledge use. The study of
knowledge utilization per se (which involves primarily the social sciences) has slowed
somewhat. Still, we surely do not know all there is to know about how social science
research is diffused and used. Booth (1990) provides examples of methodological and
conceptual difficulties of measuring policy research outcomes, uncovering several
weaknesses of case studies, insider accounts, and impact studies, the three primary methods
of researching research.
MacRae (1991) has found that knowledge use in specific cases can be made easier by
intermediaries, such as the short-term policy analyst. The development of this role, bridging
the gap between researchers and users, changes the questions we must ask in studying
knowledge use (p. 37). Knott and Wildavsky (1980) devised a scale of knowledge
utilization comprised of seven stages. This abstract representation of the various levels at
which knowledge might be incorporated into policy decision making provides an excellent
example of why it is so difficult to measure actual instances of knowledge utilization.
57


Consider their cognition' stage, wherein the policymaker reads, digests, and understands
the studies: how do we know when she understands or acts upon?
Lester and Wilds (1990) have also constructed a conceptual model of knowledge
utilization. Three types of variables contextual, technical, and bureaucratic are
identified as determinants of knowledge utilization by policymakers; these include variables
such as political feasibility, study timing, methodological adequacy, and decision-maker
style. Generally, attempts to pinpoint the determinants of knowledge utilization have not met
with much success.
Carrying the idea of knowledge utilization a step or two further, Sabatier and Jenkins-
Smith (1993) investigated several phenomena related to use of knowledge in policy-making.
These include different factors that affect the influence of analysis, and which may lead to
policy changes (described as policy learning).
The authors have developed the advocacy coalition framework as a model of how long-
term policy learning takes place. Their assessment of several recent deliberations suggests
that the use of information and analysis is most likely in settings where the level of conflict
is moderate and that a professionalized forum appears to enhance the role of analysis.
3.2.1 Schools of Thought: An Interdisciplinary Challenge
Besides potential problems with supplying useful policy knowledge, the demand side
also could provide barriers. For instance, how prepared are knowledge consumers to
interpret analyses addressing the public policy issue at hand does their educational or
professional background determine their predisposition to some or all policy orientations?
How does academic background affect a persons predisposition toward research produced
by individuals working in different disciplines (or, for that matter, representing a different
type of organization or institution)?
58


Leggett contends that curriculum fragmentation even at the secondary level tends to
impede understanding of technology and the environment (1996). Public- and private-sector
scientists and managers involved in air pollution control issues are realizing that continued
policy program success will demand more multidisciplinary management; conferences on
how to achieve a multidisciplinary approach for synthesizing and resolving international
issues are appearing on the horizon (Air & Waste Management Association, 1997).
Scientists pursuing different disciplines have done little to develop a coordinated
foundation for teaching cross-cutting theory or applying multidisciplinary analytical
techniques (Kline, 1995). With respect to regulatory policies, economists develop and apply
calculation techniques, such as cost-benefit analysis, and also examine the justifications for
public intervention into private markets (e.g.. to correct market failure). Attorneys and legal
scholars address administrative, regulatory, analytical, and some economic issues. Scientists
in a variety of disciplines are exploring the many facets of quantitative risk assessment,
which has become an important component of regulatory decision-making processes.
Political science and public policy research also offer clues to understanding the application
of social science knowledge in the regulatory arena. And because many decisions are made,
or strongly influenced, by agencies administering the programs, administrative law and
political behavior significantly affect the ways in which scientific research and economic
studies are employed in regulatory decision-making.
Researchers have made only minimal attempts to evaluate the extent to which academic
background or curriculum fragmentation biases a persons receptiveness toward research.
For example, some quantitative studies simply identify 'level of education as an
independent variable (Lester. 1993). With respect to the regulatory process, level of
education is not necessarily a reliable measure of the ability to produce or apply appropriate
policy analysis most problems involve concepts from several disciplines. Even if we were
able to tie utilization definitively to academic background, it might be more meaningful to
assess the importance of different curricula, such as programs that teach synthesis techniques
(would an economist with three diplomas be more equipped to prepare or interpret an
59


interdisciplinary study than an analyst with only two, but covering more diverse subject
matter?).
Like other areas subjected to policy analysis, the study of regulation is often confined to
separate disciplines. Comprehensive study of regulatory policy should inherently involve
application of findings from several academic areas, such as economics, public
administration, engineering, and law. However, regulatory scholars have not always crossed
academic boundaries to arrive at their conclusions. High (1991) noted a lack of sufficient
cross-discipline teaming and communication, finding that many economists and historians
studying similar, regulatory-oriented subject matter were not talking to each other in
fact important authors did not so much as refer to one another (p. 6). High did find that
economists were generally more guilty of not looking outside their own discipline than were
historians, citing McCraws 1984 study of regulation as an example. This might reveal a
need for better bridging institutions or knowledge networks, as discussed by MacRae (1991).
Some argue that comprehensive policy analysis simply asks too much, and doubt
analysts' ability to possess sufficient knowledge, experience, and sense of perspective to
identify, quantify, and recommend policies that adequately address real-world social
problems. Resource constraints impose limitations that are usually out of the control of the
analyst.
However, a different type of constraint is added by the analysts own intellectual biases
or inclinations. Allison's 1971 study of the Cuban missile crisis shows how theoretical lenses
affect the way the world looks to different disciplinary observers, and that a particular
decision-making process can appear to be rational, political, or something else, depending on
ones perspective. Policy analysis and knowledge orientation are closely allied.
Holzner and Marx (1979) put it succinctly: Any specific inquiry as a deliberate process
to construct and ascertain knowledge remains necessarily embedded in a context that itself is
not the object of the inquiry (p. 99). Bobrow and Dryzek (1987) have also considered that:
The very essence of policy analysis lies in knowledge-based interventions in public
policy making. Theories about the content, production, dissemination, and
interpretation of knowledge may therefore be expected to have implications for the
60


nature, content, and role of the policy field, and for the qualities demanded of the
contributions of frames of reference, (p. 16)
Reaching even beyond knowledge orientation, Dunn (1993) identifies perspectivity and
objectivity as two threats to the adequacy of useable knowledge. Ingram and Schneider
(1990) have investigated the subtle ways in which policy tools implicitly incorporate
behavioral assumptions into their designs. Bobrow and Dryzek (1987) have illustrated the
effects of relying on different frames of reference, such as welfare economics or public
choice. They warn that good policy analysis may depend on understanding different
theoretical lenses and knowledge orientations and then choosing the appropriate one for
each scenario.
Educational disciplines are not necessarily the answer to reducing the distortions caused
by various theoretical or behavioral lenses they may be the unexpected source of the
problem. Lasswell (1971) declared that a common result of professional training is fostering
self-deception rather than self-analysis. Much of the conceptual basis for analysis was
derived originally from theories in public choice and political economy, leading several
authors to call for better incorporation of values into analysis (Stone, 1988; Wildavsky,
1987). Furthermore, policy sciences have typically emphasized quantitative skills, although
many problems do not lend themselves to modeling or calculation (Patton & Sawicki, 1986;
Quade, 1975).
In discussing the social processes through which knowledge is applied to problem
solving, Lindblom (1990) describes various social and intellectual biases or limitations as
impairments. He argues that "not biology but social interactions cause impairments, some
intended, some not. If so, people can reduce them (p. 285). Lindblom also suggests that a
competition of ideas offers some escape from impairment to useful social inquiry. Rein and
Schon (1993) note that controversies cannot be resolved by separating questions of value
from questions of fact, because we integrate facts, values, and theories into the way we
frame problems.
To make matters worse, human behavior offers challenges even more fundamental than
educational background (Tversky. 1993). Gilovich (1991) has examined several human
61


cognitive, motivational, and social determinants that reinforce questionable beliefs.
Kahneman, Slovic, and Tversky (1982) studied humans* tendency to maintain certain beliefs
despite consistent data to the contrary'.
These factors often color how individuals interpret new information, and thus could
affect how certain knowledge is created, transferred, or applied in the policy arena. How
information is interpreted is also related to how it is communicated. Gilovich (1991) has
explored the importance of inadvertent distortion, sharpening, and leveling in the transfer of
secondhand information. In the policy world, this could be especially important, since a great
deal of analysis involves interpreting other investigators' assessment of previous, or
secondhand, research.
The role of the subject population is another important aspect of knowledge utilization.
Lindblom and Cohen (1979) distinguish between serving the needs of officials and of
ordinary citizens, recognizing that the latter are the ones who gain or lose. Boggs (1992)
points out that many models of knowledge transfer frame it as a two-way exchange,
excluding those targeted or benefitted by use of the transferred knowledge: Social
knowledge is not only created and used by people, it is also about people (p. 29).
Environmental regulation is a good example of a scenario where intended policy
beneficiaries (the public) and targets (affected industry groups) actively participate in both
the production and consumption of relevant policy information.
The standing of potential targets and beneficiaries in policy deliberations is also related
to the so-called democratization of the policy sciences (deLeon, 1992). Finally, a dual
decision-maker/analyst role is also a potential determinant of knowledge use. Lester (1993)
discovered that one independent (or perhaps intervening) variable affecting knowledge
utilization is whether the decision maker participated in the development of the policy
analysis. Within a federal agency, a decision maker might be biased toward studies
performed by analysts within his organization.
Recent trends in the policy studies promise some improvement, at least where over-
reliance on welfare economics or number-crunching are concerned. A principal tenet of
62


policy design an approach discussed in much of the current literature calls for
inclusion of more value and goal clarification in analysis (Bobrow and Dryzek, 1987).
Contrast this with the recent emphasis on cost-benefit analysis in regulatory policy: As
Lave (1996) explains, this makes efficiency the goal, not clean air, or whatever the original
goal was. In this instance, embracing a more all-encompassing policy design could introduce
more value and goal clarification into the policy process.
3.2.2 Complexity
Among the many challenges to sound policy analysis, one complexity appears
unavoidable (Dror, 1968; Gramlich. 1981). Scholars have considered this phenomenon in
many contexts, finding that even defining simple or complex depends on perspective
(Simon, 1981; Slobodkin, 1992). Attempts to resolve important policy problems often
involve simplification; this might come in the form of assumptions about human behavior
(e.g.. rationality), narrowed problem definitions, use of abbreviated lists of alternatives, or
restrictions on standing when policy impacts are estimated. Resource constraints are cited as
another reason for analysts inability to fully consider alternatives; bounded rationality is
another (Meltsner, 1986; Simon, 1945). Overall, the policy literature acknowledges the
complexity of social problems, while still searching for simplifying themes. A preference for
identifying causation is currently evident, even if it requires resolution of many complex
variables; consider, for example, development of complicated policy-change models (Radin,
1997; Sabatier & Jenkins-Smith, 1993).
Generally, modem science is grow ing in size and complexity, although natural scientists
tend to believe that focusing on simple systems is a good approach to research (McNamee,
1994; Price, 1986). One way this is achieved is through increasing specialization (Slobodkin,
1992). Over the past 25 years, these trends have characterized the legal and regulatory
arenas. Recent years have seen a massive explosion in the frequency and complexity of the
63


rules governing society (Epstein, 1995; Meltsner, 1990), although a form of simplicity is
achieved through specialization in areas such as the controlling water pollution. The types of
industries affected by federal regulation continues to grow, while scholars focus on
increasingly narrow issues (Noll, 1985).
Adding to complexity, the options available to all policy participants are increasing.
Regulations are being negotiated by stakeholders; the public is increasingly involved in
regulatory decisions, especially on environmental policies (Vig & Kraft, 1994).
Devolution (transferring program responsibility from the federal level to the state or
local governments) poses other problems, such as a lack of analytical resources at those
levels, and reliance on a patchwork quilt of analytical methods across jurisdictions.
Meanwhile, states are establishing their own analytical requirements, such as passing bills
mandating that agency rules be scrutinized using cost-benefit analysis, some even going so
far as to specify that administrators must choose the most "cost-effective alternative.
3.2.3 Bridging Between Disciplines
Communication barriers between researchers and practitioners are another threat to the
utilization of policy analysis. A 1975 National Research Council report describes the need
for feedback (program-specific information flowing back to the decision maker) as well as
"feed-forward information (program information stored in a data base to support future
decisions involving similar issues).
Bunker (1978) points out the need for structures that support two-way communication
between those who furnish "paradigmatic knowledge and those who produce specific or
"situational knowledge. Weimer (1993) contends that few authors have explicitly addressed
policy design discussions "to practitioners who have a great deal to gain from this body of
academic work.... Although this steadily growing stream of literature certainly addresses
fundamental issues in the policy sciences, its generality and level of abstraction make its
64


immediate usefulness to practicing policy analysts obscure (p. 110). He offers a craft of
policy design techniques and strategies in an effort to assist administrators and others who
must perform analyses. This underscores the lack of communication channels between
researchers, analysts, and decision makers.
Citation analysis and co-citation analysis are methods in which investigators examine
communication structures or quantify relationships between separate academic disciplines.
These approaches are also used to measure the impact of research results produced by
individual contributors or to assess the importance of individual journals (Cottrill, 1989;
MacRae, 1996), although not without their critics. MacRoberts and MacRoberts (1996)
depart from the traditional scientific view and contend that citations are unreliable quality
indicators.
However, when these techniques are applied to document formal communication
between diverse groups or separate academic disciplines, the concerns about quality and
research impact are not as problematic. This is consistent with the definition of bibliometrics
offered by Schrader (1981): "The scientific study of recorded discourse. Citation analysis is
also used to assess a librarys collection by evaluating the bibliometric information related to
a landmark article (Soehner. 1992).
Generally, policy theory is investigated by scientists working in various specialties more
than by analysts employing diverse skills. Academic programs typically reinforce these
boundaries: university programs are oriented in terms of disciplines (e.g., economics,
political science), and are more Quonset huts than cozy homes for interdisciplinary, policy-
directed research scholars and training (deLeon. 1988, p. 111). Granted, certain institutions
do provide a means of synthesizing multi-disciplinary pursuits. Some academic journals
address specific niches not bound by one academic discipline MacRae (1987) and
MacRae and Whittington (1997) describe these developments as the formation of informal or
invisible colleges, where groups of scientists focus on broad-based problems without
benefit of a formal institutional structure.
Given these barriers separating researchers working in various disciplines, and the
different needs of scholars and practitioners, one might ask what systems help the analyst
65


uncover the research most pertinent to his problem. Policy-related information is an
important commodity in today's decision-making climate: "Without access to expertise (or
counter-expertise), an interest group today cannot participate effectively in the policy
process (Fischer, 1993, p. 36).
Because they often must look beyond the peer-reviewed literature, policy analysts
depend on a body of knowledge that lacks organization to some extent. MacRae (1991)
discusses communication between researchers and analysts seeking to apply that research,
noting that current processes do not provide a means of knowing which research is used by
analysts. He notes that today?s knowledge use network:
may not give enough incentives for researchers to supply needed knowledge or for
analysts to seek it. .. Both groups may also fail to provide incentives for its
production (i.e., to overcome academic researchers' interests in promoting
disciplinary theory over practical application). An efficient indexing system for
policy-related research might serve as an incentive, if it could help producers of
knowledge be rewarded for its use. (p. 27)
A researcher may find it difficult to determine where her contributions have the greatest
impact on actual policy developments. MacRae (1991) notes the general lack of organization
of various policy knowledge bases; in his examination of some recent analyses of speed
limits, he found that different researchers relied on different studies, with little to no overlap
between the works cited by various investigators. While policy elites work within a process
that uses citology partly to measure the usefulness of their contributions to other elites, there
is no mechanism for tracking citations by analysts such a system could close a feedback
loop by indicating the usefulness of research to practitioners (as Eads [1981] emphasized,
good research is not necessarily useful research). This, in turn, could help establish research
priorities and close some of the theory/practice gap addressed by Weimer (1993).
Think tanks, policy institutes, and other types of nonprofit research centers are
proliferating (Fischer, 1993; Radin, 1997). Ad hoc groups are being formed to investigate
particular policy problems (e.g., the National Environmental Policy Institute [NEPI], 1996).
The mission statements for these organizations can vary widely, but often involve advocating
66


reforms, promoting policy dialogues, or providing research support to the public and private
sector. Whatever their purpose, the function of these organizations often depends on
coordinated actions by scientists, analysts, policy experts, and other professionals with
diverse skills and academic backgrounds; thus, their activities may result in a bridging
function, spanning boundaries in ways that are not encouraged by conventional university,
governmental, and industry organizational structures.
MacRae and Whittington (1997) have explored technical communities that support the
definition of policy problems by both researchers and laymen. They highlight important
differences between traditional scientific communities and those that encourage expertise on
social policy. Among other things, technical communities could perform a quality control
function through peer review of policy designs. Issues to be addressed include the
appropriate balance between general and specific knowledge, and boundaries between the
expertise of a particular community and other available knowledge. Depending on the
circumstances, the sponsor of the community or bridging structure could be a concern (i.e.,
are these communities always depoliticized?).
A bridging institution capable of de-politicizing knowledge transfer or the sharing of
analytical results could make an important contribution to policy discourse. Naturally,
knowledge transferred within policy-related technical communities is not immune to biases
based on professional or institutional affiliations. Albaek (1995) cautions against establishing
inappropriate criteria forjudging the value of scientific contributions: If the mode of
dissemination and application of scientific hypotheses and results are made the criteria of
their validity and benefit, research loses its fundamental justification and is no different from
other interests whose most important function is to kowtow to power (p. 97).
67


3.3 Summary of the Policy Literature
Several key findings from the literature review have guided this research study. First,
Wilson (1985) questioned whether regulatory analysts in the 1970s and early 1980s neglected
to evaluate alternate interventions, understand alternative organizational structures, and then
decide what difference the choices make. For example, in outlining an agenda for attacking
regulatory problems, in 1981 Eads focused significantly more attention on estimating effects
than on any other aspect of the policy process; this author's review of recent doctoral
dissertations also supports Wilsons observation (1985).
Andrews (1994) discussed the reliance on quantitative risk assessment in environmental
regulatory decision-making despite its tendency toward over-simplification and susceptibility
to value judgments. However, developments in policy analysis, administrative procedures, and
theoretical understanding appear to have expanded the scope of regulatory analysis (Breyer,
1993; JasanofT. 1993).
Both High (1991) and MacRae and Whittington (1997) highlight communication gaps
among individuals working in the regulatory arena, revealing a need for more exploration of
roles for bridging institutions encompassing academics and practitioners across disciplines, and
possibly for more organized feedback and communication of policy knowledge among
technical communities; more communication among analysts could improve the chances that
knowledge in this field will continue to be cumulative.
Bobrow and Dryzek (1987) and Allison (1971) demonstrate the influence of different
theoretical frameworks on identification and resolution of policy problems, while Williams and
Matheny (1995) illustrate our inability to synthesize all the aspects of social regulation using
one language. Nevertheless, federal agencies appear to be devoting a substantial portion of
their workday analytical efforts to quantitative risk assessment and cost-benefit analysis.
Finally, much of the recent policy research raises important questions concerning how analysts
use languages of expertise in the framing of policy problems and potential solutions, asking
how this use of rhetoric assists or constrains their efforts, or both (Fischer & Forester, 1993).
68



What implications does this literature have on environmental regulation? It suggests
that:
1. Methods of analysis have multiplied and the policy research process has become more
multidimensional;
2. Analysts have become more inclined to incorporate multiple perspectives into their
work;
. The environmental programs have matured and evolved over the past 25 years;
4. During the 1970s, policy analysts tended to study only one or two aspects of a regulatory
issue, but have since broadened their research scope;
5. Analysis of a particular regulatory issue is not entirely cumulative in a Kuhnsian sense,
in that different researchers sometimes cite few of the same references; and
6. The evidentiary base for assessing environmental issues has changed over the past 25
years due to new policy and regulatory' theories, adoption of cost-benefit and risk-based
criteria by various decision-makers, and experience with implementation of
environmental statutes.
69


4. Methodology
If there is no struggle, there is no progress.
Frederick Douglass
Before discussing research methodology, some working definitions may be helpful.
Discourse Hajer (1993) defines discourse as "an ensemble of ideas, concepts, and
categories through which meaning is given to phenomena (p. 45).
Policy analysis According to Dunn (1981), policy analysis is an applied social
science discipline which uses multiple methods of inquiry and argument to produce and
transform policy-relevant information that may be utilized in political settings to resolve
policy problems (p. 35).
The objective of this investigation is pinpointing how analysts participate in policy
discourse on environmental regulation. Consequently, a discursive process was examined
with an eye toward the forms and the forums employed by policy analysts and other
participants.
4.1 Scope of the Study
To allow in-depth analysis of one area of discourse, the scope was limited to decisions
affecting design or administration of federal air pollution control programs. Argumentation
concerning a series of discrete CAA-related actions during the past 25 years was examined
in detail. Furthermore, one particular aspect of policy implementation permitting
70


programs for stationary sources was evaluated. Since the inception of the federal
environmental programs, this has been an important and controversial topic.
Air quality protection raises complex, national-scale issues, and thus offers substantial
challenges to analysts and others involved across the process. Policy debate over the Title I,
Title IV, and Title V permit programs was chosen because 1) the EPA requirements date
back to the early development of the CAA program, 2) these programs are significant
enough that substantial analysis will have been performed by both proponents and critics of
the agencys actions, and 3) the question of how the federal government should (or should
not) restrict the construction and operation of point sources continues to be an important
topic in todays policy debate. Consequently, discourse over decisions related to EPAs three
permitting programs and Congress underlying legislation provided the data for
analysis. The milestones around which this debate has evolved are the decision points under
consideration.
Note that these decision points did not include EPA determinations of NAAQS.
Although those ambient standards provide the backdrop for the permit program, there are no
associated compliance requirements for industrial facilities. Only state and local
governments must demonstrate compliance with NAAQS. which significantly alters the type
of discourse taking place. On the other hand, the states' primary means of ensuring NAAQS
compliance is by enforcing permitting requirements, which brings the private sector into the
discursive process.
4.2 The Approach
To achieve the intended research objectives, a relatively uncommon approach to data
collection and analysis was required. Among other things, it was necessary to identify
characteristics representative of formal discourse as a whole and also select factors that
might separate out various forms of policy analysis. One possible means of accomplishing
71


this involves examining the quality control mechanism applicable to each discursive form.
MacRae & Whittington (1997) identify several forms of policy-related discourse, and list
them in descending order based on the extent to which they are formally subjected to quality
control (i.e., ranging from academic work to public discourse):
Research reviewed within scientific communities (basic disciplinary research,
underlying methods and principles, and policy-oriented research);
Sharable documentary policy research (or analysis) from technical communities:
Less visible documentary analyses:
Short-term policy analysis: and
Analysts* public argument, (p. 337)
Besides content, form, and presentation style, the references in sample documents were
also examined. Citations can be considered "frozen footprints in the landscape (Cronin,
1981, p. 16). Bibliometrics, the formal analysis of citations, has been described as the
scientific study of recorded discourse (Schrader. 1981, p. 151). Typically, citation and co-
citation analysis are performed to evaluate 1) the influence of a particular author,
publication, or discipline; or 2) the extent to which two authors are cited together to evaluate
the structure of a research specialty or the interdisciplinarity' of a field. In the latter case,
bibliometric information is used to study formal communication structures between two
disciplines and identify to what extent their literatures are related; cluster methods and other
statistical techniques may be used to examine the resulting data set (Cottrill, 1989). In other
cases, professionally prepared indexes are studied to reveal communication or research
patterns (Tijssen, 1992).
Besides establishing a researchers influence or outlining relationships between
academic disciplines, citation analysis offers benefits to decision makers and practitioners.
Gupta finds that bibliometrics can promote discussion between academia and industry;
address whether academic research sheds light on the problems relevant to practitioners; and
evaluate whether practitioners employ research findings to support decision making (Gupta,
1997).
72


For this dissertation, citations were analyzed to establish the degree of overlap among
bibliographic entries across this entire discourse: this addresses a number of questions, such
as: How cumulative is the knowledge in this field? To what extent do authors rely on the
same studies, and how does that behavior vary along disciplinary, institutional, or
professional lines?
Figure 4.1 illustrates the procedure used to document references commonly cited by two
documents included in the content analysis. Naturally, formally recorded communications do
not encompass all conversations" that might result in knowledge dissemination. However,
this approach does create a different snapshot of environmental policy discourse than offered
in other studies, and offers the advantage of not requiring subjective judgments, often listed
as a weakness of social science research. Also, this analysis allowed inclusion of
significantly more raw data points than several other research alternatives, such as the case
study. Finally, the content of each document was analyzed to establish possible patterns in
theoretical framework and uses of empirical data or quantitative methods.
Citing Document
A
Citing Document
B
Bibliographic Information Bibliographic Informacion
Commonly
Cited Item(s)
FIGURE 4.1
Bibliometric Analysis to Identify Common Citations
73


4.2.1 Statement of Hypotheses
Based on the state of knowledge reflected in the current literature, five hypotheses were
developed for testing:
H,,: As air quality control programs have matured, a greater proportion of policy studies
have evaluated alternative types of regulating or otherwise intervening into private-
sector activities.
H,b: As air quality control programs have matured, a greater proportion of policy studies
have assessed alternative govemmental/administrative procedures for implementing
regulatory initiatives.
H2: Compared to the early days of the environmental programs, more regulatory policy
arguments are now based on economic-efficiency, risk-reduction, or other
quantitative decision-making criteria.
H3l: As federal environmental programs have matured, there has been less interdisciplinary
research, as reflected by a trend toward less disciplinary overlap among the references
cited in policy research.
H3b: As the policy research process has evolved, environmental regulatory studies have
relied on a more diverse set of theoretical perspectives.
The first two hypotheses represent evaluation of a broader range of alternative regulatory
interventions and administrative approaches (e.g.. examination of policy tools such as
market-based allowance trading, or establishment of a "science court to make
determinations in regulatory disputes). As stated in H:, the literature suggests that
environmental discourse has become more quantitatively oriented over the past 2S years.
Furthermore, as indicated in H3, environmental analysis is believed to now be less cohesive
in that authors are relying on a larger, more fragmented knowledge base (and thus are citing
a more diverse set of references).
74


4.2.2 Data Collection and Document Review
A comprehensive content analysis was performed to achieve the research objectives. The
intent was not to pinpoint exactly where analysts' contributions were (or were not) utilized in
the decision-making process instead, this research examines the content and framing of
their arguments and identifies how various contributions were introduced or shared during
the discursive process. To take a snapshot of the pattern of discourse on air quality
permitting decisions, a variety of material addressing federal regulation of stationary source
emissions was evaluated.
As noted earlier, the purpose of this dissertation was to apply a unique approach of
examining the arguments within the universe of environmental policy discourse and assess
the types of analysis being contributed. The content and distribution of approximately 400
documents addressing various aspects of the federal CAA permitting issues were identified,
reviewed, and systematically documented. In this study, the unit of analysis was an
individual document addressing one of the specific issues being examined. Data were
collected from academic research and other professional studies in the field of environmental
regulatory policy by reviewing books, journal articles. EPA documents, and white papers.
Items were included in the content analysis regardless of whether or not they were peer-
reviewed. This is consistent with the notion of "discourse." Data were collected from 1) the
primary literature, which consists of EPA-published studies, agency memoranda, and peer-
reviewed journals; and 2) secondary resources, such as books, technical reports, conference
proceedings, white papers, judicial documents, and periodicals published by professional
societies or trade organizations.
In addition, the gray literature" in 25 associated EPA dockets was included. In cases
where a particular docket contained a prohibitive number of documents, a purposive sample
was taken to encompass contributions from a variety of stakeholder groups and representing
steps along the entire rulemaking time line (i.e.. from development of a proposal to
75


publication of final regulations). These resources comprise the universe of material for the
content analysis of written discourse on the policy aspects of federal permitting issues.
Several types of data were required to conduct this analysis. First, testing H,, and Hlb
required content analysis of a variety of studies on environmental policy in order to identify
research scopes and trends. To test H:. the analytical strategies employed by these author-
analysts were evaluated and documented. Third. H3j required tabulation of the works cited in
each document for comparison to works cited in the other documents in the study (a form of
citation analysis). For H3b, the theoretical perspectives(s) reflected in each document were
determined and tabulated.
Additional information was recorded to facilitate analysis of whether the discourse fell
along institutional or professional lines and how the documents were introduced into that
discourse (e.g., in a peer-reviewed journal or government-sponsored study). Also recorded
was whether the author of the document under review referred to any difficulty accessing
information, or in some other way addressed the idea of how EPA (or some other group)
might organize the information associated with the permitting issue at hand.
4.2.3 Database Design
Data were recorded in a relational database using Microsoft Access 95, version 6.
Because this study employed individual documents as the unit of analysis, each item was
assigned a unique code and then scored in a database for a number of characteristics (listed
in Section 4.2.4.2). In addition, the citations included in those documents were tabulated to
support the bibiiometric analysis. Appendix B provides sample data base coding forms and a
list of the queries employed to record and filter these data.
76


4.2.3.1 Citation Analysis
The citation analysis included 116 separate documents: All the works cited in those
materials were tabulated for comparison to the lists of citations in each of the other 116
documents. This process resulted in approximately 6.100 separate data base entries of cited
documents.
The data recording method involved creation of a form/subform combination to relate
documents in the data base to their citations (an example is provided in Appendix C). This
then allowed comparison of the entries in the table of cited documents to look for
interdisciplinary overlap.
4.2.3.2 Content Analysis
All told, approximately 395 separate documents were analyzed during this study; all of
those were coded during the content analysis. Items recorded for each document included:
Date and type of publication:
Academic discipline, profession, and institutional affiliation of primary author;
Mode of publication and intended audience:
Scope (e.g., investigating attractiveness of alternatives, calculation of benefits,
computation of burdens, evaluating institutions or procedures, justifying intervention,
examining political aspects or administrative components);
Inclusion of empirical data and type of data offered (economic, natural science, etc.);
Span of theoretical basis (i.e.. single or multi-disciplinary);
77


Negative/positive (pro or con regulation or neither);
Type(s) of regulation being studied (incentives, command-and-control, etc.);
Quality control mechanism (peer-reviewed?);
Threats invoked (e.g.. unemployment, health risk, injustice);
Primary theoretical dimension (e.g.. welfare economics, political science);
Consideration of values such as efficiency, equality, and equity;
Inclusion of references and/or a bibliography;
Inclusion of avenues for further research; and
Presentation of an easily recognizable synthesis or conclusion.
Data were recorded in a relational database using an online coding form containing a
series of answer lists and yes/no selectors sample forms are provided in Appendix C.
Once the coding was completed, the data were "mined" using a series of queries to look at
patterns and potential relationships.
More than 3S separate queries were designed and created to analyze the data, develop
frequency distributions, perform the citation analysis, and generate other statistical results
(an example query is illustrated in Appendix C. along with a listing of the types of queries
used in the analysis). First, the queries were used to generate cross-tabs and tables; these data
were then imported into Microsoft Excel spreadsheet software to generate the descriptive
statistics and illustrations presented in Chapter 5.
78


4.2.4 Validity and Reliability
To gauge the internal validity of this effort, a sample of the study documents were coded
independently by an outside reviewer based on the criteria developed for the content
analysis. Slightly fewer than 6% of the results conflicted with this authors coding choices.
Not surprisingly, a majority of the differences pertained to identifying the theoretical basis or
values underlying the various contributions. Based on comments received and on subsequent
observations as the content analysis progressed, one of the original measures for empirical
data was eliminated.
To confirm the accuracy of the statistics being reported, the design of the data base
tables, coding forms, and queries was evaluated by another colleague.
79


5. Findings
That which is not good tor the beehive cannot be good for the bees.
Marcus Aurelius
Based on the working definitions of policy analysis and discourse presented earlier,
some form of "analysis" is expected to form a subset of that universe of discourse. Data on
how various contributors break apart issues, use methods of inquiry, frame problems, obtain
and employ empirical data, invoke risks, and format their written communications was
collected in an effort to identify trends in policy dialogue and analysis, risk assessment, and
cost-benefit techniques.
As explained in Chapter 4. data were collected from EPA documents, peer-reviewed
literature, and a variety of other documents. With the exception of reports submitted during
the notice-and-comment process associated with specific rulemakings, few of the documents
refer to the PSD. NSR. Title V. or acid rain permitting programs specifically by name. The
remaining books, reports, and articles addressed one or more issues specifically related to
CAA regulation of stationary' sources through permits or other means these ranged from
highly theoretical constructs to pragmatic explorations of emissions trading, effluent fees,
federalism, performance measures, and technical challenges, among others.
5.1 Key Results
The content and apparent purpose of the discourse covered a wide range of issues using a
variety of presentation styles. Collecting specific information from such diverse sources
80


poses several challenges, but also provides an opportunity to gain insight into current
discursive processes. Key findings are highlighted here, followed by hypothesis testing and
an assessment of the results. The results of the citation analysis are presented first, followed
by a review of the content analysis.
5.1.1 Biblio metrics
For the 116 documents included in the bibliometric analysis, all citations 6,037
individual items were tabulated in a data base. Comparing these entries to identify
common citations revealed that 364, or 6%. were cited by at least one other citing document
in the study. As shown in Table 5.1. the most-cited item was referenced a total of 12 times:
246 were cited twice.
Among other things, these data likely reflect the fact that the core principles of the CAA
programs were studied intensely during the 1970s and early 1980s. EPA's efforts have since
become more decentralized, with the inception of the Title V and acid rain programs in 1990
one would expect analysts to be writing about a much wider variety of topics and across
more lines of discourse.
Overall. 72% of contributors to this policy discourse cited at least one reference, while a
few included more than 100 citations academic researchers/professors received a perfect
score of 100% in this area, followed by representatives of think tanks and nonprofit
organizations, at 83%. Authors employed in federal agencies cited references 74% of the
time, while industry representatives did so 62% of the time.
81


TABLE 5.1 Most Commonly Cited Documents*
number times cited number of docs description
12 1 Air Pollution and Human Health (Lave, 1977) 1
10 1 Pollution. Prices and Public Policy (Kneese, 1975)
7 4 Air Pollution and Human Health (Science, Lave, 1970)1 Pollution. Property and Prices: An Essay in Policy (Dales. 1968) The Economic Damages of Air Pollution (EPA, 1974) To Breathe Clean Air (National Commission on Air Quality, 1981)
6 6 Clean Air: The Policies and Politics (Jones. 1975) Clean Coal/Dirty Air (Ackerman. 1981) Promise and Performance (Marcus. 1980) The Occupational Health and Safety Act (Smith, 1976) Air Pollution and Respiratory Infection {Brit. J. Prev. Soc. Med.. Douglas, 1966) Environmental Regulation and U.S. Economic Growth {Rand Journal of Economics, Jorgenson. 1990)
5 14 Listed in Appendix B, Table B.3.
4 26 Listed in Appendix B. Table B.3.
66 Listed in Appendix B, Table B.3.
2 246 Listed in Appendix B, Table B.3.
*In addition, sections of various versions of the Clean Air Act, Code of Federal Regulations, and the Federal Register were cited on approximately 150 occasions.
itself;
'Lave has produced insightful environmental analysis, plus valuable assessments of the analytical process
Apparently, consensus supports this authors conclusion that he is one of the chief contributors to this field.
82


The extent to which the authors of these documents cited common references was also
examined according to date of publication. As shown in Figure 5.1, the sample documents
published later in the life of the CAA permitting programs shared common references to a
slightly greater degree. Note that the distribution of the 116 documents according to year of
publication was as follows: 23 were published between 1973 and 1979, 39 were released
from 1980-85. 26 were published during 198691. and the remaining 28 were dated between
1992 and 1997.
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
FIGURE 5.1
Documents With Common Citations
Collecting these data and performing the content analysis highlighted the lack of
consistency in how various authors document references and present related evidence. One
formatting quirk made the bibliometric study particularly challenging: Not all authors
provide a consistently formatted bibliography. For example, many law review articles simply
provide references in the supplementary discussion contained in footnotes. Generally
83


speaking, the participants in this discourse did not follow a basic set of ground rules for
referencing source material or formatting their sources.
These results also raise more important methodological questions that are particularly
relevant in this age of information overload. How much research is enough? How does an
investigator determine how far to cast her net? When "must a particular data base be
included for a research effort to be considered complete?
5.1.2 Content Analysis
The second study phase entailed content analysis of 395 sample documents. For each
item reviewed, a variety' of characteristics were evaluated and documented. While several of
these characteristics pertain directly to the arguments being made, others were intended to
pinpoint who was making the arguments, and where.
For example, where does this discourse reside? What are its spatial dimensions or
characteristics, if any? Who makes up the air pollution control community? Shapiro and
Neubauer agree that these questions are related to the content of the discourse: "The meaning
and value that statements confer are inseparable from the mapping of persons within which
the statements are deposited. (1989. p. 303)
Overall results are described below, followed by a review of the data according to
institutional affiliation or other criteria and a discussion of how various arguments were
presented. Hypothesis testing and an assessment of the data then follow.
84


5.1.2.1 Overall Results
Of the items reviewed. 30% reside in one or more EPA rulemaking dockets: the
remaining documents were obtained from other sources.
References. At least one reference document was cited by 72% of the authors, although a
bibliography (i.e.. a formal listing of references) was provided only 34% of the time.
Conclusions and recommendations: Approximately 80% of the sample documents
included a concise summary of the points being presented. Although 68% of the authors
made a policy recommendation or identified at least one action item for EPA or another
organization, only 6% included areas for further investigation or research.
Empirical data: Analysis revealed that 71% of authors provided at least minimal
information specific to the CAA or to one or more permitting programs (e.g.. references
to specific EPA rules). However, economic data was included only 26% of the time.
Slightly more than 30% of the documents presented some type of "hard science" data
(e.g.. emission rates or volumes, pollutant concentration trends, or health risk factors).
Quality control: Based on the content analysis, it was not feasible to categorize the
documents based on the types of quality control mechanisms identified by MacRae and
Whittington (1997) cited earlier; however, it was evident that some documents were more
oriented toward what they call short-term" analysis and others were more in keeping
with products of traditional research communities. The articles appearing in research
journals and the books published directly by universities or research institutes are likely
to have been through a formal peer-review process. In many other cases, it was not
possible to determine what type of review had been conducted.
85


Quantitative decision-support methods: Cost-benefit analysis techniques were
employed or somehow addressed in 21% of the sample documents: various aspects of
risk assessment calculations or comparisons were discussed by 10% of the authors.
Values: Nearly two-thirds of the study documents noted (either explicitly or implicitly)
efficiency and/or effectiveness as a policy program value. Equity and equality were
considered 45% and 18% of the time, respectively. (Note, however, that these
determinations were particularly challenging, as discussed further in Section 5.2.2.
Not surprisingly, the orientation of most of the discussion centered around economics,
public administration and policy, environmental regulations, and law. as shown in Figure 5.2
below.
g Economic Issues
g Public Policy/ Analysis/ Ethics
4% 2%
Environmental Regulation
7%
Law, Admin Law, or Court System
g Natural Science
g Engineering
g Political Science/Political Economy
Other
17%
FIGURE 5.2
Document Orientation
86


As the authors presented their arguments or described their analytical results, a variety of
threats were invoked. The frequencies at which individual concepts appeared were as follows
(note the numbers total more than 100%, because individual authors often identified multiple
threats):
Economic burdens on the regulated community: 60%:
Need for environmental protection: 52%;
Injustice: 34%:
Costs to government agencies and/or taxpayers: 31%:
Risks to human health: 25%:
Sound science: 22%:
Political unacceptability: 16%: and
Unemployment: 6%.
Many contributors were unclear as to precisely which audience(s) they were addressing
(e.g.. Congress. EPA. etc). To the extent these determinations were possible, the data are as
shown in Table 5.2. Not surprisingly. EPA was the chief recipient of this policy information,
since much of the data pertained to grass-roots regulation (i.e., development of permitting
programs).
Typically, formal environmental policy discourse materializes in documents distributed
within five types of distribution channels, as shown in Table 5.3. This is consistent with the
notion that the public policy field is both book- and journal-oriented with respect to the
publication patterns of academic researchers.
Determining the extent to which these documents were subject to some type of peer
review process was not always clear-cut. Contributions fitting the traditional pattern of
academic publishing included documents appearing in a collection of articles or symposium
volume (12%), articles published in a peer-reviewed research journal (7%), and books
published by a university press (3%). Commercial publishers released 8% of the sample
documents, while think tanks or other nonprofit institutions published 14%. Various
87


government-sponsored documents made up 30% of the samples: the extent of any potential
peer-review process was undetermined for the remaining items.
TABLE 5.2 Audience Addressed by Authors
Audience % of documents
Government agency, including EPA 38
Congress/state legislatures 28
Academics, think tank researchers, and environmental research community 23
Private sector 8
Students (i.e.. text) 1
General public 1
Executive branch (federal) 0.5
Note: Total does not add to 100 due to rounding.
TABLE 5.3 Discourse Channels
Document distribution % of documents reviewed
Freely distributed material (e.g., reports submitted to dockets) 32
Commercial book offered for sale 26
Study or report sponsored by government agency 23
Private sector or nonprofit institute study or report 11
Subscription-based material (e.g.. journal) 8
88


5.1.2.2 Contributor Characteristics
To look for patterns among these results, many of the data were examined according to
the authors' institutional affiliations. Some fundamental characteristics were as follows:
Who are the contributors? The breakdown of authors by institutional affiliation is
shown in Figure 5.3. A cross-section of organizations participated in this discourse.
g Federal government
B Think tank or nonprofit
Unirersity
Trade association
q Private industry
g State, local govt
g Private individual
Other/not determined
FIGURE 5.3
Authors Institutional Affiliation
Where and what do they contribute? Table 5.4 identifies the document distribution
mechanisms employed for each institutional affiliation. Federal agencies sponsored a
significant percentage of the reports addressing the three permitting programs. Note that
much of the information identified as freely distributed material' consisted of white
papers, letters, or reports submitted to rulemaking dockets.
89