Citation
Assessment of space shuttle program cost estimating methods

Material Information

Title:
Assessment of space shuttle program cost estimating methods
Creator:
Mandell, Humboldt Casad
Place of Publication:
Denver, Colo.
Publisher:
University of Colorado Denver
Publication Date:
Language:
English
Physical Description:
xvii, 433 leaves : charts, forms ; 29 cm

Thesis/Dissertation Information

Degree:
Doctorate ( Doctor of Public Administration)
Degree Grantor:
University of Colorado Denver
Degree Divisions:
School of Public Affairs, CU Denver
Degree Disciplines:
Public Administration
Committee Chair:
Hyde, Albert C.
Committee Members:
Shafritz, Jay M.
Dubnick, Edwin G.

Subjects

Subjects / Keywords:
Space shuttles -- Costs ( lcsh )
Space shuttles -- Costs ( fast )
Genre:
bibliography ( marcgt )
theses ( marcgt )
non-fiction ( marcgt )

Notes

Bibliography:
Includes bibliographical references.
General Note:
Submitted in partial fulfillment of the requirements for the degree, Doctor of Public Administration, Graduate School of Public Affairs.
General Note:
School of Public Affairs
Statement of Responsibility:
by Humboldt Casad Mandell, Jr.

Record Information

Source Institution:
|University of Colorado Denver
Holding Location:
Auraria Library
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
14080426 ( OCLC )
ocm14080426
Classification:
LD1190.P86 1983d .M36 ( lcc )

Full Text
ASSESSMENT OF SPACE SHUTTLE PROGRAM
COST ESTIMATING METHODS
by
Humboldt Casad Mandell, Jr.
^
B.S., Hie University of Texas, 1957
M.S.E., Southern Methodist University, 1965
A thesis submitted to the
Faculty of the Graduate School of Public Affairs of the
University of Colorado in partial fulfillment
of the requirements for the degree of
Doctor of Public Administration
Graduate School of Public Affairs
1983


APPROVAL PAGE
This thesis for the Doctor of Public Administration
degree by
Humboldt Casad Mandell, Jr.
has been approved for the
Graduate School
of Public Affairs
by
Date August, 1983


Mandell, Humboldt Casad, Jr. (D.P.A., Public Administration)
Assessment of Space Shuttle Program Cost Estimating Methods
Thesis directed by Professor Albert C. Hyde.
On November 11, 1982, the Space Shuttle "Columbia" flew for the
fifth time, marking the official end of the design, development,
test, and evaluation (DDT&E) phase of the Space Shuttle Program, an
appropriate time to perform an assessment of the analytical methods
employed to manage the program. Of particular interest were the cost
estimating methods employed, which were evaluated on the basis of
accuracy, influence on the Space Shuttle design, utility to
management, influence on the external program environment, worth,
future potential, and state of the art.
This research has concluded that Space Shuttle program cost
estimates produced were realistic, accurate, and provided the detail
needed by vehicle preliminary designers. Corrected actual program
DDT&E costs ranged from $5,158 billion to $5,465 billion (1971
dollars), as compared to estimates made by these methods of $5,200
billion. Opinions of senior managers of the program led to the
conclusion that the accurate cost estimates were the second most
influential factor in producing the favorable cost outcome (second
only to actions of the program manager). However, the value of
accurate cost estimates in selling the program was found to be


iv
inversely correlated with program level of the respondent: managers
from higher program levels expressed the opinion that accurate cost
estimates could actually be a detriment to selling new programs.
From a survey of 116 professional cost estimators it was found
that, since 1972, problem areas associated with cost estimating have
changed substantially: methodological problems have become less
acute; data problems have remained consistently major, and problems
of improper customer/contractor relationships have increased as
budgetary constraints have become more severe. In comparison to the
industry state of the art, Space Shuttle cost estimating methods were
representative of the best in 1972, but lack some features of the
better current methods.
Hie form and content of this abstract are approved. I recommend its
publication.
Signed
Faculty Member in Charge of Thesis


The incentives to estimate low are much
greater than the penalties, if indeed
there are any penalties.
David Novick, June 1962


ACKNOWLEDGEMENTS
It would be almost impossible to acknowledge everyone who
made a positive contribution to this research without adding much
more to the already voluminous contents of this work. However,
there are two categories of people who must be recognized: those
who participated, and those who provided direct support and
logistics. Working in the National Aeronautics and Space
Administration has always been enormous fun, partly because of the
spacecraft we have flown; but perhaps even more important than the
spacecraft has been the continual association with the enormously
talented people who work with us. This association has as a side
effect the development(of an appreciation for talent. Early in
this doctoral program, it became clearly evident that NASA had no
monopoly on genius. It was the genius of the University of
Colorado faculty and the quality of the instructional program which
kept most of us interested enough to struggle through four years of
full time education combined with full time jobs, to reach the
point of being able to begin this research. In particular,
Professors R. Wayne Boss, Philip M. Burgess, and Floyd Mann opened
new universes for us to explore. Phil Burgess deserves the credit
for starting this research on the right path. Jay Shafritz
deserves the credit for gluing together the parts of the program


VI1
and insistently guiding us through. And Albert C. Hyde, visiting
professor from the University of Houston at Clear Lake City,
deserves all the credit for salvaging a very rough research project
and turning it into a work of some significance. Al Hyde, the
third of my superbly talented advisors, clearly made the difference
between the success and failure of this effort.
People who supported the effort are numerous. Perhaps my
NASA leaders deserve the most credit, for covering up my omissions
while I struggled with this research; James Bone and Daniel Nebrig
deserve special mention. The other members of the NASA DPA class
provided help, support, and guidance, and the highest levels of
peer pressure to succeed in this endeavor. The support received
from the Denver campus, from faculty, staff, and students reduced
the formidable problem of distance to a manageable nuisance, at
worst, and expanded our sphere of close friendships to include
people from another culture. Colorado-based students such as Carol
Betson provided the evidence that NASA had no monopoly on bright,
motivated students, and helped to set the standards for interstate
cooperation and for academic excellence which we all attempted to
meet. Those NASA managers who gave freely of their time to respond
to the survey instrument, from former Administrator Fletcher on
down, insured the usefulness of this project. And Keith Burbridge
and my colleagues on the Space Systems Cost Analysis Working Group
and within the membership of the International Society of
Parametric Analysts provided enormous support by distributing my
second survey instrument to their entire memberships.


Vlll
Rosie Hernandez, Francis Smith, and Cheryl Min, the
tireless and talented typists who produced this document to high
professional standards, deserve special thanks. And finally, and
most important, Ginny Mandell and the other NASA wives who did
without husbands for four years must be recognized as the source of
the strength which produced these results.


CONTENTS
CHAPTER
1. HISTORY OF SPACE SHUTTLE PROGRAM COST ESTIMATING ..... 1
Subject of the Research ............................ 3
Purpose of the Research ............................ 9
A Summary History of the Cost Estimating Methods
Employed for the Space Shuttle Program............ 12
Hie Program Management Environment.................... 19
Hie Political Environment........................ 21
Hie Cultural and Techno-Cultural Environment... 29
Hie NASA Planning Environment................... 36
Hie Budget Environment........................... 39
The Design Evolution ................................. 47
Description of Space Shuttle Cost Estimating
Methods ............................................ 51
Definition of Terms in Cost Estimating......... 53
Principles and Processes of Parametric
Estimation................................ 55
Budget Simulation: Cost/Time Relationships ... 60
Preparation for Developing the Methods .......... 65
Space Shuttle Cost Estimating Methods Used
to Establish the Agency Commitment .............. 71
NASA Methods in 1982 ................................. 74


X
Overview of the Assessment ......................... 75
History of Space Shuttle Program Cost
Estimating....................................... 75
Literature Analysis ............................. 77
Evaluation Methodology .......................... 79
Results of the Evaluation ...................... 80
Summary of Findings, Conclusions, and
Recommendations ................................. 81
Appendices ...................................... 82
Glossary ........................................ 83
End Motes .......................................... 84
CHAPTER
2. THE LITERATURE OF SPACE PROGRAM COST ESTIMATING....... 86
Status of the Methodology in 1972 (1950 1972) .... 88
General Literature: Theories, Principles,
Process Descriptions, and Problem Areas ........ 89
The Processes of Cost Estimating
(1950 1972)................................... 93
General Problem Areas (1950 1972) ............ 97
Summary of the Principles, Theories, and
Problems of Cost Estimating (1972)........ 103
The Literature of Cost Growth..................... 104
Literature on Data Analysis, Data Banks, Data
Bases (1950-1972)................................. 107
Cost Estimating Methods for Aerospace Vehicles. 110
Cost/Design Syntheses Methods (1950-1972)....... 116
Annual Cost Prediction and Budget Simulation
Methods (1950-1972)............................. 118


XX
Summary of the Status of Methods in 1972......... 119
Improvement in the State of the Art of Methodology
from 1972 to 1982................................... 121
General Literature: Theory, Principles,
Process Descriptions, and Problem Areas....... 121
Improvements in Data Banks and Data
Availability..................................... 128 '
Improvements in Cost Estimating Methods.......... 129
Improvements in Cost/Design Synthesis Methods.. 133
Improvement in Annual Cost Prediction and
Budget Simulation Methods (1972 to 1982)......... 135
Summary of the Status of Methods in 1982......... 136
Summary of the Literature Analysis ................... 138
End Nbtes............................................. 139
CHAPTER
3. METHODOLOGY............................................. 140
Method and Approach................................... 141
Accomplishment of Originally Intended
Purposes....................................... 143
Management Opinions ............................. 144
Comparison with the General Industry
State of the Art................................. 144
Analysis Approach ............................... 146
Literature Search .................................... 148
Instrument Design and Test............................ 151
Space Shuttle Program Management Survey
Instrument................................... 153
Professional Cost Analysts Instrument............ 162
Respondent Profile ............................ 167


Xll
Analysis of the Instruments.......................... 170
Analysis of Instrument 1 (Program Management
Instrument)................................... 172
Analysis of Instrument 2 (Professional Cost
Analyst Instrument)........................... 172
Analysis of Program Actual Cost....................... 174
Summary of the Methodology ........................... 177
End Notes ........................................... 178
CHAPTER
4. RESULTS OF THE EVALUATION............................... 179
Results of the Literature Analysis.................... 180
Evaluation By Comparison With the Actual Cost
Outcome............................................... 184
Space Shuttle Program Actual Costs.............. 185
Comparisons of Estimated and Actual Costs ...... 187
Comparisons With Cost Growths from Other
Programs ....................................... 193
Summary of the Comparisons ..................... 197
Results of the Analysis of Instrument 1 (Program
Management)......................................... 197
Response Profile ............................... 197
Evaluation of Space Shuttle Cost
Estimating Methods ............................. 198
Summative Evaluation of Management Instrument.. 218
Other Management Responses ..................... 220
Variations in Responses by Program Level ....... 223
Factor Analysis ................................ 229


Xlll
Results of the Analysis of Instrument 2 (Professional
Cost Analysts) ..................................... 229
'ftie Major Problem Areas ...................... 230
Comparison With Baker' s Results .............. 237
Correlation With Organizational Type of
Respondents .................................... 256
Summary of Instrument 2 Results ................. 261
Significance of the Evaluation of NASA Methods ..... 263
Significance of the Results to Assesment
of Space Shuttle Cost Estimating Methods .... 270
End Notes............................................ 275
CHAPTER
5. SUMMARY OF FINDINGS................................... 277
Accomplishment of Originally Intended Purpose ........ 277
Assessment of Effectiveness .......................... 280
Cost Effectiveness (Worth) .......................... 282
State of the Art ..................................... 283
Assessment of Future Potential ................... 288
Effects of Management Level on Findings .............. 289
Summary............................................... 290
End Note ............................................. 293
CHAPTER
6. CONCLUSIONS AND RECOMMENDATIONS FOR FURTHER STUDY .... 294
Conclusions Relating to Effectiveness and Worth of
the Methods....................................... 298
Conclusions Related to the State of the Art ........ 300
Recommendations for Future Study and Action ........ 302
Methodological Improvements.............'...... 303


XIV
Improvements in Application of Methods ......... 305
Improvements in Study Methodology................ 306
Suggestions for Future Research ...................... 308
Summary............................................. 310
Major Conclusions ............................... 311
Conclusions Relating to Effectiveness and
Worth of the Methods........................... 312
Conclusions Related to the State of the Art ... 312
Recommendations ................................. 313
REFERENCES .................................................. 315
GLOSSARY..................................................... 329
APPENDIX
A. . SELECTED BIBLIOGRAPHY............................... 333
B. THE INSTRUMENTS .................................... 343
C. FREQUENCY RESULTS................................. 372
D. FACTOR ANALYSIS.................................... 396
E. INDUSTRY INTERVIEWS ................................ 425
F. SPACE SHUTTLE ACTUAL COSTS ......................... 431


FIGURES
Figure
1-1 Life cycle program management ........................
1-2 Conceptual/preliminary design phase...................
1-3 Chronological trend of government program
cost growth ....................................
1-4 History of Space Shuttle cost estimating method
development ....................................
1-5 Space Shuttle program history: Early planning
milestones .....................................
1-6 Space Task Group program recommendation ............
1-7 NASA budget history ................................
1-8 NASA budget projections ............................
1-9 NASA planning budget chronology, early years of the
Space Shuttle Program ..................................
1-10 Space Shuttle Program history .....................
1-11 Funding guideline effect on system concepts ......
1-12 Development cost scaling ..........................
1-13 Steps in parametric estimating ....................
1-14 Apollo CSM R&D POP trends .........................
1-15 Space Station budget simulation logic .............
1-16 Studies performed since 1962 by JSC ...............
1-17 Industry interview program (1971) .................


xv i
1- 18 NASA/industry program discussions ..................... 70
2- 1 Chronology of aerospace cost estimating methods .......... 112
3- 1 Method summary............................................ 142
3-2 Study method and approach ................................ 147
3-3 Abstracts utilized ....................................... 150
3-4 Design of management instrument ......................... 158
3-5 Instrument 1 (Program management) profile of
respondents ........................................... 161
3-6 Baker's results .......................................... 164
3-7 Instrument 2 (professional analysts) profile
of respondents by grade level ...................... 168
3- 8 Instrument 2 (professional analysts) profile
of respondents by organization type ................. 169
4- 1 Results of the literature analysis........................ 182
4-2 Space Shuttle Actual DDT&E costs ........................ 186
4-3 Space Shuttle DDT&E Cost Actuals ......................... 188
4-4 Comparison of Space Shuttle commitment estimates
vs. actuals ........................................... 196
4-5 Effectiveness of cost estimating methods ................ 204
4-6 Comparative management opinions of methods................ 219
4-7 Corellation of results by program level ................. 225
4-8 Professional cost analyst instrumant, state of
the art as reflected by seriousness of
problem areas ................................................. 231
4-9 Instrument 2: index to problem and group numbers ......... 232
4-10 Rank order means of responses ....................... 233
4-11 Instrument 2 (professional analysts), relative
seriousness of problems as ranked ................... 236


XV 11
4-12 Baker's "Post-hoc classification of responses
regarding most important problems of cost
estimating and analysis" ..................................... 240
4-13 Professional cost analysts instrument, comparison
of resultant problem rankings....................... 241
4-14 Correlation of rankings (Baker vs. Instrument 2) ....... 243
4-15 Ranking comparisons ..................................... 245
4-16 Comparison of problem rankings ......................... 246
4-17 Comparison of Baker and Instrument 2 results ........... 248
4-18 Professional cost analyst instrument factor analysis .. 250
4-19 Factor analysis tests of independence of problem
groups .............................................. 252
4-20 Major problem summary by organization type.............. 257
4-21 Summary of rankings by organization type ................ 258
4-22 Correlation of group rankings with aggregate
ranking ............................................. 259
4-23A Top 3 problems ....................................... 264
4-23B Top 3 problems (cont.) ......................*......... 265
4-24A Major non-data problems ................................ 266
4-24B Major non-data problems (cont.) ...................... 267
4-25 NASA role in cost estimating .......................... 269


CHAPTER 1
HISTORY OF SPACE SHUTTLE COST ESTIMATING
On November 11, 1982, the Space Shuttle "Columbia" flew for
the fifth time, marking the official end of the design, development,
test, and evaluation (DDT&E) phase of the Space Shuttle Program.
Earlier, on September 30, 1982, the funding for DDT&E was completed.
Thus, it is now an appropriate time to evaluate the program (End Note
1) in a comprehensive manner. In fiscal year 1971, NASA estimated
that the costs of the Space Shuttle DDT&E phase would be $5.15
billion (in 1971 dollars). This estimate was later revised, by
agreement with the Office of Management and Budget, to $5.20 billion,
as will be described in detail (see U.S. House of Representatives,
1978a, pp. 1-7). As of September 30, 1982, the actual expenditures
were approximately $5.46 billion (1971 dollars). On the surface,
then, it would appear that the initial estimate was successful. This
research will examine that estimate, but more significantly, will ex-
amine the methodology employed to produce the estimate, as a means to
determine its value to both NASA and to the science of cost
estimating in general.
In the relatively short history of the National Aeronautics
and Space Administration (NASA), a quarter century in 1983, some
spectacular successes have been recorded and chronicled (e.g.,


2
Brooks, Greenwood, and Swenson, 1979; Bilstein, 1980; Newell, 1980).
These authors attribute the successes of NASA to a synergistic combi-
nation of technological skill, imagination, and business management
expertise. Much has been written on the technological features of
the space vehicles themselves, but very little scholarly work has
been performed to document the effectiveness of the various
technologies associated with successful program management and
administration.
The spectrum of these management methods is extensive, both
in breadth and depth. In breadth, they cover the entire life cycle
of a program. In depth, they involve the most minute details, rang-
ing from performance measurement systems controlling thousands of de-
tailed milestones; to configuration management systems controlling
the introduction of tens of thousands of technical changes into all
levels of the program; to a computerized master-measurements data
base controlling all critical dimensions of the space vehicle; to de-
tailed budgeting systems; to high-level program assessment methods
based on detailed project simulations; to runout cost prediction
simulations; etc. Although no one has ever catalogued the entire set
of methods employed, many are described in Johnson Space Center 07700
series documents, Volumes 1 through 16 (issued by the Space Shuttle
Program Manager's Office).
Within this spectrum of techniques there are few which have
been of as much interest or have caused as much comment as those
employed during the planning phases of the program (loosely defined
as the conceptual definition and preliminary design phases). In


3
particular, because of Congressional concerns associated with the
agency budget growth of fiscal years 1977 through 1981 (End Note 2),
there has been nationwide publicity of the initial cost estimates
made for the program. Much of it has been erroneous (e.g., Walters,
1982). Ibis research will document the methods employed to estimate
program costs, and will measure their effectiveness in terms of quan-
titative measures (such as actual costs versus original estimates),
and also in terms of subjective assessments by users and program
managers (e.g., their value as aids to problem avoidance, aids to
management decision making).
Figure 1-1 illustrates the context of the major methods
employed in the "Life-Cycle" Management of the program, and high-
lights those methods to be described and evaluated by this research.
Figure 1-2 presents a topical outline of the methods involved. Of
the four methods shown, cost estimating will receive the major
emphasis, because it, above the others, is the basis for the success
or failure of the entire program planning process.
Subject of the Research
The subject of this research is the cost estimating methods
used in planning the Space Shuttle program. The research will ex-
plore the subject of accurate cost estimates for high technology
programs, principally to perform an evaluation of the mechanical
processes of how estimates are made. Equally important, the subject
of how costs may be controlled to make the predictions come true will
be analyzed, although not treated in detail.


MANAGEMENT PROGRAM
METHODS EMPLOYED PHASE
FIGURE 1-1
LIFE CYCLE PROGRAM MANAGEMENT METHODS
PROGRAM DEFINITION & CONCEPTUAL PRELIMINARY DESIGN A DESIGN A DEVELOPMENT HARDWARE DESIGN A BUILD AND TEST A PRODUCTION A OPERATIONS
A 1 B | C D
COST ESTIMATING DESIGN/COST TRADES SCHEDULE ESTIMATING OPERATIONS COST ANALYSIS DESIGN/COST TRADES PRODUCTION COST ANALYSIS CHANGE COST ESTIMATING COST GROWTH ASSESSMENT SCHEDULE/COST/RISK ASSESSMENT PERFORMANCE MEASUREMENT COST GROWTH ESTIMATION SCHEDULE GROWTH ESTIMATION

ft
RESEARCH
SUBJECT
*>


FIGURE 1-2. METHODS EMPLOYED FOR CONCEPTUAL,
PRELIMINARY DESIGN, AND PROGRAM DEFINITION PHASES
COST ESTIMATING
ANALOGY DEVELOPMENT
DATA BASE DEVELOPMENT
WORK BREAKDOWN STRUCTURE DEFINITION
TECHNICAL DEFINITION
DESIGN COST TRADES
RESEARCH EMPHASIS
IDENTIFICATION OF COST SENSITIVITIES
TECHNICAL SIMULATION
COST/DESIGN SIMULATION
COST OPTIMIZATION OF DESIGN (DESIGN TO COST, DESIGN TO PERFORMANCE)
DESIGN TO ANNUAL COST CONSTRAINTS
SCHEDULE ESTIMATING
ANALOGY AND DATA BASE DEVELOPMENT
PROGRAM ANALYSIS
ESTIMATING METHODOLOGY DEVELOPMENT
BUDGETARY PREDICTION
DEVELOPMENT OF COST/SCHEDULE RELATIONSHIPS (GOING IN, LOOKING BACK)
ANALYSIS OF ANNUAL EXPECTATIONS (POLITICAL ENVIRONMENT)
ANALYSIS OF COST RISK, POTENTIAL COST/SCHEDULE GROWTH
SHAPING OF THE ESTIMATES TO EXPECTED REQUIREMENTS
DEVELOPMENT OF BUDGETING STRATEGIES FOR ALL PROGRAM LEVELS


6
Seldom recognized, except by those integrally involved, is
that the cost estimation process is, strictly speaking, prediction of
the future. For NASA, the future is influenced by innumerable
probabilistic variables, such as unknown and unknowable technological
factors, accidents, test failures, equipment failures, and human
failures, to name a few. The higher the technology involved (or more
precisely, the higher the degree of technological advancement
involved) the greater will be the uncertainty in the estimates (see
Marshall & Meckling, 1959). The probability of being able to predict
a future cost exactly is virtually zero (see Quade 1964, Chap. 15).
Therefore, given that exactness is inpossible, the next ques-
tion which must be asked involves the degree of accuracy desired,
i.e., "What purpose is the cost estimate to serve?" Some suggested
possibilities are:
1. To make economic benefit comparisons among candidates
(Hitch & McKean 1961);
2. To provide direct feedback to the design process (see, for
example, Carlson, 1980; Kelsey and Sitney, 1972; as well as
all references in Appendix A under the heading of "Cost/Design
Synthesis.");
3. To provide data for advanced planning at all levels up to
and including the Congressional and Presidential budgets (see,
for exanple Novick, 1965a; Novick 1965b);
4. To provide a basis for national policy formulation and
optimization (McKean 1958);
5. To provide an annual program operating budget;


7
6. To assess program status and evaluate performance;
7. To evaluate program success at program end (See End Note
3).
These seven factors will be used in the development of evaluation
criteria in Chapter 3.
A common error made by those not familiar with the estimation
process is to treat a program cost estimate as a deterministic value,
when in reality it is highly probabilistic (or "stochastic") (See
Fisher, 1962; Fisher, 1973). Thus, to the outsider, if the Air Force
testifies to Congress that it can build the B-l Bomber for N billion
dollars, he/she expects that the Congress should be able to purchase
a B-l, fully developed, for that price. If the program actually
costs twice N billion dollars, the layperson accuses the Air Force of
incompetence, treachery, and often dishonesty.
Whether or not these criticisms are justified is not
determinable from the comparison of the two cost estimates. Indeed,
the comparison is barely relevant to the questions of competence and
integrity. Cost estimates produced with the best possible methods
employed by the best available professionals are only a first step in
insuring that a program meets its original estimates. Indeed, the
initial estimates must be sufficient to the task at hand. Otherwise,
lack of adequate budgeting may become a disruptive factor to the
program (see Large, 1974). But far more important to the question of
"accuracy" are factors such as;


8
1. At what point in the program are the estimates made (i.e.,
what is known and what remains unknown)? (See Summers, 1965,
Section 2; Quade, 1964, Chap. 15.)
2. How is the program managed? Are adequate cost management
tools available? Is the program manager adequately cost
conscious?
3. How adequately is the program funded? Often, budgetary
processes of the Congress and the Office of Management and
Budget result in annual budgets varying from those planned by
program management, causing disruptions in the program, with
very expensive results.
4. How much pressure are the initial cost estimators under to
conform to predetermined cost constraints or annual budgetary
constraints determined by the political process rather than by
program requirements? (See U.S. General Accounting Office,
Selected Acquisition Reports, 1981, for a discussion of the
causes of cost growth.)
The evaluation process must separate factors such as these,
which are independent of the estimation process, from those factors
which the cost estimating process can influence. This research deals
with the issue of the original adequacy of the estimate, and with the
influence of the estimate on the program. The research also
evaluates and documents the processes successfully employed in
developing the original planning estimates for the Space Shuttle
Program. As background, a description will be given of the program


9
planning environment, including the political environment, to provide
the basis for observing how the processes were adapted to meet
environmental demands.
Purpose of the Research
Hie purpose of the research is to document and evaluate the
effectiveness of Space Shuttle cost estimating methods, by analyzing
their effects on the program, and by comparing them to similar
methods. Perhaps the most common criticism levied against the
federal sector in general and the aerospace community specifically is
that actual costs of programs are much larger than were orignally
predicted (Scherer, 1964, Chap. 1; Summers, 1965, p. 12; Novick,
1970). Although it has often been shown that the phenomenon of cost
growth is not at all unique to the aerospace community (e.g., Summers
1965, Chap. 1; Novick, 1970), because of the large sums of money
involved, the open political debate surrounding the acquisition of
most high technology vehicles, and the necessity for accountability
of such large programs to the taxpaying public, aerospace programs
are often singled out as symbols of government "waste" (Baker 1972,
Chap. 1; Novick, 1970).
In fact, the causes of cost growth are currently fairly well
understood. A number of high quality analytical studies have
established and described the causes (U.S. General Accounting Office,
1981; Harman, 1971; Large, 1974; Marshall & Meckling, 1959). Baker
(1972) and others have noted additional factors, such as management


10
pressures on cost estimators, and the perceived need to submit low
bids to be considered for contract awards (so-called "buying-in")
(also see Summers, 1965, p. 53). This research, although it will not
deal explicitly with the issue of cost growth, does address one of
the essential ingredients of avoiding cost growth, i.e., successful
cost estimating and program management, particularly the influence of
the accuracy of the initial cost estimate. A discussion, drawn from
Space Shuttle Program experience, is presented to illustrate how ac-
curate estimates contribute to the efficiency of the management
processes (see Summers, 1965; Large, 1974).
As will be further illustrated, a mature cost estimating ca-
pability for a number of years did not exist within the NASA,
probably because resources were plentiful and program costs of secon-
dary importance to mission accomplishment. In 1964, six years after
its founding by the National Aeronautics and Space Act of 1958, the
most authoritative NASA reference for cost estimating source was a
small section in the Handbook of Astronautical Engineering, an
encyclopaedia of basic aerospace engineering knowledge edited by
Heinz Hermann Koelle, a member of the von Braun team, then at
Huntsville, Alabama (Koelle, 1961). The treatment was so shallow
that the few equations presented were more of a hazard than a help to
the aerospace community. An overview of this history and how the
NASA experience fits within the overall evolution of aerospace indus-
try cost estimating methodology is presented in detail later in this
chapter.


11
A few within NASA realized (End Note 4), even in those early
years of relative budgetary plenty, that sooner or later space pro-
grams would be constrained by cost. The peak NASA budget came in
1967 at approximately $6 billion, which in today's dollars (1982)
would be approximately $18 billion, or almost three times the current
budget of the agency. (See NASA, 1982 for historical NASA budget
details). Those involved in the 1960's with planning the long range
future of manned space flight were missing a very critical tool of
the designer's trade, i.e., cost estimating capability coupled with
the design process, to enable the performance of trades between the
theoretically possible and the politically feasible.
Probably as a result of the strong engineering and research
heritage of NASA, most NASA designers and managers lacked knowledge
of how costs were estimated. A few (within the von Braun group at
the NASA Marshall Space Flight Center, and the Faget organization at
the NASA Manned Spacecraft Center) began to review the work of the
Rand Corporation, to be referenced often herein. Rand had been com-
missioned by the Air Force to produce cost estimating and other
methodologies to be used in performing trades among possible
alternatives for strategic war systems (Novick, 1961; Novick, 1962;
Hitch & McKean, 1961, Chaps. 1, 2, and 3). The Air Force in that
time period had already entered a period of resource scarcity, and
needed to know in advance how much their systems would cost. But
even more important, methods were needed by which to compare very
different systems with different war-making capacities. Economics


12
provided the only common denominator among the candidates, and cost
estimating methods were the very basis of the methodology.
NASA therefore turned first to Rand, then to the aerospace
community's new operations research groups. At first, the methods
developed were makeshift inventions born of the necessities of
engineers concerned about being able to trade cost and performance,
cost and weight, or cost and any given design parameter. As the
Space Shuttle era approached, a significant new body of methods was
produced to enable the most cost-effective vehicle to be designed.
It is the purpose of this work to trace and describe the development
of these methods, to document the influences which produced their
form and function, to illustrate how they were employed, and to mea-
sure and explain their success, their current utility, and their
maturity.
A Summary History of the Cost Estimating Methods
Employed for the Space Shuttle Program
Cost estimating methods are probably as old as monetary
systems, because as long as there has been money, people have
undoubtedly been concerned with cost and its prediction. As long as
there have been government programs there has been effort to improve
how programs are conceived, planned, and managed. Novick (1970)
traces cost growth and mismanagement of government programs to Roman
aqueducts, and observes that such problems are not unique to govern-
ment programs. Summers (1965) points out:
The United States is studded with railroads, canals,
tunnels, bridges, and highways that cost a great deal


13
more than was originally expected ... the Troy and
Greenfield Railroad was more than ten times as much as
the original estimate .... The Suez and Panama Canals
tell much the same story .... The year before digging
on the Suez Canial actually began, the cost estimate was
still low by a factor of three.
Summers goes on to discuss case after case, from canals to nuclear
power plants, which greatly exceeded their original estimates.
Since the principle concern here is with technological
developments, the historical examples related next are from the cur-
rent century. In 1901, for example, Karl Pearson, premier
mathematician and member of the British Royal Society, was commis-
sioned by James Swan, a British inventor and entrepeneur, to develop
a mathematical relationship to predict the cost of developing
Marconi's wireless telegraph into a ship-to-shore and ship-to-ship
system. Three years later, after being paid 1000 pounds sterling,
Pearson concluded that the task was much too large an undertaking,
and gave up (Burbridge, 1982). ,
In 1931, a group of eminent aircraft builders, (including
Martin and Curtis from the United States) met in Paris to develop
methods to be used for prediction of aircraft operations costs.
This effort was to bear more fruit, however, when Neville S. Norway
(whose pen name is Neville Shute, author of On The Beach)
successfully developed mathematical relationships used to predict
operations costs for aircraft from that time until World War II
(Burbridge, 1982).
Efforts in the United States to provide more rationality for
government decisionmaking have matured only in the past forty


14
years. Up to the beginning of the rapid growth of government
programs, beginning with the New Deal, there was little
sophistication, by today's standards, in analytical program planning
processes. Although some use of cost-benefit analysis was made by
the U.S. military as early as 1886 (Quade, 1971, p. 2), the uses
were very limited. Perhaps the first modern use of cost-related
methods was by the Bureau of Reclamation in the 1930's (Quade, 1971;
McKean, 1965). However, it was not until World War II, where sur-
vival of nations depended on the ability to make maximum use of
scarce resources, that analytical methods reached the levels of
maturity recognized today as important for efficient program manage-
ment in government. (See Hitch in Quade, 1964, p. 2; Churchman,
1968, p. viii, ix; Quade, 1971, Chap. 1).
World Whr II had greatly stimulated the development of ana-
lytical methods for systems analysis. Realizing the value of these
disciplines, the Air Force in the late 1940's created the Rand Cor-
poration to serve as a "think tank", to assist the development of
future weapons systems and to develop and perfect those
methodologies associated with systems analysis and program
evaluation. It was Rand which provided the theoretical and analyti-
cal basis for much of today's cost estimating and program assessment
methodology. A scanning of the reference list of this research and
Appendix A will reveal the broad extent of subjects dealt with by
Rand. Novick's System and Total Force Cost Analysis (Novick, 1961)
still serves as a basic reference in the construction of cost
estimating models. Roland McKean's Economics of Defense


15
(McKean, 1964), and the book published by Charles Hitch and McKean,
The Economics of Defense In the Nuclear Age, (Hitch and McKean,
1960) are landmark works in describing the uniquely economic
problems associated with major R&D programs. These works provide a
major theoretical base for the methodology to be analyzed in this
research.
High technology programs of the 1950's overran their origi-
nal estimates by an average of 226% (Marschak, Glennan, and Summers
1962, pp. 150-152; Peck and Scherer, 1962, show similar results).
By the 1960's, estimating accuracy had shown appreciable
improvement, improving by at least 25% (Baker, 1972, p. 35). Smith
and Friedman (1980) also show increases in estimating accuracy, as a
function of calendar year, for aircraft, missile, and helicopter
programs. Figure 1-3 summarizes the improvement of costing methods
with time. Whether or not the methods used for the Space Shuttle
Program followed this general trend of improvement will be pursued
in detail by this research.
Margolis (1966, p. 1) recognizes "the aerospace industry,
DOD, and NASA" as "the breeding ground of cost-effectiveness analy-
sis becuase of their uniquely complicated planning problems." Rand,
a southern California non profit corporation, not only sought out
and employed the best analytical talent, but nationalized the scope
of their studies by the use of experts from other areas of the
nation. Thus, until current times, Rand dominated the literature of
cost estimating methodology. (See Chapter 2).


FIGURE 1-3. CHRONOLOGICAL TREND OF
GOVERNMENT COST GROWTH
250%
200%
150%
AVERAGE COST GROWTH
OF PROGRAMS STUDIED
ACTUAL-ESTIMATE
ESTIMATE 100%
REFERENCES
1. DEWS. D., AND SMITH, G. K. ACQUISITION POLICY
EFFECTIVENESS. SANTA MONICA. CALIF.: THE RAND
CORP. OCT 1979 .
2. HARMAN. A. J. ACQUISITION COST EXPERIENCE
AND PREDICTABILITY. SANTA MONICA, CALIF.; THE
RAND CORP., JAN 1971.
3. HIGGINS. G. IN BAKER, B. N. IMPROVING COST
ESTIMATING AND ANALYSIS IN DOD AND NASA.
WASHINGTON. D.C.: THE GEORGE WASHINGTON
UNIVERSITY. 1972.
4. MARCHAK. T.. GLENNAN. T.K.. JR.. AND SUMMERS,
R. STRATEGY FOR R&D: STUDIES IN
MICROECONOMICS OF DEVELOPMENT. NEW YORK
SPRINGER-VERLAG. 1962.
5. PECK. M. AND SCHERER. F. M. THE WEAPONS
ACQUISITION PROCESS: AN ECONOMIC ANALYSIS.
BOSTON: HARVARD PRESS. 1962.
6. PERRY. R.. SMITH. G. K. HARMAN. A. J., AND
HENDRICKSON. S. SYSTEM ACQUISITION
STRATEGIES. SANTA MONICA, CALIF.: THE RAND
CORP JUNE, 1971.
50%
0
- 50%


17
During the Apollo Program, NASA chose Rand as a member of an
industry-government team designed to adapt the methods pioneered by
Rand into a new methodology for the costing and cost effectiveness
analysis of space programs. Several works were produced under this
effort (e.g., String, 1967). Complex computer cost models were de-
veloped for spacecraft for NASA Johnson Space Center by General
Dynamics (Brents, 1966); and for launch vehicles for the NASA
Marshall Space Flight Center.
In the spacecraft research and development (R&D) industry,
the earliest treatment of the subject is the rather primative work
previously mentioned, by H. H. Koelle, and his team at the NASA
Marshall Space Flight Center (MSFC) (Koelle, 1961). Subsequently
this team was to lead the way in developing cost analysis methods
for NASA. Their work was based on the considerable efforts per-
formed during the McNamara era at the Department of Defense (DOD),
again notably the work of various individuals at the Rand
Corporation. DOD was probably the first to assemble and formalize
the various disciplines involved into a training curriculum for cost
estimators. Formal classes in cost estimating were taught by the
Air Force from the late 1950's through the mid 1960's. (See, for
example, Margolis 1966; Fisher, 1962; Noah, 1962). These Air Force
efforts were, of course, primarily aircraft oriented, reflecting the
substantial data bases developed in the 1950's by Rand and various
airframe manufacturers. Lockheed and General Dynamics were the
first major outside contractors to perform funded studies for the
NASA, utilizing their aircraft and military missile techniques and


18
data bases in the development of methods applicable to NASA launch
vehicles and spacecraft (Brents, 1966).
Despite the fact that most airframe manufacturers continued
to rely on industrial engineering methods as their primary mode of
cost estimating, (see Crain, 1969), it should be noted that almost
all were involved to some extent in the development of new
parametric methods, because the competitive advantage of having
accurate cost estimating methods (particularly in an era of
increasing fee incentives based on cost performance) was very
evident. The commercial airplane establishments, notably the Boeing
Company, were also heavily involved in methodology development. In
this instance, for reasons of competition, efforts were jealously
guarded and were not made available to NASA. However, Boeing
personnel very generously provided guidance in the building of early
NASA models.
Launch vehicle cost estimating efforts at the MSFC were soon
followed (in 1963) by efforts at the Manned Spacecraft Center (now
the Lyndon B. Johnson Space Center, JSC), associated with
spacecraft. Where launch vehicles involved perhaps five or six
subsystems, (i.e., structures, propulsion, guidance, stabilization,
instrumentation, and communications) the spacecraft efforts included
some twelve subsystems, (those mentioned above plus environmental
control, crew systems, orbital attitude control, re-entry thermal
control, orbital maneuvering, and recovery) requiring much more
extensive data bases. Study contracts were awarded to General
Dynamics to develop a computer cost model (Brents, 1966). And to


19
alleviate some of the proprietary problems associated with data
interchange among competiny aerospace companies, Booze, Allen, and
Hamilton was commissioned to develop estimating relationships from
the few available data points. At that time only Gemini and partial
Apollo data were available for spacecraft subsystems. (See Booz
Allen Applied Research, 1966).
Given the state of the art in computing equipment of the
early 1960's, these models proved cumbersome and expensive to
operate. Their applicability was largely limited to ballistic
spacecraft, just as the MSFC models were limited to launch vehicles
of the Apollo/Saturn class. Clearly there was a need for more
general methods which could be applied to the complete spectrum of
manned vehicles then under study by NASA (i.e., advanced earth
orbital and lunar exploration vehicles, large space stations, and
manned planetary expedition vehicles). Study contracts were awarded
by NASA to a number of contractors to develop conceptual designs and
cost estimates for these vehicles (see String, 1967). Because of
the scarcity of analogous data, the resulting estimates were, at
best, first order approximations of actual costs. As it became
clear in the late 1960's that NASA was to focus more on the
exploitation of near-earth orbital space, these efforts were
gradually abandoned. Major events and milestones of this chronology
are shown in Figure 1-4.
The Program Management Environment
The beginning of the Space Shuttle program occured in a
management environment scarcely aware of the severe fiscal


FIGURE 1-4. HISTORY OF SPACE SHUTTLE COST
ESTIMATING METHOD DEVELOPMENT
| wwn 1
58
A NATIONAL SPACE ACT ESTABLISHED NASA
A CHURCHMAN, ACKOFF, ARNOFF, 1957 (MAJOR TEXT IN O.R.)
OPERATIONS RESEARCH \
A IBM 704 COMPUTER
____-COST EFFECTIVENESSAHITCH AND j AQUADE'S HISTORY (QUADE. 1971)
MC KEAN, 1960
51
A FIRST RAND REPORTS:
LEARNING, AIRFRAME CER'S
A PECK AND SCHERER, 1962 (WEAPONS ACQUISITION)
QUADE, ed 1964 A AVON BERTALANFY, 1968
SYSTEMS ANALYSIS, SYSTEMS APPROACH, GENERAL SYSTEMS THEORY
SYSTEMS ANALYSIS, SYSTEMS APPROACH, GENERAL SYSTEMS THEORY
63
FIRST NASA COST MODELS A
(RAND. LOCKHEED,
GENERAL DYNAMICS)
66
A FIRST SPACECRAFT COST MODELS
1 APOLLO COST / (ROCKWELL, GRUMMAN.\
J DATA STUDIES \ MC DONNELL-DOUGLAS) /
NASA SPACE SHUTTLE METHOD IMPROVEMENT | | ]
1940
I
1950
RCA "PRICE" MODEL
_]____________________L_
1960 1970
I
1980
ro
o


21
constraints to come. Even in the excitement of the lunar triumph of
the Apollo program (End Note 5), however, there were those such as
Hugh Dryden, deputy NASA Administrator, who recognized that another
national commitment such as Apollo was not likely to happen, at
least for the next major program (Brooks, et al., 1979, p. 362).
The emphasis was clearly to be on exploitation of Apollo technology
and upon economy of operations. However, the makeup of the program
management force within NASA was Culturally adapted to an environ-
ment of urgency, virtually without funding limits (Brooks et al.,
1979, pp. 362-366).
This section briefly describes several aspects of the man-
agement environment in order to illustrate the environmental
influences on advanced program planning. In particular, the cost
estimating process which was to be a key to program definition for
the Space Shuttle is examined in the context of the political,
technological, and budgetary environments of NASA.
The Political Environment
"In October 1957 the nation was stunned by the successful
orbital flight of the first Russian Sputnik, and on April 12, 1961,
by the first manned orbital flight, by Russian Cbsmonaut Yuri
Gagarin." (Brooks, et al., 1979, p. xiii). Almost from that moment
until 1969, the lunar landing was to be a preeminent national goal;
the $20 billion price (1965 dollars) was to be often debated but
never seriously challenged by the Congress. The Apollo Program was
also born in the pre-Vietnam period, an era of relative prosperity


22
and of low inflation, which also brought the New Frontier and Great
Society Programs to the nation. By the time the lunar landing
approached, defense spending had taken a sharp upward turn because
of the Vietnam war; Great Society programs had rapidly eliminated
the slim budgetary surplusses inherited by Lyndon Johnson; the tur-
moil of the 60's had shifted Congressional attention away from
Space; and the nation was slipping toward recession (Cleveland and
Biederman, 1982).
Additionally, the Russians had become less of a perceived
threat, and the Communist threat in general had been reduced in
stature by the Nixon China policy. The Russians, who supposedly
were in a "race" with the United States, had been unable to mount a
serious manned program. Lacking the resources and technical
expertise, particularly in large rockets and spacecraft computers,
the Soviet lunar program had been reduced to an unmanned lunar sam-
ple return effort, one spacecraft of which they launched the very
week the United States launched Apollo 11 (Brooks, et al., 1979, p.
340).
Perceptions within the Congress in the last days of the
1960's were that the nation should concentrate on exploitation of
the large Apollo expenditure, and forego such missions as manned
planetary exploration (Brooks, et al., 1979, p. 362). (See
technological environment, below). Coming into the Space Shuttle
decade, then, NASA in general, and George Mueller, Associate admin-
istrator for Manned Space Flight, in particular, began to emphasize
the need for economy of operations in space (End Note 6).


23
The political climate was also to be influenced by a presi-
dential task force, chaired by Vice President Spiro T. Agnew and
co-chaired by the presidential science adviser, Lee A. DuBridge.
Robert Seamans, Air Force Secretary, and Thomas Paine, NASA
Administrator, were the other members. Ibis committee was formed in
the Spring of 1969 to make recommendations for U.S. space policy for
the 1970's (U.S. Senate, 1969, pp. 658, 675). Already, in 1968,
NASA had developed in-house an integrated program plan which defined
three program elements: an advanced lunar exploration program, an
advanced earth orbital program, and a manned planetary exploration
program. In support of the presidential task force (called Space
Task Group), NASA modified its planning to reflect the needs of the
committee, and submitted cost, schedule, and technical data for
three program options. These were to be variations of the integrat-
ed plan of 1968 (see Figure 1-5 for a summary of the key events in
this policy making process). The elements of this plan will be de-
scribed later in this chapter.
The Task Force presented the three options to the President,
the highest cost option (I) calling for all three program elements
(lunar, orbital, and manned planetary); the medium option (II)
calling for a delay in the manned planetary program, but also in-
cluding all three program elements; the lowest cost option (III)
calling for only the advanced lunar and earth orbital program
elements. Option II was recommended. However, the presidential
direction (based on the other demands for national resources) was


FIGURE 1-5. SPACE SHUTTLE PROGRAM HISTORY
EARLY PLANNING MILESTONES
1968
APOLLO
NASA
INTEGRATED
PLAN
A
VTTT
1969
^ XI
FIRST LUNAR
LANDING
1970

PRESIDENT APPOINTS
SPACE TASK GROUP
SPACE TASK A
GROUP REPORTS
A
PRESIDENT STATES
SPACE POLICY
ro
4S


25
to be more like option III, with a further deferral of advanced
lunar exploration. Annual costs of the three options are presented
in Figure 1-6. (See NASA, 1969a.)
In December of 1970, executives from both the public and
private sector sides of the aerospace industry were called by the
Congress to discuss their views on the national space program
"Present and Future". Concern in the.House Committee on Science and
Astronautics Subcommittee on NASA Oversight was that:
Even before the successful manned lunar landing in
1969, ..., the national space effort had begun to
diminish.... The aerospace industry is in a state of
distress, ironically after a decade of progress and
accomplishment unequalled in history. And despite
numerous truly remarkable successes, public enthusiasm
for the NASA program seems to have waned. (U. S.
House of Representatives, 1970, p. v.).
The committee therefore scheduled public hearings for the
purpose of setting goals for the national space program "for the
next decade." Perhaps an indication of the priority of the space
programs, the hearings were cancelled before they occurred.
However, in the papers submitted by those who would have testified,
there was general agreement that "a vigorous aeronautics and space
program were in the national interest, since it sets the pace for
the country's technological advancement," and that the program
should be revitalized.
The first paper in the compendium was submitted by the
President of the United States, who, while cautioning that "we
should not try to do everything at once," went on to state three
"general purposes" and "six specific objectives" for the space


FIGURE 1-6. SPACE TASK GROUP PROGRAM RECOMMENDATION
ANNUAL EXPENDITURES
W
cc
s
_]
o
Q
C>
u.
O
CO
z
O
m
60 64 68 72 76 80 84 88
FISCAL YEAR
ro
CTi


27
program of the 1970's. The influence of the task force was strongly
in evidence. Ohe three purposes were: "Exploration"; "Scientific
Knowledge a greater systematic understanding about ourselves and
our universe;" and "Practical Application turning the lessons we
learned in space to the early benefit of life on earth." (Emphasis
added.)
The six objectives were:
1. "...continue to explore the Moon"
2. "...Move ahead with bold exploration of the planets and
the universe," (unmanned);
3. "Reduce substantial^ the cost of space operations"
(emphasis added); "... we must devise less costly and less
complicated ways of transporting payloads into space.... We
are currently examining in greater detail the feasibility of
reusable shuttles as one way of achieving this objective."
4. "... extend man's capability to live and work in space.
The experimental space station (XSS) a large orbiting
workshop will be an important part..." and "could become a
building block for manned interplanetary travel."
5. "... hasten and expand the practical applications of
space technology;" and
6. "... encourage greater international cooperation in
space." (U.S. House of Representatives, December 10, 1970,
pp. 1-5).
Despite these highly ambitious goals and objectives, a bud-
get to support only a modest portion of the objectives was to


28
materialize, as will be shown below. Gone, for the time being at
least, was any consideration of large permanent space stations, ad-
vanced lunar colonies, or manned planetary flight. What was needed
at that time, in keeping with the mood of the nation for economy,
was a less expensive way to get to space and back, the "space road"
which would open up space as the railroad had opened up the West.
The name "Shuttle" was chosen by Associate Administrator Mueller and
Administrator Fletcher to connote a low cost operating system, in
keeping with the political mood of the times. Given the position
taken in the President's paper, both the concept and the name chosen
were adopted readily by the Administration.
One final element of the political environment was the dedi-
cation of the Nixon administration to decentralization, or moving as
much of the government as possible out of Washington. This was
partly the reason that the Johnson Space Center was to be named the
management center or "lead center" for the Space Shuttle Program.
Unlike Apollo, the Shuttle was to be managed in the field. At
least, that was the intent in 1969. With increasing management re-
sponsibility came the necessity for JSC to become more sophisticated
in the estimation and management of costs. If the mood of the na-
tion was changed, the management structure and cultural inheritance
of NASA were not. These cultural influences on the development of
skills and methods involved in program planning and definition must
next be discussed.


29
The Cultural and Techno-Cultural Environment
A number of well-researched histories have been written of
NASA and its predecessor, the National Advisory Committee for Aero-
nautics (the NACA) (e.g., Bilstein, 1980; Anderson, 1981; Brooks, et
al., 1979). From these, the major activities which influenced the
development of advanced design and related cost estimating processes
can be summarized (see End Note 7). As has been recounted, NASA was
created out of the national need to provide the basic research and
development to insure continued technological leadership for the
United States (National Aeronautics and Space Act, 1958).
Superimposed on the research and development effort, and giving
direction to it, were the space programs themselves, both unmanned
and manned. Because the earliest managers of the NASA manned pro-
gram had come from a research center (Langley Research Center,
Virginia), the agency very early in its development realized the
need to establish a management philosophy and infuse management tal-
ent more suited to the management of large programs (End Note 8).
Professional program managers were hired, principally from
the aerospace and related industries, to manage the large programs.
The Langley contingent, however, continued to exercise a great deal
of influence during this transitional period from Apollo to Space
Shuttle. Professional program managers (e.g., Charles V. Frick,
Joseph F. Shea, Carroll H. Bollender) came and went, but the Langley
contingent, which had molded the technical capability of the manned
program, stayed on to influence the technical management culture as
well. Thus, as the Apollo management structure was disbanded during


30
the late 1960's, the surviving organizations at the Johnson Space
Center (then the Manned Spacecraft Center) were composed primarily
of technical engineering and operations specialists, organized
functionally.
The functional engineering disciplines (e.g., structural
engineers, electronic engineers, environmental control system
engineers, etc.), which are the main ones involved in the develop-
ment of a new program, were integrated not by 3 formal systems inte-
gration function but largely through the talents of one individual,
Maxime A. Faget, whose influence had either solely or very strongly
affected the evolution of the Mercury, Gemini, and Apollo spacecraft
configurations (Grimwood, 1963, pp. 12, 14, 18, 30, 65, 176-178;
Hacker and Grimwood, 1977, pp. 29, 30, 46, 89; Brooks, Grimwood, and
Swenson, 1979, pp. 17, 21, 26, 35, 37, 99; Ertell and Morse, 1969,
especially Vol. I). As chronicled by these authors, Faget's primary
imprint on the program was perhaps the genius of invention. Not an
experienced manager of large programs, his main role was that of the
inventor or innovator. When the early NASA JSC Shuttle study team
was assembled, there was little expertise available among its mem-
bers in the management of large programs or in the technologies of
management (most of the experienced program managers remaining at
JSC were still managing the Apollo program).
There was at least one clear exception. One division of
Faget's Engineering and Development Directorate had for nine years
pursued the study of advanced missions (i.e., those to be initiated


31
subsequent to on-going programs). NASA had organized several
advanced planning activities, out of the mainstream programs and
isolated from them. Partially because the influence and power was
usually close to the on-going programs, the long range planners
generally were not highly influential. But because Paget was so
highly interested in invention, in 1973 he established his own
advanced planning group, called the Advanced Spacecraft Technology
Division, to design spacecraft for advanced missions. This
organization was headed by William E. Stoney, a former Faget protege
and manager of the unmanned Scout launch vehicle program. Stoney
had just returned to NASA from the Sloan School at the Massachusetts
Institute of Technology where he had studied technological
management. Soon after his arrival, Stoney created a small group to
develop methods for estimating spacecraft costs to enable designers
to size budgets for the many new concepts to be proposed by the new
division.
Concepts studied by this Faget/Stoney group (later headed by
Caldwell Johnson) included advanced lunar missions, manned planetary
landing and flyby missions, and earth orbital missions of a wide
variety. Major contributions made by this organization were the
initial Skylab configuration studies; the conceptual definition of
the Space Shuttle (then called Integrated Launch/Re-entry Vehicle,
or ILRV); and the docking adapter for the Apollo-Soyuz Test Project
(ASTP)(see Newkirk, Ertel, and Brooks, 1977, pp. 22-39, 105, 168,
175; Ezell and Ezell, 1978, pp. 113, 155, 170-173). But in
addition, out of this organization was born the JSC capability to


32
estimate the costs of large spacecraft programs. This capability,
along with that of the Marshall Space Flight Center (MSFC) at
Huntsville, Alabama, under H. H. Koelle (Brooks, et al., 1979, pp.
5, 23) and his successors, became the basis for all agency advanced
program cost estimating.
While Faget and Stoney were both innovative designers, by
training and background, they were aeronautical engineers.
Engineering schools then, as now, placed little or no emphasis on
the economics of design. So, even after the estimating group was
formed (of two engineers and two mathematicians), it was considered
as an adjunct to the design process. It was only later that it
would become more integrated during the design of the Shuttle.
Several other factors influenced the character of the cost
estimating group. First, the group was formed late in the staffing
process of the new center and never had the ability to recruit tal-
ent freely, as had the other engineering disciplines. Second, ex-
cept for the influence of Robert S. McNamara at the Department of
Defense and that of the Rand Corporation, there was little attention
paid by the aerospace education industry to the development of vehi-
cle designers equally skilled in cost estimating. As a result,
there was no ready pool of cost estimating designers from which to
draw. What few were available were being drawn to Washington, D.C.,
area consulting firms to support the McNamara reforms within DOD.
In a scarce supply market, NASA salaries usually were not very
competitive.


33
Therefore, Stoney authorized his group to hire and train ju-
nior people in the skills needed. Still, from 1963 to 1971, the or-
ganization at JSC never numbered over four analysts. A similar
small group at the MSFC, the residue of the group formed by H. H.
Koelle, completed the entire complement of cost estimators for the
agency. At the start of the Shuttle program conceptual phase in
1969, within a manned space flight organization of some 20,000, NASA
could claim at most eight people qualified to perform design-related
cost analysis. And yet, it was the ability to relate designs to
costs in an integral fashion which was to become one of the most
significant influences during the design phase of the Space Shuttle
(as will be illustrated later in this Chapter).
The work of the Stoney group was heavily influenced by the
efforts of Heinz Hermann Koelle at the NASA Marshall Space Flight
Center. Although a native German, Koelle was too young to have been
a member of the early von Braun team. However, he was a skilled
young engineer whom von Braun admired and brought from Germany to
perform conceptual design studies associated with advanced space
programs. Early in his career at the Marshall Space Flight Center,
Koelle recognized the need for a cost estimating capability and
recruited a small group from the aerospace indistry (primarily Gen-
eral Dynamics) to accomplish this work (End Note 9). This group
would be highly instrumental in the development of cost estimating
methods throughout NASA, mainly by pioneering many of the necessary
methodological studies. (More details on these studies are
presented in Chapter 2).


34
At NASA Headquarters, the factors influencing the style or
culture of the management process are harder to ascertain from the
references cited. However, it appears that the form of the Office
of Manned Space Flight, to which the Johnson Space-Center has always
reported (either for program responsibility, administratively, or
both), has been mainly influenced by individual personalties in the
form of the incumbent in the office of the Associate Administrator
for Manned Space Flight. At the time of the transition to the Space
Shuttle, George E. Mueller was the incumbent.
Mueller had previously been vice president for research and
development of Space Technology Laboratories, Inc., of Los Angeles,
a military high technology systems supplier. He had a PhD in Phys-
ics and 23 years of professional experience, much of it in
spacecraft research and development (Brooks, et al., 1978, pp.
128-129). A highly experienced manager, perhaps his best credential
came from his development and implementation of the resource and
schedule planning and control systems for the Air Force Minuteman
intercontinental ballistic missile program. Some of his management
innovations (subsystem managers, program control officers) remain a
part of the NASA management environment today. These methodologies
have application, however, principally to the management of an
on-going R&D program, and not to the early conceptual planning
phases. But Mueller was also skilled in program planning, as will
be further described.
Mueller and Faget were clearly the dominant forces on the
NASA side of the Space Shuttle concept design. But there were also


35
those in the private sector who heavily influenced the early Space
Shuttle technological environment, but who made lesser contributions
to the technologies of program planning and cost estimating. While
the private sector participants in the Space Shuttle conceptual def-
inition and preliminary design phases (to be referred to as Phases A
and B, respectively) were highly innovative in their spacecraft
designs, few if any had developed an effective capability to trade
design capability against program cost. This was revealed both by
the performance of the private sector during early funded studies
and by the industry interview program (described later in this
chapter). Of significance to this research is the fact that NASA,
while undertaking a very modest effort in methods development, was
able to recognize this deficiency and to effect some partial
remedies by the time the preliminary design process began in
1970-1971.
In summary, the technological/cultural environment which
influenced the development and use of the methodologies to be de-
scribed and evaluated by this research was a product of the existing
NASA management culture. It had evolved from the Langley Research
Center through several major programs. Hie personal influences of
at least four individuals within this organizaton, i.e, Mueller of
NASA HQ, Faget and Stoney of MSC/JSC; and Koelle of the MSFC; and
the methodological studies performed by Rand, NASA MSFC, and NASA
JSC had been major influences.
What all of these influences had combined to produce by 1969
were two small groups of qualified designer/estimators. Each group


36
was fairly well equipped with methods developed by the Rand
Corporation, General Dynamics, Lockheed, and by themselves; each had
enough analogous cost and technical data to begin a credible design
process and to produce rational estimates. It should be clear from
this discussion that NASA was heavily oriented toward the "pure"
engineering disciplines. Never in the short history of the agency
had there been a major need for an integrated design-cost estimating
process. But times were to change dramatically between 1969 and
1973 when the current Space Shuttle design was finally derived.
The NASA Planning Environment
Realizing that the end of the Apollo program was at hand,
NASA had already in 1968 assembled high level planners from all
centers to define a plan for the decade of the 1970's. The plan
that resulted, as has been stated, included three mission elements,
plus a transportation element to support the other three. First was
an earth orbital program, which called for Saturn Workshops (small
space stations made from Apollo Saturn stages); Skylab was a direct
result. Also suggested were space stations, both in low and
synchronous orbits, and a large space base by the end of the decade,
none of which were to materialize. Second was an advanced lunar
program, evolving from Apollo to "Extended Apollo," which actually
was to reach friution in the form of the lunar roving vehicles used
by latter-day Apollo astronauts. A lunar orbiting station and a
lunar surface base, called for by the end of the decade, were never
to be given serious consideration. The third mission element in the
plan was a planetary program beginning with unmanned programs, and


37
climaxed by a manned Mars expedition late in the decade. Of these,
the Mariner and Viking projects were the only ones to materialize.
The transportation program was envisioned, even then, as a
fully reusable two-stage "fly back" vehicle, only half of which (the
Shuttle Orbiter) was to actually materialize. A space tug (still a
viable candidate) was called for by mid decade, and a heavy lift
tanker, a nuclear shuttle for planetary propellants, by the end of
the decade (NASA 1969a, and End Note 10). All in all, this initial
planning effort constituted little more than a compilation of the
wishes of the entire agency. Had funding stayed at the Apollo
levels, the plans might have actually materialized. However, the
cost requirements of all but a few of the projects kept them from
serious consideration when the realities of the 1970'a budgets
emerged (see the next section for details of the budgetary
environment).
As mentioned, NASA in 1969 again undertook a major program
planning activity, this time in support of the presidential Space
Task Group. Coming as it did so close to the 1968 long range plan,
the result of that effort bore a considerable resemblance to it,
including, in at least one of the three options presented, almost
every element of the 1968 plan. The fiscal environment within NASA
at that time Was still optimistic. Few anticipated that the triumph
of the lunar landing would be rewarded by anything other than an in-
creased budget.
Consequently, even the plan's most pessimistic options re-
quired almost three times the budget which was actually to


38
materialize. The "maximum pace bound" which suggested manned
planetary flights by 1981, peaked at over ten billion 1969 dollars
per year. The agency was to receive a maximum of just over two
billion 1969 dollars per year, a level at which it currently
remains. But at least by this period some serious cost estimating
was taking place within the agency. Hie Bureau of the Budget and
the Congressional staffs in particular were asking more pointed cost
questions. By this time, the mood of Congress was changing to one
of questioning the benefits of space expenditures (U.S. Senate,
1970, pp. 3-40; U.S. Senate, 1969, pp. 658-687). This more
cost-conscious mood stimulated serious and fairly realistic cost
estimating by NASA.
In summary, the NASA planning environment coming into the
beginning of the Space Shuttle program gave very few hints of the
severity of cost issues to come. Uiere was still no major stimulus
to develop a better cost estimating capability than the somewhat
primitive and modestly staffed one (by today's standards) that
existed at that time. Virtually no capability existed at the NASA ~
HQ. MSC/JSC and MSFC had the small groups mentioned previously, but
these were not well equipped for the large tasks to come. One symp-
tom of the problem was that, in 1969, NASA was having to perform
special studies just to find out exactly how much the Apollo
subsystems had actually cost (Rockwell, 1971, and End Note 11).
Ironically, no one had collected the appropriate data during the
program. It was simply not a high priority.


39
'The Budget Environment
The maximum NASA budget appropriation occurred in 1965 at
$5,250 million (1965 dollars). Total outlays peaked the following
year at $5,933 million (1966 dollars). (See NASA, 1982.) In 1981
the budget outlays were numerically approaching those of 1966 and
appropriations had reached the $5,523 million (1981 dollar level),
numerically exceeding the 1965 peak. However, NASA found itself
operating with the lowest levels of buying power it had had since
the early 1960's. With a staff of approximately 23,000, compared to
the early 1960 level of about 10,000, NASA was operating at roughly
the same budget levels, but with only one-third of the peak budgets
of 1965 and 1966. Actual NASA expenditures in fiscal year 1981 were
$5421.2 million (1981 dollars), or $1573 million (1966 dollars), the
lowest levels since 1962). See Figure 1-7, taken from NASA (1982),
for a historical budget summary.
Some of the political factors leading to this decline have
been previously outlined. Hiis section examines later developments,
those which more directly influenced the form of the Space Shuttle
program, occurring from 1968 through the end of the program. The
budgetary time periods may be summarized as follows:
1. 1968-1969, the conceptual phase of the program, or Phase
A: assumption of return to Apollo funding levels or higher;
2. 1970-1971, most of the preliminary design and definition
phase of the program, or Phase B: assumption of constant
$3.5 billion level, escalated to keep pace with inflation;


APPROPRIATIONS, MILLIONS
FIGURE 1-7. NASA BUDGET HISTORY
1959-1981
FISCAL YEAR, PLOTTED AT END
* BASED ON ACTUAL PROGRAM ESCALATION
-is


41
3. 1972-1973, completion of preliminary design and program
definition, and start of the design phase (Phase C/D):
assumption of constant $3.4 billion level, (1972 dollars),
escalated;
4. 1973-1974, the first two full years of the design phase
(Phase C/D): assumption of constant $3.1 billion level,
(1973 dollars), escalated;
5. 1974 to the present, the remainder of the design,
development, test and evaluation phase of the program (Phase
C/D): a constant-level agency budget (in constant value
dollars: see Figure 1-8).
In 1968 and 1969, as NASA was conducting the very early
studies which were to lead to the Space Shuttle program, the
assumption was that whatever levels of funding were required would
be forthcoming, as had happened during the Apollo era (see
"Political Environment," above). The Presidential Space Task Group
had assumed in 1969 a return to Apollo funding levels by the mid
1970's (Figure 1-6). By 1970, however, national emphasis had
shifted away from space. Technology in general was suspect, as a
cause of many of the nation's ills (Burke, 1969). The newly
reorganized Office of Management and Budget (OMB) instructed NASA to
plan on no more than a constant $3.5 billion budget (1970 dollars,
escalated) from that time forward. The President's budget,
submitted to Congress in February, 1970, was at this level (Figure
1-8).


8
7
6
5
4
3
2
1
0
FIGURE 1-8. NASA BUDGET PREDICTIONS
STG
A
5%
ESCALATION
MODEL
f FEB 70:
NIXON 3.5
JAN 72:
FLETCHER 3.4
JAN 73:
FLETCHER 3.1
PROGRAM PHASE A B B1 C/D >

64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80
FISCAL YEAR
4S
M


43
Space Shuttle planners had set about to design vehicles
which could meet the program objectives of low cost per flight,
satisfy stated performance goals (NASA Space Shuttle Task Group,
1969), and could be developed within the $3.5 billion total agency
budgetary constraint. The vehicles being designed were of the
two-stage, fully reusable variety, requiring peak program funding of
over $2 billion (1969 dollars). It was not to be realized, until
two years of study had been made, that such vehicles, despite their
low flight cost, were not affordable on an annual budget basis. The
cost estimating methods described above were becoming critical
design tools at this point in time, and the dynamic behavior which
the budget was to display for the next few years was to increase the
urgency to develop even more responsive methods.
The assumption of the $3.5 billion budget was to persist
through most of the preliminary design phase (Phase B) of the Space
Shuttle (see Figure 1-8). After almost two years of study and
attempting to design vehicles which could be built for the available
budget, an additional budget constraint was imposed as a result of
budget discussion between Administrator James Fletcher and the OMB.
In January 1972, Fletcher informed the Shuttle program director that
the new agency budget would be only 3.4 billion (1972 constant
dollars). At first the $100 million reduction did not appear too
significant. However, two years of high inflation had reduced the
value of the dollar by 11%; in terms of 1970 dollars, the new budget
was Only $3.06 billion, 87% of the budget on which the Shuttle
program was being based.


44
The Space Shuttle program as then defined could not have
been accomplished within the budgetary levels predicted in 1970; by
1972, the problem was 13% worse (Figure 1-9). The Space Shuttle
share of the total agency budget was reduced to $1.1-1.2 billion
(1972 dollars). By this time preliminary design was all but
complete on an an unaffordable vehicle. (See Figure 1-10 for a
chronology of the Space Shuttle Program.) But this was not to be
the last budget surprise. One year later, in January of 1973,
Administrator Fletcher informed the agency that the budget had been
again revised downward, this time to $3.1 billion (1973 constant
dollars), as a result of administration action. By this action, the
planning budget was reduced to $2.6 billion (1970 dollars), or only
75% of the budget expectations of three years earlier. She effects
on the program of this 25% reduction in budget were, of course,
enormous. Effects on the development of program cost estimating and
design methods were equally pronounced.
Between 1970 and 1972, the highly unstable planning budget
caused literally hundreds of design iterations to be studied. A
cost for each had to be estimated and peak funding requirements
compared to the available budget. By 1973, however, it was too late
to accommodate budget reductions by changing the Shuttle design.
The final design phase of the program by then had been underway for
over 6 months, the approval received from the Administration and the
Congress; and cost agreements had been made. NASA had agreed to
design and develop the Space Shuttle for $5.2 billion (1971
dollars), and to do it within a $1.2 billion (1971 dollar) peak


FIGURE 1-9. NASA PLANNING BUDGET CHRONOLOGY,
EARLY YEARS OF THE SPACE SHUTTLE PROGRAM*
DATE OF BUDGET PREDICTION AMOUNT PREDICTED (DOLLARS OF THAT YEAR BILLIONS) AMOUNT PREDICTED (1970 DOLLARS, BILLIONS) PERCENT OF FEB 1970 PREDICTION
FEB 1970 3.50 3.50 100.0%
JAN 1972 3.40 3.06 87.5%
JAN 1973 3.10 2.64 75.4%
ESCALATION FACTORS ARE FROM ACTUAL PROGRAM EXPERIENCE (See Erickson, L.R.,
SHUTTLE INFLATION HISTORY AND EFFECTS ON THE AGENCY COMMITMENT COST ESTIMATES
(MANAGEMENT BRIEFING). HOUSTON, TEXAS: NASA JOHNSON SPACE CENTER, MAY 14, 1979.)


FIGURE 1-10. SPACE SHUTTLE PROGRAM HISTORY
EARLY EVENTS AND PROGRAM PHASES
4N
On


47
funding constraint. The only remaining variable was program content
(e.g. test hardware, flight articles), which was to be reduced
dramatically, as will be further discussed.
The Design Evolution
Space Shuttle design evolution and the corresponding cost
estimating methodologies closely followed the budgetary evolution
described above. At the time the conceptual design was beginning
(Phase A of the program, as defined by phased program planning
processes; see End Note 12), there were no known program cost
constraints. The stated design criteria (NASA Space Station Task
Group, 1969, Volume II) were being translated into conceptual
"paper" designs by four aerospace contractors; General Dynamics,
Lockheed, McDonnell- Douglas, and North American Space Division
(later Rockwell Space Division).
This phase lasted from January through October in 1969 and
produced designs capable of transporting a 15 foot by 60 foot,
50,000 lb payload to a low altitude orbit of 55 degrees
inclination. Many designs were analyzed; but perhaps the most fa-
vored were of a two-stage fully reusable configuration, meaning a
recoverable/reusable orbiter vehicle launched by a recoverable,
"fly-back" booster vehicle of similar (but much larger) design.
These designs all reduced operations costs substantially, according
to contractor reports; costs-per-flight of from $3 to $7 million
(1969 dollars) were estimated (End Note 13).


48
As will be described later in this chapter, a major review
of contractor program management methods was performed by NASA to
evaluate the capabilities of the various contractors. One conclu-
sion drawn from this set of interviews was that not one of the
contractors was then able to provide convincing cost estimates for
the Space Shuttle. Cost estimating methods, where they existed,
were mainly based on aircraft manufacturing experiences of the vari-
ous contractors. They contained very few points of valid spacecraft
data (if any at all). Many times, processes were utilized which had
been demonstrated to have quite poor capability in early program
phases to predict the actual costs of a program (see, for example
Zussman, 1969; Henry, 1976; Summers, 1965). The need to develop
better cost estimating methods became quite evident to NASA through
this process.
By the end of the Phase A conceptual studies, the Space Task
Group report had been made (see Figure 1-10); but even that report
recognized few budgetary constraints. By the beginning of the next
program phase, (June 1970), the President had issued the $3.5
billion (1970 dollar) limit for the total agency budget, as
previously described. However, such was the state of the art in
cost estimating and budgetary analysis, that the full effect of the
budgetary constraint was not yet realized. Vehicle sizes at the be-
ginning of the phase ranged from 2.8 million pounds at liftoff to
4.5 million pounds. The many concepts of Phase A had by then been
reduced to a single concept, the two-stage-fully-reusable vehicle
previously described, which was shown to have the lowest operational


49
costs. Because the full implications of the budget constraint had
not yet been realized, work continued even until early 1972 on these
large vehicles (see final reports from Space Shuttle Phase B and
Phase B extensions, End Note 14).
By mid 1971, the severe effect of the budgetary limit became
more obvious to program management. Suddenly there existed designs
which could not be accommodated within the billion dollar peak annu-
al limit imposed upon the program. As is traced in Figure 1-11,
(taken from a NASA management briefing of 1973) the Phase B studies
were extended to seek vehicles with lower peak development cost
requirements. First, vehicles were made smaller by removing some of
the propellants from their interiors and replacing them with
propellants in throw-away external tanks. Because parts of the ve-
hicle were now to be expended, cost-per-flight rose, while peak
funding was only slightly reduced.
Next, various expendable booster concepts were explored,
which showed promise of achieving the required development cost
reductions. But cost per flight rose dramatically, to over ten mil-
lion (1971) dollars. Only a solid rocket booster configuration,
with very low development cost requirements, satisfied the peak an-
nual cost constraint, but at the expense of increased cost per
flight.
Between May of 1971, when the peak funding problem was fully
recognized, and March of 1972, the Johnson Space Center studied over
49 separate vehicle configurations. For each configuration, a de-
tailed cost estimate had to be prepared. Each estimate not only


FIGURE 1-11. FUNDING GUIDELINE EFFECT ON SYSTEM CONCEPTS
$ BILLIONS (1971 DOLLARS)
2.0 i-
1.6
1.2
FULLY
REUSABLE
PEAK ANNUAL FUNDING
PARTIALLY
REUSABLE
$ MILLIONS (1971 DOLLARS)
101
6 -
REDUCE COST
PER FLIGHT
REDUCE PEAK
ANNUAL FUNDING
1969
MID PHASE B
1971

COST PER FLIGHT
TIME
2.0
1.6
1.2
.8
10
START PHASE C
1972
VJ1
o


51
had to show the designers whether or not they were going in the
proper direction, but also be convincing enough to program manage-
ment that they would agree to give up a desireable design or perfor-
mance feature to realize the cost objective. First total
reusability was sacrificed; next went the flyback booster; and fi-
nally went the more expensive liquid propellant boosters, in favor
of less costly solids. Every step was a conscious compromise to
eventual operational costs, as is shown in Figure 1-11. With such
intense scrutiny being given to costs, the estimating processes were
forced to mature almost over night.
In 1969 it took NASA several months to develop credible cost
estimates for the Integrated Launch/Re-entry Vehicle (IIRV)
studies. By mid 1971 the capability was considerably improved, but
still required weeks to perform a complete new estimate. However,
when configurations were being designed at the rate of greater than
one per week, the developers of the methods were forced to respond,
and, as will be shown, produced a capability truly representative of
the most advanced state of the art in the industry at that time. To
provide the basis for the evaluation of these methods, as they had
evolved through the evolutionary processes described above, a com-
plete description of the methods must be presented.
Description of Space Shuttle Cost Estimating Methods
Prior to beginning the description of the evaluation
process, it is important to describe the methods being analyzed.
Methods on which the Space Shuttle program cost estimating methods


52
were based were described above, and will be further described in
explaining the literature (Chapter 2). Space Shuttle methods were
not unique in the theories or general approaches used; i.e., they
were data-based methods, employing statistical inference as a pre-
diction medium. However, the application of the methods involved
not only skillful use of the mechanistic approaches borrowed from
the external community, but also compensated for the limitations of
the purely mathematical techniques by augmeting the analogous data
base and exploring methods employed by the industry in program
implementation, i.e., in making the estimates realize fruition.
Thus the unique element of the methods employed for the Space
Shuttle was not in the analytical theory, but rather in the
innovative use of the available data and the somewhat fortuitous ap-
plication of lessons learned from previous programs (these lessons
are summarized later in this Chapter, and described in detail in
Appendix E).
The evolution of any technology is dynamic, and especially
so in the case of the Space Shuttle cost estimating methods during
the years 1971-1973. To provide an accurate chronological reference
for this evaluation, the cost estimating methods as they existed in
March of 1972 were chosen. Uiis choice was based on the fact that a
description is available of the methods as they existed at that
time. In addition, this was close to the time when the NASA/OMB
cost commitment agreements were being formulated, and as such was a
time period likely to be remembered by those managers to whom the


53
first survey instrument was to be administered. A generic descrip-
tion of NASA cost estimating methods must first be presented, and
the unique features of the 1972 methods then described.
Definition of Terms in Cost Estimating
A few terms which involve common words in unique meanings
should be described:
1. Parametric: "of, relating to, or in terms of s
parameter." (Webster's Third New International Dictionary, 1961, p.
1638).
2. Parameter: "A quantity which describes a statistical
population"... an independent variable through functions of which
other functions may be expressed." (Webster's Third New
International Dictionary, 1961, p. 1638).
3. Parametric Methods: "methods which assume that one or
more parameters of the program explain the cost." (Seldon, 1979, p.
25). A parametric technique is one which employs a measurement
medium, generally quantifiable. Very simply stated, parametric cost
estimating techniques include those methods which employ a physical
measurement of some attribute of an object to predict its cost.
Uius virtually all estimating techniques are included. There is a
school of thought which would distinguish parametrics from
"detailed" or "grass roots" estimating (the process described in the
previous section), but this is falsely dichotomous, because success-
ful estimating at any level, including that done by the departmental
estimator, is usually parametric to some degree.


54
The only exception to this is the case of the exact analogy,
which requires no parametric scaling. All cost estimating is per-
formed by analogy. Either the analogy is exact (e.g., the item can
be bought from a catalog), or is inexact to some degree. Successful
methods are those which best use their available analogies. An
analogy is established by reference to a data base (or, more
precisely, by inferences drawn from the data base), which can be in-
formal or highly structured. Since data are the key to all
estimating, more elaboration is required on this subject.
4. Learning; the reduction in cost to procure a given
product, caused by experience gained as a function of the quantity
of the items produced; i.e., "the amount of effort required to manu-
facture an item decreases with each successive item produced"
(Seldon, 1979, p. 55). Analytical expressions have been developed
by Wright and Crawford to describe the behavior in mathematical
terms (Seldon 1979, pp. 55-56).
5. Work Breakdown Structure (WBS): a method for organizing
a cost estimate "to insure that all elements of work are covered."
(Seldon 1979, p. 34). The method is hierarchial in nature, e.g.,
programs are made up of systems, systems made up of subsystems,
etc. The Work Breakdown Structure is in essence a logical simula-
tion of the relationship of the elements in a program.
6. Cost Model: "A method, based on technical and
programmatic parameters, of estimating costs. It converts the ver-
bal description of the program into a step-by-step procedure to cal-
culate costs." (Seldon 1979, pp. 157-158). Seldon suggests that


55
the broadest use of the word model "would incorporate the whole
costing process, including management review actions and
decisions." (Seldon 1979, p. 159). As used here, however, the term
"cost model" will generally apply only to the mathematical
algorithm.
Principles and Processes of Parametric Estimation
ftiis section represents the synthesis of a number of works
into a general process of cost estimating, principles stated are
primarily those of the Rand works referenced in Chapter 2 (see
especially Fisher, 1962, for an overview; see Batchelder, et al.,
1969, for the process description). Estimating is the process of
converting something which is unknown to something which is known
through a process of inference. Inferences can be derived either
statistically, or by another process, such as a physical law or an
empirically-derived scaling relationship. In the case of an R&D
program, no well-established cost history usually exists. This
precludes the use of statistical inference in most cases; the
availability of one or two closely analogous points of data is con-
sidered fortuitous, and the development of a statistical relation-
ship (even a straight line linear relationship) is frivolous or
inpossible.
Thus the usual regression techniques are virtually worthless
by themselves. Fortunately, however, over the years, through a
large body of eirpirical evidence, it has been learned that certain
types of costs have their own scaling relationships. For exanple,


56
it has been found that developmental costs for items of the same
subsystem family scale exponentially with their weights, roughly in
a 1/2 power, or square root, relationship (graphically displayed in
Figure 1-12). It should be noted that this power will vary with the
level at which the item being estimated lies in the hierarchy of the
system. This 1/2 power relationship applies to spacecraft
subsystems, the first level below the system itself. This simple
observation has the effect of amplifying any available point of data
to an infinite degree. Thus, if the family relationship between the
unknown and a single known item is precise, only a single point of
data and a weight estimate are required to estimate the unknown
cost. This must be tempered by other empirical rules, however, as
it has been found that this technique is accurate only over limited
ranges of size (two to three).
On these few principles are based most of the existing
methods of parametric cost estimation. The resulting methods may be
conveniently divided into a six-step process for estimating the mag-
nitude of a cost (or cost at completion), plus a two-step simulation
of program budgetary requirements (time phasing the estimate at com-
pletion over the expected duration of the program). The six-step
process of estimating cost magnitude at completion will first be
described.
The six discrete steps of estimating the magnitude of a cost
are shown in Figure 1-13 (similar to Seldon, 1979). First, the item
(system) to be costed must be broken into a well-structured
hierarchy (work breakdown structure) of its components (i.e.,


FIGURE 1-12. DEVELOPMENT COST SCALING
.1
Weight
Ln
*4


FIGURE 1-13. STEPS IN PARAMETRIC ESTIMATING

TECHNICAL DESCRIPTION WORK BREAKDOWN STRUCTURE DATA ESTIMATING METHOD DEVELOPMENT COST MODEL COST ESTIMATE


59
subsystems). Second, each component subsystem must be described in
terms of its performance, state of the art, capacity, and size. Any
combination of these physical attributes may be used as a predictive
parameter. Different subsystems are best described by different
parameters. A structure is usually best be described by its size
(weight) and the material of which it is constructed (titanium,
aluminum). Engines are usually best described by capacity (thrust)
and propellants (e.g., hydrogen/oxygen, which are in effect both
performance and complexity, or state of the art, measures, since
hydrogen technology is considered to be more advanced than that of
propellants like kerosene, and provides a higher performance level).
The third step in estimating is to establish the data base.
This step may take any form from the simple acquisition of a single
point of data from an existing data base to the undertaking of a ma-
jor series of data analysis studies, such as those performed for the
Space Shuttle Program (to be described later in this chapter). The
magnitude of this activity is determined by the precision required
of the estimate, the precision of analogy available in existing
data, and by availability of other data sources. This is the key
step in estimating, and all other steps revolve around it. For
example, it may sometimes be necessary to re-structure the work
breakdown structure to match the availability of analogous data.
The fourth step in cost estimating is the development of the
parametric cost estimating relationship (CER). This is the process
of deriving a formal inferential expression, usually an equation or
graph, which relates the attributes of the item to its cost. Here


60
is where either a statistical technique (e.g., regression, if ade-
quate data are available) or an empirical scaling relationship is
applied to the existing data for each subsystem.
The fifth step entails the combination of individual cost
estimating relationships into a cost model, in a hierarchial
fashion, as determined by the work breakdown structure. The sixth
and final step is the making of the subsystem estimates themselves
and combining them, in accordance with the cost model, to form the
total estimate. It should be noted that all of these steps, includ-
ing development of the data base, must be performed in a constant
year dollar base.
Budget Simulation; Cost/Time Relationships
Up to this point, only the estimation of total costs has
been covered in this discussion. But in the preparation of a
budget, it is equally as important to predict the chronological
occurance of the costs, and in many cases, to make these predictions
well in advance (two years or more) of the actual need for funding.
Thus some means must be devised for dividing the estimate into
chronological slices. This process, loosely called "spreading," has
been widely studied, but poorly documented.
With the advent of logic-network scheduling systems (e.g.,
PERT), companion cost systems were derived which simply gave cost
values to every activity in the network. Those systems then provid-
ed a cost/time relationship, which in some cases, often
disastrously, were used for budgets. There is a clear explanation


61
for why such systems should not be used for budgeting in an R&D
environment. Hie planning networks established at the outset of any
R&D program suffer from the inpossibility of predicting every
activity which will occur over the life of the program. Thus, R&D
program networks must constantly be updated to provide useful man-
agement information. As the program progresses, invariably new ac-
tivities must be added or existing activities lengthened.
If one plots the sum of all costs estimated for the activi-
ties at periodic intervals over the actual life of a program, the
curves resemble those found in Figure 1-14 (the actual Apollo Com-
mand and Service Module budget history). Hiis behavior, whereby
each successive budget estimate produces a curve which is slightly
higher, and with a peak shifted to a later chronological period, has
been described as the "bow-wave" effect, an analogy to the wave
which precedes the bow of a ship in the ocean. In developing a
long-range program budget, the final cost and actual spending pro-
file should be used (the actual cost curve, or the curve farthest to
the right in Figure 1-14). Otherwise, the appropriation process can
not be synchronized with the actual requirements for resources.
It was clear early in the JSC method development process
that networking approaches to the time-phasing of costs were both
too cumbersome and too inaccurate for usage. Methods were therefore
derived to simulate the programmatic behavior illustrated by Figure
1-14. Hie resulting methods are described in Figure 1-15, which was
taken from a recent NASA/JSC briefing paper explaining the process
to Space Station Program management. NASA used this method to


ANNUAL COST,
MILLIONS OF REAL YEAR DOLLARS
FIGURE 1-14. APOLLO CSM R&D POP TRENDS
63 AND PRIOR 64 65 66 67 68 69 70 71
T
FISCAL YEAR
ro


CONSTANT $
FIGURE 1-15. SPACE STATION BUDGET SIMULATION LOGIC
ASSUME
FUTURE PROGRAM (SPACE STATION) WILL EXHIBIT COST/SCHEDULE DYNAMICS SIMILAR TO PAST NASA
AND AEROSPACE PROGRAMS
NO SUBSTANTIAL CHANGE IN NASA MANAGEMENT PRACTICES
PHASED MODULAR CONFIGURATION
REMOVE RESERVES FROM PARAMETRIC ESTIMATES
DISTRIBUTE REMAINDER OF PARAMETRIC ESTIMATE TO CURRENTLY-PLANNED SCHEDULE; DISTRIBUTE
RESERVES FROM PROGRAM PEAK TO 1.3 X CURRENTLY-PLANNED SCHEDULE
ESCALATE ESTIMATES TO REAL DOLLARS, BUDGET-YEAR DOLLARS
DISTRIBUTE BUDGETS TO PROJECTS WITHOUT PRO-RATA SHARE OF PROGRAM-LEVEL RESERVES
SIMILARLY, EACH PROJECT SHOULD CONTRACT FOR "GOING-IN" VALUES (ADDITIONAL)
SOURCE: NASA JOHNSON SPACE CENTER. SPACE STATION BUDGET ISSUES
HOUSTON, TEXAS: AUTHOR, FEBRUARY 1983 (MANAGEMENT PRESENTATION).


64
predict what program actual costs would be "looking back" in time,
i.e., at the end of the program. It should be noted that the
parametric methods for predicting the magnitude of costs are also
"looking-back" methods (they estimate actual costs at program end),
since they are all based on end-of-program cost actuals from
historical data.
The essence of the problem then becomes one of predicting
the actual time a program would consume. To £o this, a series of
methods have been developed to estimate schedules in a manner
analogous to that of the cost estimating methods previously
described (i.e., based on physical attributes of the system). (See
Vought Missiles and Space Corporation, 1972.) These methods are
quite detailed, and involved a major developmental effort on the
part of NASA. Once this capability was developed, however, the
simulation of program cost/time relationships became a matter of
developing mathematical functions to simulate the shape of the curve
illustrated in Figure 1-15, and applying these functions to the
times predicted by the time prediction methods.
This was not quite as simple as might be expected, because
different programs were found to have differing shapes. A series of
studies by E. D. Lupo and others (Smith and Riedlinger, 1968)
finally developed a family of curves which could describe a large
variety of programs, based on similarities with other programs
(e.g., launch vehicle programs were shown to have distinctly
different cost envelopes from those of spacecraft). With these
methods in hand, the final steps of the process can be taken. The


65
steps beyond the six basic estimating steps are, first, to develop a
predicted actual program length. The final step is to "spread" the
costs with the appropriate function over the predicted length. The
appropriate escalation indices must be applied to convert the con-
stant year dollars to the "real-year" dollars employed in the feder-
al budget process.
The above discussion has presumed that the physical attri-
butes employed to develop program costs were themselves known with
certainty. This is of course seldom true, and an equally demanding
problem is to develop reliable cost model inputs, or, alternatively,
to test the results of uncertainties. (See Fisher, 1962; Black and
Wilder, 1980). The subject of confidence and risk must be discussed
further.
Preparation for Developing the Methods
While useful theories and processes for cost estimating were
available to the planners of the Space Shuttle Program, no data base
existed with which to develop a comprehensive, subsystem level cost
model specifically for the type of spacecraft/aircraft represented
by the Shuttle. Because of the heavy dependence of the available
theories on historical data, a major data gathering and analysis ef-
fort was conducted by NASA. A discussion of this effort follows.
Data Studies. At the outset of the Space Shuttle Program,
applicable cost estimating methods were very scarce. To provide the
most analogous and current cost data base possible, extensive data
studies were performed by the JSC Qperatins Analysis Office from


66
1969 through 1973. Since the Space Shuttle is an airplane, large
airframe cost data were compiled from the C-5A, B-70, and B-52
programs, using actual DOD and contractor accounting records.
Subsystem data were by then available from the Apollo Command and
Service Module (CSM) (Rockwell, 1971) and Lunar Module (LM)
(Grumman, 1971, and End Note 11), and from the Gemini Program
(McDonnell-Douglas, 1970). High-speed airframe data were developed
by General Dynamics from the B-58 program (General Dynamics, 1972).
Avionics subsystems presented a special set of problems be-
cause of the rapidly changing state of the art. Apollo avionics da-
ta were gathered (Honeywell, 1969), but because of the obsolescence
of the analogies, a separate series of studies was commissioned with
RCA Price Systems Division and the General Research Corporation
(GRC) to relate avionics costs to some measure of state of the art
change (RCA, 1976; Waller and Dodson, 1974). Figure 1-16 lists the
studies performed by a single NASA Center (JSC) from the beginning
of the Space Shuttle cost collection effort in 1969 to the present
time.
In the review of the literature in Chapter 2, the reference
list, and Appendix A, no similar precedent for a data gathering ef-
fort such as this one, dedicated as it was toward estimating costs
of a single advanced spacecraft (the Space Shuttle) was found. The
worth of the effort in providing accurate planning estimates has
been very evident in managing the program, and will be further
evaluated by this research. It is clear that any advanced R&D ef-
fort can benefit immeasurably from this type of precursor analysis.


FIGURE 1-16. STUDIES PERFORMED SINCE 1962 BY JSC
FUNDED
KEY
GD
MDC
NAR
Rl
GAC
L/GA
FW
LAD
GENERAL DYNAMICS CORP
McDonnell douglas aerospace corp
NORTH AMERICAN/ROCKWELL CORP
ROCKWELL INTERNATIONAL SPACE
DIVISION
GRUMMAN AEROSPACE COMPANY
LOCKHEED GEORGIA CO.
FORT WORTH DIVISION
LOS ANGELES DIVISION
LMSC LOCKHEED MISSILES & SPACE CO.
SPACECRAFT SYSTEMS COST MODEL (GD)
SPACECRAFT CER STUDIES (BOOZ-ALLEN, TEXAS A&M)
ADVANCED SPACECRAFT COST ANALYSIS
(ASSCA) (MDC, HONEYWELL)
APOLLO COST STUDIES (SEVERAL YEARS) (NAR, GAC)
CSM COST DATA STUDY (Rl)
LM COST DATA STUDY (GAC)
C-5A COST DATA STUDY (L/GA)
B-58 COST DATA STUDY (GD/FW)
B-70 COST DATA STUDY (RI/LAD)
AGENA COST STUDIES (LMSC)
SHUTTLE CER STUDY (LTV)
SCHEDULE ESTIMATING TECHNIQUES (LTV)
SCHEDULE TECHNIQUE IMPROVEMENT (LTV)
PROGRAM CHANGE STUDY (LTV)
AVIONICS COST/SCHEDULE STUDY (RCA)
PAYLOAD COST STUDY (RCA)
SPECIFICATION COST STUDY (RCA)
AVIONICS COST/DESIGN SYNTHESIS STUDY (GRC)
SOURCE: NASA JOHNSON SPACE CENTER
INDUSTRY EXPERIENCE AS CONSIDERED BY NASA IN SHUTTLE PROGRAM DEFINITION. HOUSTON,
TEXAS: AUTHOR, JUNE, 1970
LTV
GRC
RCA
i LTV AEROSPACE CORP
| GENERAL RESEARCH CORP
; RCA FEDERAL PRODUCTS DIVISION


68
It had long been felt by NASA based on results of early cost
estimating methodology studies, (later substantiated by a study
performed by RCA for the NASA Johnson Space Center in 1976: RCA,
1976) that the management environment (organizational culture) plays
a large part in determining the magnitude of program cost. To
ensure that the cost data being collected were indeed accurate
analogs for the Space Shuttle, a series of industry interviews was
conducted, which served to provide ideas both on the estimation and
effective control of program costs. Hiis activity, summarized
below, and described in detail in Appendix E, also provided the
opportunity for NASA to evaluate contractor capability to estimate
program costs.
Industry Interviews. A structured set of interviews was
conducted, as shown in Figure 1-17 (see Appendix E), with senior
managers of management teams from the highest technology
aeronautical programs then existing. These included the SR-71
Strategic Reconnaissance aircraft of the United States Air Force
(Kelly Johnson, Program Manager, Lockheed Aircraft Co.); the Boeing
700 series of aircraft (George Schairer, vice president, R&D, Boeing
Aerospace Co.); the Dynasoar (Roy Rotelli, program manager, Boeing
Aerospace Co); the B-58 (Robert Widmer, vice president, General
Dynamics/Ft. Worth), and numerous others, related to these and to
other programs judged to be analogous in some way to the
then-forthcoming Space Shuttle Program. Figure 1-18 is a matrix of
subjects discussed at four of the sites.
Perhaps the most striking result of the activity was the
general consensus in management opinions concerning ways to reduce


FIGURE 1-17.
COMPANY
GRUMMAN
LOCKHEED
MCDONNELL
DOUGLAS
NORTH AMERICAN
BOEING
GENERAL
DYNAMICS
INDUSTRY INTERVIEW PROGRAM (1971)
INTERVIEWS CONDUCTED
PERSON (POSITION) PROGRAMS
REPRESENTED
PROGRAM MANAGER F-14
CHIEF ENGINEER F-14
PRESIDENT LMSC N/A
PROGRAM MANAGER POLARIS/ POSEIDON
CHIEF ENGINEER C-5A
PROGRAM MANAGER S-3A
PROGRAM MANAGER L-1011
PROGRAM MANAGER YF-12/SR-71
COMPTROLLER C-5A
PRESIDENT
PROGRAM MANAGER DC-10
CHIEF ENGINEER DC-10
CHIEF ENGINEER X-15
PROGRAM MANAGER B-70
CONTRACTS MANAGER B-1
PROGRAM MANAGER B-52
MANAGER OF PRICING SRAM
PROGRAM MANAGER GRAND TOUR SPACECRAFT
PROGRAM MANAGER DYNASOAR
PROGRAM MANAGER BOEING 727
CHIEF ENGINEER B-58
PROGRAM MANAGER F-111
CHIEF ENGINEER B-36


FIGURE 1-18. NASA/INDUSTRY PROGRAM DISCUSSIONS
MATRIX OF SUBJECTS FOR DISCUSSION
SUBJECTS DISCUSSED LOCKHEED wo |S o W 5 2 P OO ? C T U 0.0. w w u u BOEING ce ce m CM ^ m w _i Q p i NR o w r* *7 m X B-58 F-111 C5 o B-36
VEHICLE DESCRIPTION X X X X X X X X
PROGRAM DESCRIPTION X X XXX XX
TEST PROGRAM X X X X X XXX
DEVELOPMENT PROBLEMS X XX X X X X X
DEVELOPMENT PLANS X X X X X X X
CORPORATE EXPERIENCE X X X. X
COST ESTIMATING X X X X
ORIGINAL ESTIMATES X X X X X
ESTIMATING METHODS X X X X X X
COST GROWTH X X X X X X X
COST CONTROL METHODS X X X X X X X X
SCHEDULES X X X X X X
ORIGINAL ESTIMATES X X X X X
ESTIMATING METHODS
SCHEDULE GROWTH X X X X X
DATA AVAILABILITY X XXX X XXX X X XXX
PROGRAM INNOVATIONS X XXX X X X X X X


71
costs in government programs, particularly when the findings are
contrasted with the current NASA program management style. That
earlier study (as documented in Appendix E) reached the somewhat
subjective conclusions that, to reduce program costs, according to
the opinions of those interviewed, NASA should:
1. State requirements as objectives, and leave them rela-
tively unconstrained.
2. Not start building flight hardware until all major
technological uncertainties have been resolved.
3. Utilize small, hand-picked government program offices
and contractor teams.
4. Eliminate government-imposed changes.
5. Allow the contractor maximum autonomy.
6. Perform the program in as short a time as possible.
NASA management circles discussed and even agreed to try
many of these potential cost saving cultural differences; however,
the cultural inheritance produced by using many of the same manage-
ment and contractor teams employed by the Apollo program soon
overcame many of the planning ambitions. The original culture was
not appreciably changed, except where it had to be adapted to sur-
vive in the newly cost constrained environment. Appendix E contains
further details of this interview process.
Space Shuttle Cost Estimating Methods Used To Establish the
Agency Committment
In form, the cost estimating methods used for the Space
Shuttle program follow precisely the above description. In order to


72
complete their description, it remains only to describe the elements
which gave them their unique character.
Data Bases. In quantity of data, the methods were generally
quite modest. The cost of the aluminum structure, for example, was
estimated from a single point of data, i.e., the cost of the B-52
airframe. As mentioned, however, the scarcity of data was overcome
by (1) insuring the accuracy and precision of the single data point;
(2) by careful selection of the scaling algorithm; and (3) by se-
lecting a data point analog very close in size to the Shuttle
structure, to avoid the necessity for large extrapolation. Zussman
(1969) is a proponent of this means of estimating, and Waller and
Dwyer (1981) suggest similar alternatives to overcome the problems
associated with imprecise data bases.
Cost Estimators. To insure the proper integration of the
estimating and design processes, cost estimators were chosen for
their knowledge of technical areas. For example, the structural
cost estimator had been a structural design engineer for over 30
years prior to being assigned to the cost estimating team. To in-
sure that the best expertise was made available from across the
agency, an inter-center cost estimating team was formed with members
from both the Johnson and Marshall centers.
Contractor Cost Estimates. As a verification of the NASA
cost estimates, estimates produced by the various NASA study
contractors were employed in a comparative process. Where major
variances occured, they were explained and reconciled.


73
Cost Reserves. In addition to the cost reserves inherently
produced by the "looking-back" parametric process, allowances had to
be made for the residual uncertainties in technical descriptions.
These were accommodated by applying such devices as weight growth
statistics to revise predicted weights to the same "looking-back"
basis. Realistic allowances for program growth were made in all
elements of the estimates.
Schedule Time Estimation. As described previously, an
ancillary methodology was employed to predict program schedule
lengths, in order to provide a realistic time frame over which to
distribute the estimated costs. This schedule estimation
methodology is very similar to that employed for cost estimation,
i.e., employing performance and size parameters to predict develop-
ment time (see Vought Corporation, 1972, for a description of the
methods).
Cost Risk. Hie methods, like their industry counterparts of
the day, did not explicitly compute the risk associated with cost
estimates. Hie issue of risk was treated only by the provision of
adequate cost reserves, as described above.
Cost/Design Synthesis. Hie NASA methods in 1972
incorporated a capability to develop sensitivities to variation in
design inputs, in a closed-loop fashion, using a digital computer
(Ramos, et al., 1973). Based on the methodology of Kelsey and
Sitney (1972) of the Aerospace Corporation, the JSC model allowed
the variation of orbiter and booster sizes, specific inpulse of the
engines, and staging velocities. This capability enabled the


74
development of numerical partial derviatives of cost with respect to
the design variables, and enormously reduced the time required to
estimate the costs of the many design variations being explored in
that time period (Ramos, et al, 1973).
The Model Algorithm. The basic algorithms of the methods
used for Space Shuttle cost estimating are described by Schneider
and Lupo (1967). The method, including its data bases, has been
extensively documented, but never published in its entirety (End
Note 15).
NASA Methods In 1982
Since 1972, there have been several primary improvements in
NASA manned spacecraft cost estimating methods.
Improved Data Bases. Both JSC and MSFC have extensively
expanded their cost estimating data bases since 1972. MSFC has
contracted with Planning Research Corporation of Huntsville,
Alabama, to gather and analyze data from hundreds of program
sources. These are currently stored in a special library facility.
Some of the more frequently used data have been translated to
computer format (Turney, 1982). JSC has developed an extensively
indexed microfische and hard copy data base, containing over 2000
references of cost-related data.
Improved Cost Models. Since 1971, NASA JSC and MSFC have
developed a number of cost models tailored to advanced programs
being studied. The latest is a cost model (also developed by
Planning Research Corporation for NASA) for advanced manned space


75
stations. This model has been programmed on several micro computers
for interactive use (See Turney, 1982).
Cost-Risk Analysis. NASA has adapted several risk analysis
models for use with their cost estimating models. Methods being
used currently are simular to those described by Black and Wilder
(1980).
Overview of the Assessment
History of Space Shuttle Program Cost Estimating
To present the background for this research effort, which
was the evaluation of cost estimating methods, an overview of the
evolution of modern cost estimating methods has been provided in
this chapter. A brief history of the Space Shuttle program, along
with a sketch of the NASA management, program planning, and
budgetary environments were presented to illustrate the forces which
shaped the estimating methods and gave them their unique form and
capabilities. The methods were derived by a small group of
relatively inexperienced estimators at the NASA Manned Spacecraft
Center and the NASA Marshall Space Flight Center, with some
contracted private sector support. Methods were based on Rand
Corporation estimating theory and processes, as originally developed
for the United States Air Force just after World War II to provide
more rationality to the weapon system selection process. Hie
program management environment at NASA was an outgrowth of three
strong cultural heritages, i.e., the research center culture brought
to Houston by the early Langley Research Center Space Task Group,


76
the German rocket development culture brought to Huntsville,
Alabama, by the Werner von Braun organization, and the aerospace
program management culture brought to NASA by a number of senior
program managers hired from the private sector to manage early
manned space flight programs.
Superimposed on these cultural influences was a rapidly
decreasing NASA budget resulting from, the changing national
priorities of the late 1960's, and the manpower constraints which
resulted from the federal civil service system. The result of these
influences was an estimating capability mandated by the decreasing
budget, demanded by those from the industrial management culture,
but understaffed as a result of NASA inexperience with the needs
created by a severely limited budget, and destined to remain
modestly staffed by the inflexibility and relative low salary
structure of the federal civil service. Neither was the necessity
for such a capability fully accepted by some from the research
center culture.
The Space Shuttle was itself a compromise between the
national need for less expensive orbital transportation and the
increasing competition for budget authority, which demanded that a
less capable Shuttle be built than was needed to provide lowest cost
transportation. As the design was iterated, cost estimators were
forced to respond, often with estimates for more than one major
configuration per week. Estimating technology was thus forced to
reach maturity rapidly. The methods which resulted were to
incorporate an integral capability to concurrently re-size and


77
re-cost a spacecraft, a capability not previously available to
NASA. And because of the increased attention paid to cost
estimates, innovative measures were taken to insure their accuracy.
These measures included the development of extensive data bases, and
the incorporation of lessons learned from aerospace industry
managers into the design and use of the methodology. Using these
methods, estimates were made, budgets submitted, and agreements
consumated with the Office of Management and Budget for the runout
costs of the Space Shuttle program. It was agreed that the design,
development, test, and evaluation of the Shuttle would cost $5.15
billion 1971 dollars.
Literature Analysis
As related in Chapter 2, the literature search and analysis
has provided more than background data for the research; i.e., the
literature was used extensively to describe the industry standards
against which the Space Shuttle methods could be compared, as one
basis for the evaluation process. Because of the longitudinal na-
ture of a part of the evaluation process, the literature was divided
into two epochal periods. First, to provide a basis for understand-
ing how the Space Shuttle methods evolved, the literature on cost
estimating from the 1950's to 1972 (the period when Space Shuttle
estimating reached the maximum activity level) was grouped and
analyzed, by functional area (e.g., processes, problem areas, cost
growth, data bases, design synthesis, budget simulation, and risk
analysis).


78
Next, the literature for the decade of the evaluation (1972
to 1982) was compiled and analyzed, again by function, to provide a
measure of the improvement in the estimating state of the art during
that time period. This provided the basis for evaluating the cur-
rent state of the art of NASA methods, as compared again to those of
the industry at large. The literature of space program cost
estimating was found to be relatively sparse. However, when coupled
with documentation related to the parent aircraft and missile indus-
try cost estimating, an extensive basis was found. For a long peri-
od of time, from the 1950's through the 1970's, the literature was
heavily dominated by the work of the Rand Corporation, funded by the
United States Air Force. A distinct chronological trend was found.
In the 1950's, research was based on the large cost data bases com-
piled during World War II, and produced several enduring
contributions to the science of estimating. A learning theory was
developed, along with the general conceptual framework of the
estimation process as it is employed today, and an extensive body of
cost estimating relationships for subsonic aircraft was produced.
Classified work began on ballistic missile cost models.
In the 1960's, early spacecfaft cost models were produced by
NASA, largely from aircraft data bases, which greatly limited their
capability and utility. Aircraft cost models were maturing to in-
clude the capability to estimate the costs of supersonic vehicles
and large transports. Cost growths of research and development pro-
grams in the period remained rather large, but the problem showed
signs of improving toward the end of the decade. And NASA, faced


79
with the ending of the lunar program, developed large program cost
simulation capability to cope with the demands of planning space
programs for the 1970's. By 1972, the NASA capability had reached
the level of maturity demanded by the design of the Space Shuttle,
as discussed above.
The literature of the decade from 1972 to 1982 showed a
decline in the effort devoted to aircraft cost estimating, because
the available data bases had largely been exhausted, and the advent
of data from new programs had slowed appreciably. Budgetary
constraints and the increased costs of aircraft development had
sharply limited new programs from which to derive data. Neither was
there a great deal of improvement in space program cost estimation
capability, except for some notable efforts by NASA to improve the
content of its cost estimating data base. The creative energy of
the cost estimating community was transferred to the improvement of
the methodology in other ways, the most noteworthy one being the
increased capability to quantitatively express the risks associated
with probabilistic cost estimates.
Evaluation Methodology
The following evaluation criteria were utilized:
accomplishment of the purposes originally intended for the
estimating methods (accuracy of outcome and utility in the design
process); effectiveness, in the form of utility to management and
influence on the external environment (mainly the Office of
Management and Budget and the Congress); cost effectiveness; state


80
of the art, as compared to that of industry methods; need for
improvement; and potential for use in future programs.
The methodological approach to this evaluation was rather
complex, involving several components. Included were a longitudinal
study to determine time trends of the state of the art in cost
estimating, a management survey to define the utility, influence,
and need for improvement of the methods, an analysis of actual Space
Shuttle Program costs, and an extensive literature analysis to es-
tablish a basis for comparing the NASA methods to those used in oth-
er parts of the aerospace industry and to determine the future po-
tential uses of the methods. This approach is fully explained in
Chapter 3.
Results of the Evaluation
Results of this evaluation are presented in several parts:
literature analysis, a comparison of actual and predicted program
costs, a comparison of cost growths of other programs with that of
the Space Shuttle, and results of the analysis of the two survey
instruments. As detailed in Chapter 4, the literature analysis
revealed that Space Shuttle cost estimating methods were, in 1972,
representative of the best of the industry state of the art in every
respect except data content. However, the state of the art had
progressed significantly from 1972 to 1982, indicating room for im-
provement of the Space Shuttle methods. Actual costs of the Shuttle
program at completion of development were within 6% of those pre-
dicted by the methods in 1972, a result commensurate with or


81
better that those achieved by the remainder of the industry during
that period. The management survey instruments indicated general
satisfaction by management with the methods, particularly in their
influence on the favorable cost outcome of the program. Managers at
higher levels of the program tended to see accurate cost estimation
as less of an asset than did those at lower program levels.
The survey of professional cost analysts produced data which
showed that the state of the art in cost estimating, as represented
by the problems which existed in 1972 and 1982, had changed -
significantly during the decade. Methodological problems had been
reduced in severity, data problems had remained severe, and cost
problems related to customer-client relationships had greatly
increased in significance.
Summary of Findings, Conclusions, and Recommendations
As related in Chapters 5 and 6, the analysis has found that
the Space Shuttle cost estimating methods generally satisfied their
originally intended purposes, particularly in influencing the design
process and in producing accurate cost estimates to be employed in
budgeting and selling the program. Management found the internal
utility of the methods to be satisfactory, and attributed to the
accurate estimates a major role in producing the generally favorable
cost outcome of the program. The state of the art of the methods
was determined, both from literature analysis and survey of
professional analysts, to be representative of the best of the
industry state of the art in 1972, but needing some improvement if


82
they are to have utility in planning future programs. Cost
effectiveness of the estimating methods was shown to be positive.
Ancillary findings which resulted included a call by manage-
ment for revision of the NASA budgetary process, and a partially
anecdotal finding that those levels of management who participate in
the political processes of the agency view the ability to develop
accurate cost estimates as a questionable asset in the process of
selling a program. It was also found that pressures on an agency
and its contractors to "buy in" to a program have probably increased
in the past ten years, an influence which tends to counteract
estimating accuracy. These latter findings suggest a need to fur-
ther explore the dynamics of the acquisition process for high tech-
nology programs, particularly the willingness of program managers to
assume more risk of underestimating in order to sell a program.
Such a study is recommended.
Appendices
Six appendices are provided. First, a selected
bibliography, organized by subject, is presented as a resource to be
used in future investigations of cost estimating methods (Appendix
A). Second, the two instruments used in this research are
presented, along with the list of managers to whom the management
instrument was sent (Appendix B). Third is presented the raw fre-
quency response data from both survey instruments (Appendix C).
Fourth, the details of the factor analysis of the instruments are
explained (Appendix D). Fifth, details of the industry interview
process employed in the initial design of Space Shuttle cost


83
estimating methods are enumerated (Appendix E). And finally, the
sixth Appendix (F) presents actual costs of the Space Shuttle pro-
gram compared to the initial estimates produced by the methods which
were the subject of this research.
Glossary
Because of the large use of acronyms which is endemic to the
communication process in the aerospace industry, a glossary of
acronyms is presented just after the reference list, which follows
Chapter 6.
The next chapter of this document presents the extensive
literature base for the evaluation.