Citation
An evaluation of the procurement management of a major research and development project

Material Information

Title:
An evaluation of the procurement management of a major research and development project the National Aeronautics and Space Administration's Space Shuttle Orbiter
Creator:
Easley, Robert Eugene
Place of Publication:
Denver, CO
Publisher:
University of Colorado Denver
Publication Date:
Language:
English
Physical Description:
xv, 343 leaves : illustrations ; 29 cm

Subjects

Subjects / Keywords:
Space shuttles ( lcsh )
Armed Forces -- Procurement ( fast )
Management ( fast )
Space shuttles ( fast )
Genre:
bibliography ( marcgt )
theses ( marcgt )
non-fiction ( marcgt )

Notes

Bibliography:
Includes bibliographical references (leaves 269-274).
Thesis:
Submitted in partial fulfillment of the requirements for the degree, Doctor of Public Administration, Graduate School of Public Affairs.
General Note:
School of Public Affairs
Statement of Responsibility:
Robert Eugene Easley.

Record Information

Source Institution:
|University of Colorado Denver
Holding Location:
|Auraria Library
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
16855484 ( OCLC )
ocm16855484
Classification:
LD1190.P86 1984d .E2 ( lcc )

Downloads

This item has the following downloads:


Full Text

PAGE 1

AN EVALUATION OF THE PROCUREMENT MANAGEMENT / OF A MAJOR RESEARCH AND DEVELOPMENT PROJECT: THE NATIONAL AERONAUTICS AND SPACE ADMINISTRATION'S SPACE SHUTTLE ORBITER by Robert .. B.S., Northwestern State University of Louisiana, 1962 M.S., Northwestern State University of Louisiana, 1963 A thesis submitted to the Faculty of .the Graduate School of Public Affairs of the University of Colorado in partial fulfillment of the requirements for the degree of Doctor of Public Administration Graduate School of Public Affairs 1984

PAGE 2

APPROVAL PAGE This thesis for the Doctor of Public Administration degree by Robert Eugene Easley has been approved for the Graduate School of Public Affairs by James L. Neal -Date October 26, 1984

PAGE 3

Copyright by Robert Eugene Easley 1984 All Rights Reserved

PAGE 4

ABSTRACT Easley, Robert Eugene (D.P.A., Public Administration) An Evaluation of theProcurement Management of a Major Research and Development The National Aeronautics and Space Administration's Space Shuttle Orbiter Thesis directed by Professor Jay M. Shafritz Since its inception 26 years ago, the National Aeronautics and Space Administration (NASA) has established the preeminent role in manned spaceflight programs. Projects Mercury, Gemini, Apollo, Skylab, and Shuttle represented significant technological achievement of the entire aerospace community. In March 1981, the reusable Space Shuttle flew for the first time, culminating almost a decade of research and development activity by NASA and its supporting contractors. Billions of dollars were spent for the research and development of the Space Shuttle Orbiter project. This study reviews and evaluates the procurement management of the Shuttle Orbiter design, development, test, and evaluation project from three perspectives: the effectiveness of the award fee type of contract; a comparative evaluation of the cost growth of the Orbiter project; and, the significance of contract changes. This research has concluded that the award fee type of contract significantly contributed to the success of the Shuttle Orbiter development project by providing a contractual arrangement with the necessary flexibility for the government to have formal

PAGE 5

v and informal involvement with the contractor during the contract work. A summary of opinions of government managers involved with the Orbiter project led to the conclusion that the award fee contract contributed to the technical success of the project, and minimized cost growth of the project. Findings from the literature review also supported the use of the award fee type of contract for major development projects. This study found that the single largest cause of cost growth was inadequate and untimely funding of the contract by the government, which caused increases to project cost by $450 million, or 25 percent of the total cost growth. A total of 1,468 contract changes, amounting to $581 million, also significantly contributed to the cost growth. Yet it was found that only a small number of contract changes represented the most substantial portions of the total cost of all changes. Technical uncertainty and development problems were primary causes for the $759 million overrun. Even though total cost growth amounted to $1,790 million, the Orbiter development project's actual costs were less than the estimates that had been committed to by NASA to the Congress and the Office of Management and Budget. The form and content of this abstract are approved. I recommend its publication.

PAGE 6

ACKNOWLEDGEMENTS This study was accomplished through the efforts of many individuals. Sincere thanks are due to the National Aeronautics and Space Administration, Lyndon B. Johnson Space Center, for establishing the project with the University of Colorado under which this study was performed. Special thanks are also due to the many people at the Johnson Space Center who contributed their time and assistance while this study was being conducted. Specifically, I should like to thank the people of the Shuttle Project Office, Shuttle Project Control Office, Shuttle Procurement Office, and Legal Office for their time and effort in responding to the survey instrument. I wish to express my appreciation to Dr. Jay Shafritz, chairman of my committee, for his encouragement and assistance during the preparation of the thesis. Appreciation is also expended to Mr. James L. Neal, Director of Procurement at the Johnson Space Center, for his guidance in conducting this study. Dr. Albert C. Hyde, one of my extremely talented advisors, clearly deserves more credit than can be given here for his tireless efforts and patience in guiding me through this study. A special thanks is due Mrs. Mae Eubank for her professional typing in the preparation of the many drafts of this manuscript and the preparation of the final product. Mrs. Eubank also provided an

PAGE 7

invaluable contribution by devoting many hours in proofreading this thesis. My wife, Judy, is to be praised for her love, encouragement, understanding, and patience the entire doctoral program, and especially support during the conduct of this study. And finally, my children, Bobby and Chris, provided tremendous support and encouragement to me while this study was being conducted. vii

PAGE 8

CONTENTS CHAPTER I RESEARCH AND DEVELOPMENT PROCUREMENT IN NASA............... 1 Procurement Reform....................................... 1 Procurement Issues of the 1960's....................... 3 Procurement Issues of the 1970's....................... 7 Procurement Issues of the 1980's....................... 10 Olt1B Circular A-109 -..................................... 13 The Commission on Government Procurement............... 15 Proposal for a Uniform Federal Procurement System................................... 16 Government Sponsored Resear. ch and Development............ 19 Legal Framework of Federal Procurement................... 24 Regulatory Framework of Federal Procurement............ 28 Methods of Government Procurement 30 Contract Types 33 The Space Shuttle System................................. 38 Procurement Planning for the Space Shuttle Program...................................... 44Program Planning and Contracting Strategy.............. 47 MethOdology.............................................. 56 The Survey Instrument.................................. 59 Content Analysis....................................... 62 Comparative Evaluation of Cost Growth and Changes. . . . . . . . . . . . . . . . . . . 64

PAGE 9

ix Overview. . . . . . . . . . . . . . . . . . . . . . . 65 End Notes. . . . . . . . . . . . . . . . . 70 CHAPTER II. LITERATURE REVIEW. 7 5 The Importance of Contract Type.......................... 75 Origins of the Cost-Plus-Award-Fee Method of Contracting. . . . . . . . . . . . . . . . . . . . 80 The Award Fee Contract as a Method of Management....... 86 Cost Growth.............................................. 92 The Rand Reports....................................... 92 Interview with David Novick............................ 114 The General Accounting Office Reports 116 Case Studies on Cost Growth 124 Other Reference Material on Cost Growth................ 126 Contract Changes ............. 130 Summary of the Literature Review 135 End Notes................................................ 137 CHAPTER III THE AWARD FEE TYPE OF CONTRACT FOR THE ORBITER PROJECT..... 144 The Orbiter Contract Award Fee Plan 144 Analysis of Award Fee Survey Data 150 Strengths and Weaknesses of Award Fee 158 Strengths of Award Fee .......... 158 Weaknesses of Award Fee 166 Survey Respondents Recommended Contract Type 174

PAGE 10

X Summary ............................................ 176 End Notes ............................................ 178 CHAPTER IV COST GROWTH OF THE ORBITER PROJECT 179 Introduction ............................................. 179 Project Cost Compared with Commitments Congress and 01-m 180 Cost Growth Attributable to the Government 183 Contract Changes ..................................... 183 Funding Impacts ........................ 185 Comparison of Orbiter Cost Growth to Cost Performance on Major R&D Programs 187 Cost Ov'errun ................................... 193 Inflation ................................................ 194 Cost Growth Assessments by Survey Respondents 197 Inadequate or Instable Funding 199 Technical Uncertainty/Difficulty 199 Technical Advancement ...... 200 Methods NASA Could Have Used to Reduce Cost Growth 209 SutiDilary 216 End Notes .. 217 CHAPTER V CONTRA.CT CHANGES 218 Introduction .... 218 Impact of Changes .... 219

PAGE 11

xi Changes Threshold 223 Changes with Price Decreases 225 Orbiter Cost Profile ................................... 227 Orbiter Development Milestones 230 Analysis of Significant Changes 230 Orbiter Project Changes Versus Project Lambda 236 SuDJnlary 241 End Notes 242 CHAPTER VI FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS 243 SuDim.ary of Findings ..................................... 246 Award Fee Contracting .................................. 246 Cost Growth of the Orbiter Project 250 Contract Changes ....................................... 254 Conclusions .............................................. 258 Recommendations for Future Research 265 End Notes 268 BIBLIOGRAPHY 269 APPENDIX A. QUESTIONNAIRE FOR GOVERNMENT PERSONNEL, AWARD FEE CONTRACTING ... 275 B. STATISTICAL ANALYSIS OF SURVEY QUESTIONS 1-11 287 C. ANALYSIS OF STRENGTHS OF AWARD FEE CONTRACTING ......... D. ANALYSIS OF WEAKNESSES OF AWARD FEE CONTRACTING ........................... 308

PAGE 12

xii E. ANALYSIS OF TYPE OF CONTRACT PREFERENCE . 313 F. ANALYSIS OF COST GROWTH QUESTIONS . 324 G. MAJOR CAUSES OF COST GROWTH 332 H. METHODS NASA COULD HAVE USED TO REDUCE COST GROWTH OF THE ORBITER PROJECT 337

PAGE 13

FIGURES Figure 1-1 The Packard Initiatives: Major Policy Elements 8 1-2 The Carlucci Recommendations 11 1-3 NASA Budget Appropriation 23 1-4 Federal Acquisition Regulations 31 1-5 Space Shuttle Vehicle 40 1-6 Space Shuttle Vehicle 41 1-7 Orbiter Vehicle...................................... 43 1-8 Space Shuttle Management Relationships............... 48 1-9 Space Shuttle System Development Schedule............ 50 1-10 Space Shuttle Program Procurement Milestones......... 52 2-1 Types of Contracts .................................. 77 2-2 Types of Contracts ....... 79 2-3 Hallmarks of Award Fee for Research and Development Acquisitions......................................... 90 2-4 Cost 1960's Sample 110 2-5 Cost 1970's Sample.................................... 110 2-6 Categories of Cost Growth for 31 Defense Programs.... 111 2-7 The Rand Corporation Studies of Cost Growth in Major U.S. Nondefense Construction Projects 113 3-1 Definition of Adjective Rating Terms for Award Fee Contracts, Shuttle Orbiter Contract 148 3-2 Award Fee Curve, Shuttle Orbiter Contract 151 3-3 Mean and Standard Deviation for Survey Questions through 11. . . . . . . . . . . . . . . . . 153

PAGE 14

xiv 3-4 Strengths of Award Fee Type of Contracts 159 3-5 Summary Analysis of Strengths of Award Fee Contracting .. ............................. ,,........ 160 3-6 Weaknesses of Award Fee Contracting 167 3-7 Summary Analysis of Weanknesses of Award Fee Contracting 168 3-B Recommended Contract Types by Survey Respondents for Various Acquisitions 175 4-1 Initial Space Shuttle Program Estimates for Design, Development, Test, and Evaluation Phase 181 4-2 Orbiter Contract Cost Growth ..... 184 4-3 Percent of Total Cost Growth and Overrun as a Function of Total Cost ......... 188 4-4 Financial Status of Major Federal Acquisitions According to the u.s. General Accounting Office 191 4-5 Target Cost Versus Actual Cost Orbiter DDT&E.......... 195 4-6 Comparison of Contractor Proposed Inflation With Actual Consumer Price Index (CPI) 196 4-7 Survey Respondents Average (Mean) Cost Growth Rating................................................ 198 4-8 Major Causes of Cost Growth for Orbiter Project 202 4-9 Summary Analysis, Major Causes of Cost Growth 203 4-10 Methods NASA Could Have Used to Reduce Cost Growth for Orbiter Project 210 4-11 Summary Analysis, Methods NASA Could Have Used to Reduce Cost Growth of Orbiter Project 211 5-1 Summary of Orbiter Contract Cost Growth 220 5-2 Summary of Contract Changes, Orbiter DDT&E 221 5-3 Summary of Contract Changes With Price Decreases, Orbiter Design, Development, Test, and Evaluation Project ...... 226

PAGE 15

XV 5-4 Cost Profile, Shuttle Orbiter Design, Development, Test, and Evaluation Project 228 5-5 Changes by Calendar Year, Orbiter Design, Development, Test, and Evaluation Project 229 5-6 Space Shuttle Orbiter Development Schedule 231 5-7 Categories of Changes Over $5 Million, Orbiter Design, Development, Test, and Evaluation Project 233 5-8 Comparison of Cost Growth, Orbiter Project Versus Lambda Project. . . . . . . . . . . . . . . . . . . . 238 5-9 Comparison of Contract Changes With Price Increases By Dollar Category, Orbiter Project Versus Lambda Project. . . . . . . . . . . . . . . . . 239

PAGE 16

CHAPTER I RESEARCH AND DEVELOPMENT PROCUREMENT IN NASA Procurement Reform This study is a review and evaluation of the procurement management of the Space Shuttle Orbiter design, development, test, and evaluation project. This research concentrated on the effectiveness of the award fee type of contract, comparative evaluation of the cost growth of the Orbiter project, and the significance of contract changes made during the course of the development activity. The need to improve the process of acquiring major defense and aerospace systems has been suggested by almost every Presidential Administration for the past three decades. From a cost standpoint, the public's perception of the acquisition of major defense systems by the military has been one of great apprehension. Since the 1960's, a similar viewpoint has come to exist for aerospace programs, even though the responsible organization has been a civilian agency, the National Aeronautics and Space Administration (NASA). In essence, aerospace systems are of the same complexity as defense systems; the aerospace industrial community consisting of those same firms as the major defense firms. As would be expected, the strategies and complications

PAGE 17

2 involved in the research and development of major aerospace systems closely parallel those of defense systems. Prior to and especially during World War II, major defense systems consisted mainly of the mass production of weapons that were relatively unsophisticated. Procurement philosophy during that time period emphasized quantity, and was geared primarily to production in numbers of aircraft, tanks, ships, guns, etc. Then the period from 1945-1960 ushered in a new era of technological 1 revolution in defense systems. Some of the more significant developments during this period were high performance (supersonic) jet aircraft, guided missiles with nuclear payloads, and highly complex electronic equipment. Such technological advances led to product line changes with increased specialization among defense contractors. As an example, electronic developments in the 1960's, according to Stekler, became one of the most critical items in many major weapon systems developments, and in numerous cases, repre-2 sented the largest element of cost. Consequently, this dictated that almost every aircraft firm had to develop the capability to integrate the electronics into the airframe in order to be competi-tive for contractor selection. But the above example is only an isolated illustration of the new complexities involving research and development procurement. In order to understand the major issues and problem areas involving procurement of new weapons and aerospace systems, some historical background of the past three decades should be reviewed.

PAGE 18

Procurement Issues of the 1960's During the 1960's, cost growth in major weapons systems became a major concern. Cost growth was of concern not only in terms of contract cost growth, but the spiraling costs of weaponry as well. In dealing with this problem, then Secretary of Defense Robert McNamara embarked on a new course in contracting for major weapon systems. McNamara's primary objectives were two-fold: first, to shift additional risk to contractors by changing the types of contracts; and second, t.o change to the total package 3 procurement concept. Other goals of the Defense Secretary were to place more emphasis on life-cycle cost in making selection of contractors and to move toward a multiyear procurement concept. Life-cycle cost refers to the total cost of acquisition and owner-3 ship, which includes the development, production, and field support of the items. Under the multiyear procurement approach, contracts can be awarded for production programs several years in advance. Total package procurement then encompasses development, production, and post-production support. The government's desire to place more risk and responsi-bility on contractors was accomplished by moving from cost-plus-fixed-fee type of contracts to cost-plus-incentive-fee arrange-ments, and also to firm-fixed-price types of contracts. Under the firm-fixed-price type of contract, the contractor assumes all the risks associated with delivering the product on time, in

PAGE 19

4 accordance with the contract specification and at a stipulated price. Cost-plus-fixed-fee and cost-plus-incentive-fee arrangements provide for the government to reimburse the contractor for all costs that are allowable and allocable to the contract. The cost-plus-fixed-fee type of contract provides for payment of a fixed fee regardless of actual cost, whereas under the incentive arrangement the fee is determined by a formula. It is debatable whether or not this shifting of risks actually reduces the cost of weapon systems; however, there is no question that the defense industrial firms were much more sensitive to cost control by the movement to cost-plus-incentive-fee and firm-fixed-price types of contracts. The total package procurement concept introduced during the McNamara era also represented a very significant change in acquisi-tion strategy. Basically, this method provided for a single fixed-price contract for design, development, manufacture, testing, and support of the system which resulted in very large contracts over long periods of time.4 Therefore, major difficulties .were often times encountered that even well established firms could not handle from a financial standpoint This concept requires the negotiation (between the government and the contractor) of a stipulated fixed-price for the system before the system has been designed. Cost is extremely difficult to estimate accurately before a system has been designed and produced. An example of the problems experienced under this concept was the F-14 Navy fighter 5 aircraft program.

PAGE 20

5 The contractor got into such a financial overrun condition that the Navy had to obtain Congressional approval to make a multimillion dollar advance payment to bail the contractor out. Additionally, the government agreed to let the contractor off-the-hook and renegotiate subsequent prices for future F-14 purchases. One has to ask the question why the government did this? Obviously it was not to the benefit of the government to let this highly capable defense firm go bankrupt. During the 1960's, another major change in acquisition philosophy was the decision by the Department of Defense (DOD) to build a single fighter aircraft suitable for both the Air Force and 6 the Navy. This project, initially named the TFX which stood for tactical fighter aircraft, then later named the F-111, was en-visioned to significantly reduce the research and development (R&D) costs as well as production cost rather than embark on two separate aircraft developments. This was a difficult challenge since Navy fighter aircraft are basically for carrier deployment while Air Force fighter aircraft are for land deployment. The project was plagued with problems--technical problems, design problems (the aircraft weight exceeded the design goal), and cost overrun. Although a few hundred F-lll's were built, the aircraft is primarily operated by the Air Force. The aircraft could never efficiently be adopted for carrier use due to its size and complexities. Still the concept of integrated use from a single acquisition was established.

PAGE 21

6 Ronald Fox, one of the foremost authorities on government procurement, identifies the key McNamara administration innovations as: incentive contracting; formal contractor selection; develop-ment methods for cost estimating and control; systems and value engineering; total package procurement; contract performance 6 evaluation; and program evaluation review technique. A detailed discussion of this concept is not appropriate here, but all repre-sented attempts to bring about changes to the overall procurement system. Another assessment of acquisition problems encountered during the 1960's is provided by Lorette.8 These included increases in unit cost of weapons systems; severity of technical problems; system changes and modifications during development and continuing into production; and spiraling inflation and materials shortage, resulting in fewer systems delivered. All of these problems were influenced greatly by the demands of the Vietnam con-flict. NASA also made some significant changes in its acquisition policies during the 1960's. During the early 1960's period, NASA was in the midst of the development of the Apollo spacecraft, lunar module and a number of supporting systems. The Apollo was, and remains, the largest civilian R&D program ever undertaken by the federal government. Significant actions taken by NASA during the early-to-mid 1960's were the conversion of all major Apollo Program contracts to cost-plus-incentive-fee arrangements, with incentives

PAGE 22

7 on cost, schedule, and technical performance. Although, overall, the incentives were considered to be effective, they did not bring about any real answer to cost growth problems in the development of major R&D systems. Technological advances, especially in the computer and-electronic areas, coupled with program uncertainty, were major contributors to the cost growth. A comprehensive assessment of major causes of cost growth for R&D systems will be 9 dealt with in subsequent Chapter 4. Procurement Issues of the 1970's The basic assumption during the 1960's was that the choices between technical alternatives could typically be made based on the results of design studies. Once a program decision was made to proceed with a program, the development, testing, and production of 10 a system would generally follow the plan. At that time, system acquisition was still considered to be predictable, despite a plague of bad experiences in the form of cost overruns and poor system performance. At the beginning of the 1970's a new Deputy Secretary of Defense, David Packard, was highly concerned about the acquisition process. Secretary Packard's perception of the changes required to improve the acquisition process reflected a different view of the uncertainties involved. Figure 1-1 lists the Packard initiatives 11 contained in DOD Directive 5000.1 of 1971. A major element in Secretary Packard's new policies was the

PAGE 23

Figure 1-1 THE PACKARD INITIATIVES: MAJOR POLICY ELEMENTS FOR MAJOR WEAPONS SYSTEMS ACQUISITION, 1971 8 1. Provide for systematic program reviews at important decision milestones by a group of senior officials in the Office of the Secretary of Defense by establishing the Defense Systems Acquisition Review Council. 2. Improve program cost estimates and provide the Office of the Secretary of Defense (OSD) with an independent source of such estimates by establishing a Cost Analysis Improvement Group. 3. Design to cost: establish a cost goal as one of the primary program objectives, equal to schedule and performance in importance; design with operation and support costs in mind as well as production costs (life cycle costing). 4. -Increase testing objectivity by establishing agencies for operational test and evaluation independent of the Services commands responsible for development of new systems. 5. Improve the training of program managers by establishing military training courses and schools to prepare them for this job. 6. Strengthen the authority of program managers, especially by giving them a clear written charter. 7. Attract superior officers to program management, in part by providing them with superior promotion opportunities. 8. Reduce the turnover rate of program managers so that they have longer job tenure. 9. Resolve technological uncertainties during development, not during production (hence emphasize earlier and more complete hardware testing and reduce "concurrency"--the overlap between development and full-rate production). 10. Encourage competitive hardware developments to reduce risk and stimulate contractor efforts; where feasible, use primecontractor competition through full-scale development to avoid developer monopoly at the time the initial production contract is negotiated. SOURCE: Department of Defense Directive 5000.1, 1971

PAGE 24

9 need for early hardware testing. Testing of hardware increased dramatically in the 1970's, and by 1978 data available at key decision points were much better than what previously existed. Test data were needed to demonstrate the feasibility of the engineering/technical approach prior to making the decision to proceed to full-development and production of the system. There is little question that this policy change brought about a number of improvements in the weapons acquisition process. For example, positive action was taken within the Department of Defense for the first six initiatives and resulted in changes to organizational structure and operating procedures. In addition, program managers had a faster promotion track-record, and longer job tenure was "d 12 ev1. ent, The last of the Packard initiatives, which dealt with competitive hardware development, also elicited a positive response, although not as great as the hardware testing-initiative. Competitive hardware development concept included the awarding of contracts to two firms to develop and test the system to the selection of a firm for the production phase. According to a Rand Corporation study, about 65 percent of the programs that progressed to full-scale development since 1973 involved heavy use of hardware 13 competition. This was a significant change from the 1960's when hardware was seldom competed during the development phase. During the 1970's as NASA moved into the post-Apollo era, large budgets were a thing of the past. Major acquisition policy

PAGE 25

10 changes in NASA during the 1970's included the movement toward the cost-plus-award-fee (CPAF) type of contract as one way to deal with smaller budgets. During the latter part of the 1960's, NASA had begun using CPAF contracts on some sizable efforts, but not for major developments. During the early 1970's all major Shuttle Program contracts were placed on CPAF arrangements. This subjective fee (profit) determination continued all the way through the 1970's and into the 1980's. It is still considered today by NASA to be the preferred type of contract for major system development. Procurement Issues in the 1980's Major concerns during the early 1980's were those of underspending on major weapons systems as part of the Reagan administration's objectives for a large or increased defense budget. In March 1980, Deputy Secretary of Defense Frank Carlucci directed an assessment of the DOD acquisition system with the prime goal of reducing cost, improving the efficiency of the acquisition process, increasing program stability, and reducing acquisition lead-time for weapon systems. Secretary Carlucci's detailed recommendations were set forth in an April 30, 1981, memorandum to all military department secretaries. The major recommendations are 14 shown in Figure 1-2. Although it is too early to tell if the Carlucci initiatives are serving to improve the acquisition process, preliminary results indicate only a few meaningful changes. According to the

PAGE 26

Figure 1-2 THE CARLUCCI RECOMMENDATIONS FOR IMPROVEMENT IN THE WEAPONS ACQUISITION PROCESS, 1981 1. Improve long-range planning to enhance acquisition program stability. 2. More responsibility, authority, and accountability for programs must be delegated to program managers. 3. Strive for more economic production rates. 4. Realistically cost, budget, and fully fund the procurement, logistics, and manpower for major acquisition programs. 5. A strong industrial base is necessary for a strong defense. 6. Readiness and sustainability of deployed weapons are primary objectives and must be considered from the start of weapon system programs. 7. Examine evolutionary alternatives which are a lower risk approach to technology than solutions at the frontier of technology. 8. Reduce acquisition cost by having program funding stability; multi-year procurement; encouraging capital investment by contractors. 9. Shorten the acquisition time. 11 10. Move toward decentralization of the acquisition process to the Services. SOURCE: Department of Defense Memorandum entitled "Improving the Acquisition Process" by Frank C. Carlucci, April 30, 1981.

PAGE 27

August" 1982 Government Executive, the initiatives are attractive but the action is not much better than prior to the initialS tives. There have been improvements in progress payments to contractors, and in increased use of multiyear contracting. An 12 analysis in Government Executive indicated the Carlucci initiatives were not working because actions taken by the Pentagon policy-makers were markedly different from what was being done in Congress 16 and the Office of Management and Budget. Further, there was considerable variances among military field-level project cost estimators and contract managers. Additional will no doubt be needed to judge those initiatives. During the early 1980's NASA made two acquisition policy movements. First, there has been a move toward more CPIF type of contracts for post-development phases of programs, and for produc-tion hardware and spare parts. Second, a movement toward con-solidation of several prime contracts into a single contract at selected NASA Centers has been made. These changes are so new that, as of this research project, there are no data to assess how well this approach is working. The primary thrust behind each of these movements is to move toward a more efficient and less costly operational phase of the Shuttle Program. Other significant procurement reform matters during the 1970's and 1980's were the issuance of Office of Management and Budget (OMB) Circular A-109, the Commission on Government Procurement, and the proposal for a uniform federal procurement

PAGE 28

system. These items will be elaborated on in the following section. OMB Circular A-109 13 In April 1976, the Office of Management and Budget (OMB) issued a new policy for the acquisition of major systems.17 Policies are issued by the OMB in the form of "circulars". These circulars are essentially directives that are binding on the effected government agencies. While the 1976 acquisition policy was applicable to the entire executive branch, its purpose clearly was to effect reforms that would impact on military program cost overruns. This initiative from OMB arose mainly out of controversy of the previous two decades concerning whether new systems were needed and controlling the cost of systems where commitments were made. As a document, OMB Circular A-109 attempts to prescribe a methodology for acquiring major systems in the most efficient and economical way and is concerned with optimizing the entire acquisition process. This new policy directive from OMB required an integrated and systematic approach to establishing mission needs, managing programs, budgeting, and contracting. In addition, A-109 tries to focus the early direction of research and development efforts to satisfy mission needs and goals, attempting to avoid premature commitments to full-scale development and production. This policy also required ongoing communications with

PAGE 29

14 Congress in the acquisition process by relating major systems acquisitions to agency mission needs and goals. Program objectives and realistic cost estimates had not always been apparent when major weapons programs were brought before Congress. An example that illustrates the need for the A-109 concept is the B-1 Bomber Program. According to'a report by the Commission of Government Procurement, nine years and some $250 million had been spent on the B-1 Bomber Program before the Air Force stated 18 the requirement for a low altitude supersonic bomber. Under A-109 the mission analysis would have been expressed in the identification of a need stated in terms of functions to be performed and goals to be achieved. If the Air Force had followed A-109, the mission need statement would have been approved nine years earlier. While it is difficult to determine if more or less controversy would have developed, certainly issues would have been I focused more at an earlier point in development. Although OMB does not place a dollar level on the applica-bility of A-109, it is clear that it only applies to very sizable developments. NASA, for example, has placed a threshold of $100 million development cost for the applicability of the A-109 19 approach. As of this writing, it.is difficult to assess the effectiveness of A-109. In large part, new military and NASA acquisition cycles are of such long duration that the full impact of the A-109 policy cannot be realistically evaluated for some time. Since the basic intent is to spend more resources in the J

PAGE 30

15 early definition phase of the acquisition cycle in order to save money during the development and production phases, only time and future research will test the validity of this concept to the procurement of major systems. The Commission on Government Procurement Thus far, the discussion of procurement reform has focused primarily on executive initiatives. The legislative branch has also played an active role beginning with the creation of the Commission of Government Procurement (COGP) by the Congress in 20 November 1969. The objective of the Commission was to study and recommend to Congress methods to promote the economy, effi-ciency, and effectiveness of procurement by the executive branch. This was the first systematic review of federal government procurement that has been chartered by the Congress since the Hoover Commissions of 1949 and 1955.21 Even then it should be pointed out that the Hoover Commissions did not devote their activities strictly to studying the acquisition process. In fact, the Commission on Government Procurement was the first congressional chartered commission to devote their study solely to the federal procurement process. An overall goal of the COGP was to review and assess the entire federal procurement, process and recommend improvements. Placing emphasis on the total procurement process, in early 1973, the Commission delivered a four volume report with specific

PAGE 31

16 22 recommendations. The cornerstone of those recommendations was that an Office of Federal Procurement Policy (OFPP) be established to provide direction in the development of government-wide procurement policy. As a result of these recommendations, Congress 23 enacted the Office of Federal Procurement Policy Act. This 1974 law established OFPP in the Office of Management and Budget. Authorized initially for five years, OFPP's main charter covers the overall direction of procurement policies, regulations, and proce-dures for the executive branch agencies. The OFPP Act was designed to create a high level focal point for leadership in developing and coordinating executive btanch procurement policy. Among other things, this new office was charged with implementing the COGP's many recommendations for reform in the federal procurement process. In 1979, Congress extended the OFPP for an additional four years, specifically chartering it to develop a uniform procurement 24 system for the federal executive agencies. A key part of that charter was for OFPP to develop a Federal Acquisition Regulation which would replace the Defense Acquisition Regulation, the NASA Procurement Regulation, and the General Services Administration's Federal Procurement Regulation. Proposal for a Uniform Federal Procurement System On February 26, 1982, the OFPP, as part of the OMB, issued its report to the Congress appropriately entitled "Proposal for a 25 Uniform Federal Procurement System." This proposal was

PAGE 32

17 followed shortly (March 17, 1982) by the issuance of Executive Order 12352 entitled "Federal Procurement Reforms."26 The 146 page proposal represented the administration's commitment to the need for procurement reform. The basic features of the proposed federal procurement system have one intertwining objective: to satisfy agency mission needs, with central theme being effective competition in all procurements. Also, there is an amended statutory foundation which sets forth the basis for improved procurement practices and man-dates a readable, straight-forward regulation. The main features of this new system would provide for a streamlined management framework, decentralized agency procurement operations, a profes-sional workforce, meaningful standards for management and opera-tional support, and a system for feedback so that adjustments can 27 be made on a timely basis. One of the major problems with the current federal procure-ment system is that it is not an integrated system. It is a collection of statutes and policies that are often times inconsis-tent and ineffective in satisfying agency mission needs. Account-ability for results is sometimes difficult to determine. As a case in point, the basic procurement statutes for DOD and NASA are the Armed Services Procurement Act of 1947, and the Federal Property and Administrative Services Act of 1949, entailing over 4,000 28 legislative provisions and 64,000 pages of regulations. While

PAGE 33

18 there are many parts of the two basic statutes and other legislative provisions that have little relevance today, agencies, nevertheless, have to deal with them. While the OMB proposal sets forth numerous legislative changes that would be needed to implement the new system, the proposal advocates options for Congress to consolidate into one statute or amend the two existing basic statues. The response of government agencies was predictable. Larger federal agencies favor amendment to the present statutory base. Their primary concerns are the jurisdictional histories within the legislative and executive branches, the perceived problems in satisfying military or other unique agency needs, and the risks associated with enactment of a totally newstatutory base. In any event, either approach would clarify the statutory problems and reduce confusion. The President's Executive Order 12352 of March 1982, activated a series of federal procurement reforms.29 The Executive Order had several primary goals, which included establishing a system in each agency to manage procurement, simplifying the procurement process, developing a professional work force, and increasing the use of competition among firms. The Executive Order requires the Office of Federal Procurement Policy to work jointly with agencies to develop broad policy guidance for the procurement reforms.

PAGE 34

19 Approximately one year after issuance, the GAO reported on the progress of procurement reform under Executive Order 12352. 30 Problems hindering reform implementation included that agency heads were not committed to the reform of procurement, and information from the agencies on the problems in implementing the reforms was not sufficient to make meaningful assessments. Most agencies had a tendency to stress only accomplishments to the President. Another problem found by GAO was the interagency task groups that had been established to design the reforms were experiencing difficulties in meeting their objectives due to conflicts members had with agency priorities. Finally, the executive focal point at each agency did not have decision making authority to carry out its responsi-bilities. The GAO report also recommended that the Congress restore to OFPP jurisdiction over agency procurement regulations 31 pending development of the Uniform Federal Procurement System. With an understanding of the environmental context of the procurement reform issues of the 1960's to 1980's, the next section explores the significance of federal funded R&D activities in NASA, which is the focus of this research effort. Government Sponsored Research and Development Research and development (R&D) is commonly referred to as an innovative process of scientific and technological preparation for change. Government-funded R&D has helped to attain important national goals, particularly in the field of defense, space

PAGE 35

exploration, nuclear energy, and health, and to provide a broad 32 base of scientific knowledge and trained manpower. In the procurement of R&D, the federal government has two basic objec-tives: first, to support the nation's technological base; and second, to acquire the capability for manufacturing new products and rendering new services. It is important to the environmental dimensions of major R&D undertakings. Aerospace procurements, for example, are typically performed in situations with large uncertainty--uncertainty in terms of schedule (time), 20 quality, and cost. As Peck and Scherer have indicated, this is due primarily to the fact that a high percentage of aerospace procurement involves R&D activities, whose outcomes are generally 33 difficult to predict. In many circumstances, established programs of older agencies provide potentially suitable resources to accomplish certain functions. NASA provides a superb example of agency impact on, and cooperation with, other R&D programs from its earliest days, as National Advisory Committee for Aeronautics (NACA) when there was considerable interaction with DOD in-aircraft development. Joint efforts continue today in areas of aircraft 34 development, satellite, and missile R&D. In the fiscal year 1981, the federal government spent approximately $37 billion for R&D. Perhaps even more significant, however R&D now represents 22 percent of that portion of the

PAGE 36

21 total annual budget that is considered "controllable."35 (Uncontrollable budget items are those items established by law and regulation, such as, social security and veterans benefits, federal pensions, etc.) This means that the total controllable budget in 1981 was on the order of $168 billion, thereby making the R&D projects highly vulnerable each year in the annual budget authoriza-tion and appropriation cycles in Congress. Results of major R&D undertakings are normally only available several years after the project gets underway and are further jeopardized by the annual appropriation process in Congress. Notwithstanding the problems inherent with federally spon-sored R&D, the spinoff's from R&D projects have had a significant effect on each person in America and around the globe. For ex-ample, in the early 1960's NASA pioneered the R&D of communication satellites and built a technological foundation for commercial employment of spacecraft as communications relay stations. Inaugu-rated in 1965, commercial satellite systems have grown beyond the most optimistic expectations. Two-thirds of all overseas communications traffic is now carried by satellites, a number of domestic satellite systems are in operation, and there are specialized networks for such purposes as business and marine 36 traffic. This is but one of NASA's many technological transfers to commercial application. The substantial concentration of federally funded R&D in defense and space is aimed at creating final products. The

PAGE 37

22 benefit to be gained from this activity is improved, or higher quantity, or less costly, defense, communication systems, and continued significant technological spinoffs. Many military and space products, materials, and components find their way into other uses with commercial application. For example, the jet engines utilized in commercial aircraft were essentially developed for the military. A large amount of the knowledge (design) and special tooling that goes into commercial aircraft is the product 37 of military R&D. The cost of developing a method to produce penicillin was largely funded by defense R&D funds, as was much of the early work in computers. A more pertinent example to this study is the federal contribution to R&D in NASA's Apollo Program. In 1963, President Kennedy pledged to the Congress to have a lunar landing and return trip to Earth before this decade was out. In conjunction with military needs, this required enormous federal funding. R&D funding increased from $3.4 to $14.7 billion between 1956 and 1964. 38 Notwithstanding the publicity regarding con-tract overruns in major federal system acquisition, the public had started to look for benefits from space and military programs. As a civilian agency, NASA makes a sizable contribution to the total federal R&D activity. For example, during 1984 NASA total R&D expenditures are projected to be $5.7 billion, or 81 percent of NASA's total budget of $7.1 billion. NASA has consistently spent approximately 80 percent of its budget on R&D activities. Figure 1-3 depicts the NASA budget since inception

PAGE 38

Fiscal Year 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 Transition Period 1977 1978 1979 1980 1981 1982 1983 1984 Figure 1-3 NASA BUDGET APPROPRIATION 1959-1984 (IN MILLIONS) Total Appropriations $369 485 967 1,825 3,674 5,100 5,250 5,175 4,968 4,589 3,995 3,749 3,313 3,310 3,407 3,040 3,231 3,552 933 3,819 4,064 4,559 5,243 5,523 5,940 6,838 7,178 23 NOTE: Fiscal years 1959 through 1976 were from July 1 through June 30. Fiscal year 1977 and subsequent covered from October 1 through September 30. Transition period covered July 1, 1976 through September 30, 1976'. SOURCE: Comptrollers Office, NASA Headquarters, Washington, D.C., August 11,1983.

PAGE 39

24 through 1984. During the period 1963 through 1968, NASA was allocating a significant portion of its budget to the Apollo (lunar landing) Program. The first lunar landing occurred in 1969, and sizable reductions in budget appropriations began. From 1969 through 1977 the NASA budget was essentially level; however, after factoring in the effects of inflation, the budget was actually much smaller than what the agency had experienced during early times, The budgetary environment is only one dimension to view the R&D procurement problem. Another viewpoint is provided through the statutory and regulatory framework of federal procurement. Legal Framework of Federal Procurement Contracts of government organizations are not legal obligations of the United States unless they result from the proper exercise of validly conferred contracting powers. The process of determining whether a contract is legally binding on the federal government involves the consideration of a complex statutory framework covering both the authority to enter into contracts and the procedures by which the contracts are awarded. General contracting authority is constrained in the legislation creating an agency, while specific programs are authorized by "Authorization Acts," and funds for authorized programs are furnished by 39 "Appropriation Acts."

PAGE 40

25 A review of the historical development of federal procure-ment policies indicates that the most significant changes have occurred during and soon after periods of large-scale military activity. As summarized by the 1978 report of the Committee on Government Affairs, the first law regulating federal procurement was passed by the Second Congress in 1792, and required the Treasury Department to make all Army purchases, followed in 1861 when Congress passed a law requiring advertising for all government purchases, except in matters of public exigency, with the objective of maximizing competition and accountability.40 Except for during periods of war, the 1861 statute, as amended in 1910, applied to the military departments and to GSA until and 1949, respectively. From the late 1800's to the late 1940's, the major-ity of the military purchases were high volume with low unit costs (e.g., guns, ammunition, unsophisticated aircraft, etc.). Beginning with the advent of jet aircraft and missiles in the 1950's, procurement of weapons systems shifted dramatically to low volume, high impact, with high unit costs. During both World War I and World War II, the government found normal procurement procedures (advertising) too inflexible to mobilize, coordinate and utilize the resources necessary for the 41 defense efforts. Negotiated procurement procedures had to be used extensively during each of the wars. At the end of World War I, there was a return to formal advertising and peacetime

PAGE 41

procedures. This period also marked the government's first substantial involvement in the procurement of technological products from the growing aviation industry. Post World War I revelations of excessive wartime profiteering led to a ban on the cost-plus-a-percentage-of-cost (CPPC) contracting method and the prohibition of agreements for contingent fees to curb influence 42 peddling. While the CPPC method provided for an arrangement whereby the government would pay the contractor a profit based on an agreed to percent of actual cost, that arrangement obviously 26 discouraged low expenditures. As stated in OMB's 1982 report, the Vinson-Trammel Act of 1934 imposed profit limitations on contracts for aircraft and naval ships, and the Renegotiation Law of 1941 allowed the government to renegotiate certain contracts to eliminate excessive profits.43 As a result of the experiences of World War II there was a need to revamp the statutory authorities for federal military procurement. The Armed Services Procurement Act (ASPA) of 1947, which still is in effect today, governs the procurement systems of 44 the DOD, the Coast Guard, and the NASA. Shortly after passage of the ASPA, the Federal Property and Administrative Services Act (FPASA) of 1949 was enacted and delineated the legal framework for civilian such as the General Services Administration.45 While there are currently (in 1983) over 4,000 legislative provi-sions which directly affect the procurement process, the 1947 Act and the 1949 Act remain as the basic statutory framework of

PAGE 42

government contracting and the authority for federal agencies to 46 issue regulations. 27 As a result of the increased use of cost-reimbursement type contracts during the 1950's, the federal government became con-cerned over the nature and efficiency of the controls of cost. In 1962, Public Law 87-653, known as the Truth in Negotiations Act, was passed as an amendment to the Armed Services Procurement Act to strengthen safeguards and clarify procedures pertaining to negotia-47 ted procurements by DOD. The Truth in Negotiations Act re-quired defense and NASA contractors for negotiated contracts in excess of $100,000, to disclose current, accurate, and complete cost or pricing data and allowed for price reductions after contract award if defective data had been submitted to the govern-ment. Even though the procurement regulations for DOD and NASA had required essentially the same type data, the requirement now had a statutory base, thereby providing a means to take un-cooperative offenders into federal court. The federal procurement process has been used to further the socio-economic goals of the federal government. Originating in the depression years the included such federal laws as the Davis-Bacon Act, setting minimum wages on federal construction con-tracts; and the Copeland Act, which prevented salary kickbacks in federal construction work. Since World War II, efforts to use the federal acquisition process to promote socio-economic goals have

PAGE 43

28 continued in a number of areas. Small business and labor surplus areas assistance and preference programs have been intensified, since creation of the Small Business Administration was created in 1955. Additional programs fostered by the federal procurement process include health and safety regulations, hardcore unemploy-ment, minority enterprises, wage and price controls, environmental pollution, distribution of federal work, and discrimination against women and the aged. The two basic procurement statutes--the Armed Services Procurement Act of 1947 and the Federal Property and Administrative Act of 1949--are over 30 years old, have been independently amended several times and there are numerous differences between them, 48 according to the Commission on Government Procurement. Al-though some of the inconsistencies stem from special problems originally caused by only one or a limited number of agencies, most of the differences develop simply because there are two basic procurement statutes and each is amended at different times in different ways. But the statutory framework of federal procure-ment, is only one part of the process. One must also deal with the regulatory framework. Regulatory Framework of Federal Procurement The procurement regulatory system has a major effect on government procurement. Nash and Cibinic have found that the most significant impact on the procurement process is made by procurement

PAGE 44

29 49 regulations. Presidential executive orders are considered part of the procurement regulatory network. These orders are, many times, pertinent to the acquisition process and have the effect and force of law if they are issued pursuant to a direction or delega-tion of authority from Congress. Regulations issued by other agencies are sometimes applicable to procurement. For example, regulations by other government organizations such as the Department of Labor and the Small Business Administration may be applicable. Nearly all government agencies with a sizable amount of procurement activity have adopted a set of procurement regula-tions. These regulations contain the prescribed policies to be followed by procurement personnel in the acquisition of goods and services. The two major sets of regulations are the Federal Procurement Regulations (FRR) promulgated by the General Services Administration for civilian agencies, and the Defense Acquisition 50 Regulations (DAR) for military departments. Although NASA's procurement procedures are authorized under the Armed Services Procurement Act, NASA has its own set of regulations and is not subject to the DAR. However, it must be pointed out that there are no substantial differences between NASA's regulations and the DAR. Effective April 1, 1984, the government-wide Federal Acquisition Regulations a of the 1980's reform by the Office of Federal Procurement Policy, replaced the Defense Acquisition Regulations, the NASA Procurement Regulations, and the Federal Procurement Regulations. The FAR represents a major step

PAGE 45

30 toward a uniform federal procurement system. For example, all government contracts will contain consistent clauses, terms, and other general provisions. To illustrate the scope and comprehen-siveness of FAR areas of regulation, the table of contents for the FAR is shown in Figure 1-4. While each government agency utilizes this basic set of procurement regulations, many agencies have issued supplemental procurement regulations to deal with procure-ment issues unique to the particular agency. After placing the statutory and regulatory framework in perspective, the methods of government procurement will be reviewed in the following section. Methods of Government Procurement Two principal methods of procurement, formal advertising and negotiation, are employed by the federal government. Formal advertising is a sealed bid competitive system employed by the federal government to select contractors when conditions warrant. The system is structured to obtain the most advantageous contract for the government by maximizing free and open competition; preventing favoritism, collusion or fraud; and giving all 51 interested parties an opportunity to bid. The detailed and highly technical rules which characterize the system have gone through a long evolutionary period. As a method of formal advertising procurement works best with items where firm specifics-tions exist, e.g., automobiles, office equipment, etc. Under NASA Procurement Regulations, as well as the Defense

PAGE 46

Figure 1-4 FEDERAL ACQUISITION REGULATION TABLE OF CONTENTS SDICIW'TEI A CEIIEIAL Part l Federal Aequbltlon Regulation System Port 2 DellnU lono of llordo and teno 1984 Port ) laproper luelneeo Praettcu and Penonal Confllcta of Interest Port 4 Adlnlotrattvo Hatten Part Publlclrln& Contract AcUona Pert 6 (Ruarvedl SUICIW'TER I ACQUlS ltlOH PLAJINIIIG Pert 7 Plenntn1 Pert 8 Required Sourcea of Suppllu and Servlcu Part 9 Contractor Quallflutlonl Part lO Speclflcotlono, Sundordo, and Othor Purchuu Ducrlptlonl Part ll Acquloltlon and Dlatr I bull on of Co-trclal Producu Part U Contract Dellverp or Perlonance SUICHAPTEI C CONTRACTING METHODS AND CONTRACT TYPES Part U Small Purchoae and Other SJpiUlod Part 14 Porcel AdverUalnl Part U Contrectlna by Neaothtlon Part 16 typn of Contracu hrt 11 Speclel Contucuna "rhoda Part 18 -(Ruervooll SUICIWTI!R D SOCIOECONOMIC PROCIWIS Part 19 Saall luelneoa and S11al1 IIJoodvanuaed Bualntll Concnno Port 30 L.bor Surplua Area Concern Port 21 (Ruervedl Part U Application of Labor Lava to Governeent Acquhltlono Port Zl lnvlronaont ConaorvaUon, and Occupetlonal Safety Part 2 Protection of Privacy and freedom of lnformatlon Part 2S Porol&n AcqulaiUon Part 26 (laoervrdl SOURCE: Federal Acquisition Regulation, 1984 SUICKAnEI I GENERAL CONTRACTING lEQUIRDWITS Port 27 Patentl, llota, and CopJrlshto Port 28 Iondo ancl lnturonce Pan 29Tueo Port 30 Coot AccounUns Stondordo Put ]I Contract Coat Prfnclpleo and Proceduroo Port 32 Contract Flunclna Part ]] Dhpuuo ond Appuh SUBCIIAPTER F SPECIAL CATEGORIES OP COHTRACTINC Port 34 MaJor Syote11 Acquloltlon Part l5 Ru .. rch ond Dovrlopmont Controctlna Part 36 -Conotructlon and Archltoct-Enstneorlna Controcu Port 31 Servl
PAGE 47

32 Acquisition Regulation, formal advertising must be used except when it is impractical or when the program is covered by one of 17 exceptions--research and development is one exception; sole source 52 (noncompetitive firms) is another. Prior to World War II, procurement by means other than formal advertising was seldom used. Under the provisions of the First War Power Act, passed in December 1941, the executive branch was authorized to suspend the previous statutory requirements of formal advertising.53 In negotiated procurements, the government is required to obtain competition to the maximum extent practical. The negotia-tion method offers flexibility, adaptability, and speed. It is simply not practical for many government procurements to be con-ducted through formal advertising. Contracts for R&D programs, such as the Shuttle Orbiter, are awarded through negotiation because the government needs to evaluate contractor's technical capabilities, technical approaches, and management ability, as well as costs. Procurement by negotiation allows the government to select the contractor who seems best prepared to meet all the requirements of the program, and to meet them on the terms most satisfactory to the government. The advantages of the negotiation method compared to the advertised procurement method are extremely important for major systems acquisition. It would be almost impossible to advertise effectively for sealed bids for major aerospace and weapons system. Since World War II, procurement by the negotiation method

PAGE 48

has been the rule rather than the exception from a practical standpoint. In the last few years, the General Accounting Office has found that 85 to 90 percent of the federal government's needs are satisfied through the negotiated procurement process.54 In NASA, primarily due to the R&D nature of the NASA mission, the negotiated procurement method is utilized for over 90 percent of the entire procurement budget. Contract Types 33 One of the most significant aspects of procurement is the selection of an appropriate type of contract. Principally, the government employs two types of contracts--fixed-price and costreimbursement. However, there are several variations of these two types of contracts, each type differing according to the degree of responsibility and risks assumed by the contractor. In advertised procurements, some form of a firm fixed-price type of contract is used since the specifications are definite and competition is present. In negotiated procurements, the contract type, while selected by the government, is subject to negotiation and may be changed to facilitate price negotiation. The firmfixed-price or lump sum contract type places the total risk of performance on the contractor, while the cost-plus-a-fixed-fee-type contract, at the other extreme, places the cost or performance risk on the government with the contractor receiving more or

PAGE 49

34 55 less a guaranteed fee. In between these two broad types of contracts are various incentive contracts in which the contractor's responsibility for the performance costs and the profit or fee incentives offered are tailored to the uncertainties involved in contract performance. The major types of contracts utilized by the government are as defined and discussed below.56 1. Firm Fixed-Price. This type of contract provides for a price which is not subject to any adjustment by reason of the cost experience of the contractor in the performance of the contract. The risk of performance falls on the contractor. This type of contract is normally used when firm specifications for the product exist with known cost history of the product. 2. Fixed-Price Contract with Escalation. This of type contract provides for the upward and downward revision of the stated contract price upon the occurrence of certain contingencies which are specifically defined in the contract. This type of contract, seldom used by NASA but frequently by the Department of Defense, is typically used where material prices could fluctuate widely on the market. 3. Fixed-Price Incentive Contracts. This type of contract is characterized by an adjustment formula in the contract which relates to the efficiency of the contractor. Target profit and target cost are negotiated, along with a profit formula. The contractor's profit increases or decreases according to the formula

PAGE 50

35 as the actual costs are less or more, respectively, .than the target cost. The fixed-price incentive (FPI) contract is distinguished from the cost incentive contract by the inclusion of a ceiling price. Costs in excess of the ceiling price are born entirely by the contractor. Contracts of this nature are normally used for large production quantities of aircraft and other systems after the initial production of the hardware. 4. Cost Contract. With this type of contract, the contractor is reimbursed for allowable costs only and receives no fee or profit. This type of contract is t}1'ically used when contracting with universities for research. 5. Cost-Sharing Contract. Under this type of contract, the contractor receives no fee and is reimbursed for only a portion of costs. This type of contract is used where the benefits of a research and development contract accrue to both parties. 6. Cost-Plus-Incentive-Fee. This type of contract is similar to the fixed-price incentive contract, except that there is no ceiling price. The CPIF contract is a cost-reimbursement type contract with a provision for a fee which is adjusted by formula in accordance with the relationship which total allowable cost bare to target costs. This type of contract is often used for initial hardware modification after development. 7. Cost-Plus-a-Fixed-Fee. The contractor receives a set fee and is reimbursed for all costs allowable under established

PAGE 51

36 cost principles. This type of contract is used for basic research, routine supports with fluctuating levels of staffing, and other procurements with a high degree of uncertainty as to expected actual cost. 8. Cost-Plus-Award-Fee. The CPAF contract is a cost-reimbursement type of contract providing for a base fee, and for the potential award of additional fee based upon periodic, judgmental evaluations of contractor performance by the government. CPAF contracting is extensively employed in NASA and DOD for major R&D developments. 9. Time and Materials Contract. This type of contract is used for acquiring supplies or services on the basis of direct labor hours at specified fixed hourly rates and materials at' cost. This contract type is often employed for overhaul and maintenance of equipment and aircraft. Differences in the technical spectrum from the "impossible to define" to the precisely defined end product is matched to an appropriate contract type. Matching of contract form and financial risk is very critical. When the risk element is not properly recognized severe problems are often created in reconciling contractual commitments withrealities.57 While it is the government's responsibility to select the appropriate contract type, the terms and conditions are subject to negotiation. Under a cost-reimbursement contract, the contractor

PAGE 52

37 promises to try to meet the performance requirements or goals of the contract within the negotiated schedule and estimated cost. This arrangement provides for reimbursement of the contractor's cost and payment of profit (referred to as "fee"). Under a fixed-price contract, the contractor guarantees performance of the terms of the contract, and in exchange for the guarantee, the government is obligated to pay a specific price. Under a arrangement, the financial risks are born by the contractor, whereas under a cost-reimbursement contract, the risk is born by the government. The government normally uses a cost-reimbursement contract when the magnitude of the uncertainties in the work to be performed precludes the use of an acceptable fixed-price arrangement. This is typically the situation in R&D procurement for aerospace systems. Under cost-reimbursement contracts, the government is obligated to reimburse the contractor for all costs that are allowable and allocable to the contract. Allowable costs are 58 defined in Part 15 of the NASA Procurement Regulations. Usually, unallowable costs are such items as interest expense, advertising, entertainment, contributions, and bad debts. A cost-reimbursement contract includes, however, a cost limitation beyond which the contractor will not be reimbursed, and beyond which they need not continue to work (if the government does not provide additional funds). The Shuttle Orbiter contract is a cost-plus-award-fee

PAGE 53

38 (CPAF) arrangement. The CPAF contract is a cost-reimbursement type of contract, with special incentive fee provisions. Under this type of contract, the contractor's earned fee is determined subjec-tively (usually each six months) by designated, after-the-fact evaluations of the contractor's cost and technical performance. Unique features of NASA's use of the CPAF contract is that the entire award fee is determined subjectively by a small group of evaluators, the contractor can make logical tradeoffs between technical performance and cost performance and, in the end, not be penalized for cost growth if such growth is justified. This would not be the case under objective contract incentives. A major part of this study will be devoted to the CPAF type of contract and an ) evaluation of the effectiveness of the award fee arrangement on the performance by the Shuttle Orbiter contractor. At this point, it is appropriate to discuss the Space Shuttle system prior to discussing the procurement planning for the Shuttle Program. The Space Shuttle System The Space Shuttle system is being developed by the National Aeronautics and Space Administration (NASA) to achieve for the nation the capability for routine space operations in the 1980's and beyond. The Space Shuttle Transportation System consists of a reusable Orbiter, an expendable external propellant tank, two refurbishable solid rocket boosters, and the ground-based

PAGE 54

39 facilities necessary for maintenance, turnaround (prepare for relaunch), and launch of the system. Spaceflight operations include cooperative NASA and DOD use of NASA's Kennedy Space Center (KSC) in Florida and Vandenberg Air Force Base (VAFB) in California for launch and recover sites. Figure 1-5 provides an illustration of the Space Shuttle system. Figure 1-6 depicts the specific elements and size of the Space Shuttle system. The main program objective of the Space Shuttle system is to establish a national space transportation capability that will substantially reduce the cost of space operations and provide a capacity designed to support a wide range of scientific applications, defense, commercial, and international uses.59 Except for the Shuttle, all spacecraft launch vehicles are expendable and cannot be reused. The Shuttle can deliver payloads up to 65,000 pounds to 200 nautical miles circular orbit at a substantially lower operational cost per flight than, for example, the Titan rocket system, which has a payload delivery capability of only 30,000 pounds. In addition, the Space Shuttle can return to earth with up to 32,000 pounds of payload for reuse or evaluation, a capability not provided by expendable launch vehicles. Space Shuttle operational capabilities and flexibility permit the-onorbit replacement and/or servicing of malfunctioning satellites, reducing out-of-service time, and increasing operational availability. Flight plans and operations of the Space Shuttle differ

PAGE 55

Figure 1-5 SPACE SHUTTLE VEHICLE SOURCE: Office of Shuttle Procurement Johnson Space Center National Aeronautics and Space Administration May 1983 0

PAGE 56

ORBITER 161K LB (INERT) 225K LB (LIFT-OFF) ET. 1651K LB (LIFT-OFF) SRB 2587K LB (LifT-OFF) CROSS LIFT-OFF WEIGHT LB 32K LB PAYLOAD TO 50 X JOO NMI AT DEG INCLINATION Figure 1-6 SPACE. SIWTTLE VEHICLE Office.of Shuttle Procurement Johnson Space Center OVERALL LENGTH (ET + SRB) 18),8 FT OVERALL WIDTH (WING SPAN) .78.1 OVERALL (ET +ORBITER) FT National Aeronautics and Space Administration May 1983 .......

PAGE 57

42 markedly from that of the now familiar launch procedures and splashdown of the Apollo missions, which utilized the expendable Saturn-V launch vehicle. Space Shuttle vehicles provide a relatively comfortable environment for a nominal flight crew of four plus up to three payload specialists. In an emergency, an additional three persons can also be accommodated for. Such capability enables experienced scientists and technicians to accompany their payloads into space. Orbiter spacecraft contain the crew and payload for the Space Shuttle vehicle system. Capable of delivering payloads up to 65,000 pounds, with lengths up to 60 feet and diameters to 15 feet, the Orbiter is comparable in size and weight to a DC-9 passenger jet. It has a dry (without payloads) weight of approximately 160,000 pounds, a length of 122 feet, and a wing span of 78 feet. Figure 1-7 illustrates the Orbiter. NASA's Johnson Space Center (JSC) in Houston, Texas is responsible for the development of the Orbiter. NASA's Marshall Space Flight Center in Huntsville, Alabama is responsible for the development of the Orbiter's main rocket engines, the external propellant tank, and the solid rocket boosters. The most complex and costly of all the Shuttle system elements was the Orbiter, with the rocket engine the second from a difficulty and price standpoint. While the external tank was the least in complexity, the rocket booster did present some development challenges but not as

PAGE 58

Figure 1-7 ORBITER VEHICLE SOURCE: Office of Shuttle Procurement Johnson Space Center National Aeronautics and Space Administration May 1983 w

PAGE 59

44 great as the Orbiter and engine developments. This study concerns itself only with the Orbiter vehicle. The following section discusses the procurement planning for the Space Shuttle Procurement Planning for the Space Shuttle Program NASA conducted extensive studies of the feasibility of the Space Shuttle system during the late 1960's preceding the actual procurement of the various Shuttle system elements. Initial studies were of a conceptual nature and were performed mainly in-house by the government with limited public supported industrial effort. Results from the feasibility studies, coupled with the growing interest by NASA and the Department of Defense to develop a reuseable launch vehicle, confirmed the need to proceed with the preliminary design definition and program cost estimates for a Space Shuttle system. In February 1970, NASA issued a competitive Request for Proposal (RFP) to industry for preliminary design definition studies of a reusable Shuttle system.60 This procurement action resulted in the award of two parallel contracts in June 1970. Contracts were awarded on a firm-fixed price basis to Rockwell International and McDonnell Douglas. While the Johnson Space Center (JSC) in Houston, Texas managed the Rockwell contract, the Marshall Space Flight Center (MSFC) in Huntsville, Alabama managed the McDonnell Douglas contract. In addition to the two design definition contracts, three contracts for studies of alternate

PAGE 60

45 Shuttle vehicle configurations were awarded to and Chrysler. All five of the contracts. were funded through March 15, 1972. On March 16, 1972, the procurement plan for the acquisition of the Shuttle Orbiter spacecraft and the Shuttle systems integration effort was approved. A request for proposal was released on March 17, 1972, on a fully competitive basis even though the request for proposal was initially sent only to the above five firms plus Boeing, Martin Marietta, and General Dynamics. However, any firm could have submitted a proposal which would have been fully considered by the NASA. In preparation of the proposals, all interested firms had access to the data resulting from the preliminary definition studies conducted by the five firms. The data consisted primarily of the final contract documentation submitted at the conclusion of the studies. Proposals in response to the RFP were received from Grumman, McDonnell Douglas, Lockheed, and Rockwell International. In this case, the NASA source evaluation board had teams of people evaluating the technical, maintainability and ground operations, and cost areas. In all, a total of 416 people representing seven NASA centers, NASA Headquarters, and the U.S. Air Force participated in the evaluation. A source evaluation board consists of a group of government personnel knowledgeable in the technical field of the product being bought, with cost analysts, procurement personnel, and a legal consultant. In July 1972, the Space Division of Rockwell International

PAGE 61

46 was selected as the prime contractor for the Shuttle Orbiter project and Shuttle systems integration effort. Integration effort, which means the activity associated for certifying the total Shuttle system (Orbiter, engines, and rockets) is ready for flights dealt with all the analyses and interfaces required to verify that the total Shuttle system would work. This activity, amounting to approximately $400 million, spanned the entire Shuttle Program and entailed that effort necessary to integrate all Shuttle elements in to the Space Shuttle system. Actual attaching of the rocket boosters and external tank to the Orbiter was the responsibility of the Kennedy Space Center in Florida (the launch site). While this research project deals with the development phase of the Orbiter project, which amounted to approximately $4.5 billion, it is important to place the Orbiter procurement in perspective with other Shuttle system elements. The Space Shuttle system flight hardware was contracted for by awarding four separate prime contracts. These were the Orbiter spacecraft, the main engines, the external tank, and the solid rocket boosters. The acquisition of the Orbiter was the responsibility of the Johnson Space Center while the other three elements were procured and managed by Marshall Space Flight Center. The first element to be placed under contract was the main engine project. Approximately one year prior to the award of the contract for the Shuttle Orbiter, the Rocketdyne Division of Rockwell International was selected as the main engine contractor.

PAGE 62

47 Development of jet and rocket engines typically require longer periods of time than that required-for the development of the respective aircraft and spacecraft. This is due, in large part, to the extensive development and testing time required for engines of that type. Program Planning and Contracting Strategy As mentioned, the Johnson Space Center was selected as the lead Center for the total Shuttle Program. Reporting to the Space Shuttle Program Director at NASA Headquarters in Washington, D.C. from a programmatic standpoint, the JSC Program Manager reported to the JSC Center Director from an institutional standpoint. Figure 1-8 indicates these reporting relationships which are essentially a matrix management concept whereby, the program and project managers are employees of the respective centers but take program direction directly from NASA Headquarters and the JSC Program Manager, respectively. Designation of a lead Center was a major policy change from prior major NASA R&D Programs such as Gemini and Apollo. During the Gemini and Apollo Programs several NASA centers were responsible for various elements, such as development of spacecraft and boosters; but overall total program management and integration resided with a program director at NASA Headquarters. While the program manager is located at JSC, there are project offices responsible for various program elements located at

PAGE 63

Figure 1-8 SPACE SHUTTLE MANAGEMENT RELATIONSHIPS, 1972 COMPTROLLER AA/MSF SPliCE SIIUTTLE OFFICE OF fACILITIES PROGIIIIM IIIRCTOR 1 OIRCTOR .. 1 JSC --------f I SPliCE SIIUTTI rROGIIIIM _MANAGER _I [ ___ .I __ SCIIULS OrERATIONAL INTEGRA 1 --SPACE SIIUTTLE MAIN ENGINE ) SOLID ROCKET BOOSTER (SRil) OROIHR PROJECT E.XHRNAL TANK (ll) ---PROGRAMMATIC RELATIONSHIPS --INSTITUTIONAL RELATIONSHirS STAFFED BY JSC. MSFC & KSC PERSONNEL CXl

PAGE 64

49 Marshall Space Flight Center (MSFC) and Kennedy Space Center (KSC), as well as JSC. Actual procurement responsibility resided with each respective center for their designated projects. In JSC's case, the Shuttle Program Manager was responsible for carrying out all functions related to the management of all elements of the Space Shuttle. These functions included the detailed planning and scheduling for the entire design, development, and production operation effort, as well as the technical guidance and system integration for the entire Space Shuttle system. Another major responsibility of the JSC Program Manager included the submission to NASA Headquarters of a total Shuttle Program integrated budget covering all program elements. Project managers at both centers were responsible for the design and development of their respective project, including management of their contracts. Project managers were responsible for the establishment of performance, resource, schedule, and configuration control of their respective project, and supported the Shuttle Program Manager. The development schedule for the Space Shuttle system; as set forth in the RFP, is shown in Figure 1-9. This schedule highlights the lengthy period of time required for development of major R&D systems. The first manned flight occurred in March 1981, three years later than the original planned date of March 1978. This schedule slip was due mainly to inadequate funding in the late

PAGE 65

Figure 1-9 SPACE SHUTTLE SYSTEM DEVELOPMENT SCHEDULE 1 1972 1 t973 1 197 4 .1 1975 11976 1 1977 1 t97a 1 1979 1 19ao 119e1-l 1982 1 f::tATP {::; PRR f;{SRR _,PDRS /CDRS -FHF FMOF 0 I MAR 78 I UORIZONTAL TEST FLIGHTS LEGEND ATP-AUTHORITY TO PROCEED I NASA TRAFFIC MODEL RATE ( FLIGHTS PER YEAR ) 1ST YR 6 2ND YR IS JRD YR 24 4TH YR J2 5TH YR -40 6TH YR 60 PRR -PROGRAM REQUIREMENTS REVIEW SRR-SYSTEM REQUIREMENTS REVIEW PDRS -PRELIMINARY DESIGN REVIEWS CDRS -CRITICAL DESIGN REVIEWS VERTICAL TEST FLIGHTS OPERATIONAL FHFFIRST HORIZONTAL FLIGHT FMOF FIRST MANNED ORBITAL FLIGHT SOURCE: Office of Shuttle Procurement Johnson Space Center National Aeronautics and Space Administration March 1972 1./1 0

PAGE 66

51 1970's and development problems with the thermal protection system (i.e., the heat resistant tiles needed to protect the space vehicle upon reentry into the earth's atmosphere). Consistent with the approved procurement plan, NASA did not place the entire program under contract initially. Due to NASA's inability to define precisely the complete requirements in sufficient detail to permit definitive pricing and negotiating, NASA elected to structure an incremental approach to the acquisition of the Orbiter project. Thus, the contract was divided into three phases: (1) design, development, test, and evaluation (DDT&E); (2) production; and (3) the operational phase. During the DDT&E phase, the Orbiter was designed, tested, and two spacecraft subsequent delivered. The production phase included the manufacture of three additional spacecraft, while the operational phase covered the actual flying of the spacecraft to deploy satellites and other payload. The DDT&E phase was divided into two discrete increments of work due to the complexity of the development (see Figure 1-10). Increment I of DDT&E was placed under contract in August 1972, and lasted for approximately three years. It was essential that the right design planning be accomplished before placing the entire effort under contract. The initial contract increment included the development of detailed planning for components, subsystem, flight vehicles, major structural elements, and support equipment. This increment included the Shuttle system preliminary design review,

PAGE 67

Figure 1-10 SPACE SHUTTLE PROGRAM PROCUREMENT MILESTONES I I I .... ATP ,..-FMOF I l-I I INCREMENT I INITIAL DESIGN TOTAL DDT8rE INCREMENT 2REMAINING DDT&E INCLUDES 2 ORBITERS .. :1' ,,... ....... ...... -- .. I .. =c=nmw-'TFrf"W"FSRnmTJ( ... ry= J INCREMENT 3-PRODUCTION 3 ORBITERS UPGRADE/ RETROFIT2 ORBITERS "'@' 'R''W'PP'' =raum'a I INCREMENT 4-OPERATIONAL PHASE (POST DDT&E) SOURCE: Office of Shuttle Procurement Johnson Space Center National Aeronautics and Space Administration March 1972 V1 N

PAGE 68

53 and resulted in data and documentation necessary to provide a sound basis for transitioning to the second increment. Increment II of the DDT&E phase included the balance of the tasks associated with the design, development, test, and evaluation of the Shuttle Orbiter. All necessary development tests, ground tests, and flight tests were conducted to assure that Orbiter vehicles 101 and 102 (the first and second Orbiters) which were built in this increment, were safe, reliable, and capable of fulfilling the intended mission. Increment II began in July 1975 and was completed in July 1983 following four highly successful Orbital flights. Prior to the completion of the DDT&E phase, the production phase was started in June 1977, and goes through 1984. While Increment III which consisted of the production phase, was planned to include the manufacture of three additional Orbiters, upgrading and modification of the two Orbiters built under the DDT&E phase, the resultant contract for the production phase only included the production of three Orbiters and only upgrading one of the DDT&E vehicles, for a total fleet of four operational vehicles. Orbiter vehicle 101, built under DDT&E, was never intended to be a flight vehicle. Increment IV covers the operational support phase, and includes engineering support from the Orbiter contractor to support the flight operation of the Shuttle Program. The various increments of work of the Shuttle Orbiter Program are depicted

PAGE 69

in Figure 1-10. This section has established the basic procurement planning strategy for the acquisition of the Shuttle Orbiter spacecraft. 54 Thus far this introductory chapter has reviewed the federal government procurement issues over the last quarter century; discussed the statutory and regulatory framework of federal procurement; examined the methods of government procurement and types of contracts; and, overviewed the procurement and program planning for the Space Shuttle system. Procurement reform for major defense and aerospace systems has been suggested by nearly every administration for several decades. Incentive contracts were emphasized in the 1960'.s in response to the spiraling increase in weapons systems cost. While cost growth continued to be a major concern in the 1970's, it was evident that the incentive contract was not the total answer and, during the 1970's, there was a movement toward award fee contracting for major development activities. Significant acquisition policy changes by NASA during the 1960's included the movement toward incentive contracting with incentives on cost, schedule, and performance. A major acquisition policy change by NASA during the 1970's, as NASA entered the Shuttle Program, was movement toward the award fee type of contract. The NASA shift to award fee contracting followed that of the Department of Defense in the 1970's. Surprisingly enough, not

PAGE 70

55 many changes were made in the statutory and regulatory framework of federal procurement during the last 25 years. Remaining as the basic statutory framework of government procurement are the Armed Services Procurement Act of 1947 and Federal Property and Administrative Act of 1949. Procurement planning for the Space Shuttle Program in the late 1960's and early 1970's was centered around a reusable and cost effective space transportation system. Unlike all predecessor spacecraft, the Space Shuttle was mandated to have major elements that could be reused. Representing the major part of the Space Shuttle system, the Orbiter spacecraft as well as the solid rocket boosters are reusable. All major elements of the Space Shuttle system were acquired under cost-plus-award-fee arrangements during the development phase of those projects. The purpose of this study was to evaluate NASA's contracting arrangement for a major R&D project and the results which it produced. The evaluation requires an understanding of the environmental context which has been elaborated upon in the preceding pages. These include executive and legislative reform, statutory and regulatory requirements, and historical experiences, all of which are important components of _this environment. However, the evaluation requires specific criteria to analyze internally the appropriateness of the impact on the project outcome, and its flexibility for responsiveness to necessary and unnecessary change. Subsequent chapters of this study will deal with the

PAGE 71

56 with the appropriateness of the award fee type of contract for the Orbiter project, the significance of cost growth, and the impact of contract changes. All three areas require a methodology which delineates criteria for evaluations and, orders and structures information and data to conduct the evaluation. The next section of this chapter details methodology to accomplish this evaluation. Methodology This study involves an extensive review and evaluation of the procurement management of the Shuttle Orbiter design, development, and evaluation (DDT&E) project. An analysis was performed of three major elements of the acquisition process. These elements involved assessing the impact of the type of contract, evaluating the extent and significance of cost growth, and examining the impact of contract changes. Any evaluation of an administrative concept necessarily involves an assessment by the system participants. To capture some ideas of the subjective and qualitative opinion of the participants, a survey instrument was employed to gain perceptions and specific data for the Orbiter project which were then further analyzed. This method was especially appropriate given the existence of other surveys conducted in the military R&D procurement. The specific methodology used in this study began with an exhaustive literature review in the areas of award fee contracting, cost growth for major R&D projects, and contract changes on R&D projects. Results are presented in Chapter 2. Primary sources of

PAGE 72

57 material evaluated consisted mainly of NASA material, including NASA Procurement Regulations, NASA Management Directives, official procurement files in the Office of Shuttle Procurement, source evaluation board reports, financial and accounting reports prepared by NASA and those prepared by the prime contractor, official NASA contract negotiation reports, and award fee evaluation reports. Secondary sources included General Accounting Office (GAO) reports, reports by the Rand Corporation, various NASA and DOD published pertinent articles from various periodicals, congressional case studies, textbooks, and selected unpublished material. A survey of the literature regarding cost growth and changes to major R&D defense and space systems provided the necessary background of the major factors influencing the cost growth and contract changes of R&D acquisitions. Cost growth has become a phenomenon over the past 20 to 25 years and insight into previous programs was essential to understanding and drawing comparisons of the Orbiter cost growth to other comparable programs. In order to gain individual perceptions regarding the use of the award fee type of contract for the Orbiter spacecraft, and to gain perceptions regarding the causes of cost a survey instrument was utilized. Prior to discussing the specifics of the survey .instrument, it is appropriate to discuss why a survey instrument is a good means to collect the type of data desired by the researcher.

PAGE 73

58 Although the approach and the techniques of survey research often are used on any set of objects that can be well defined, survey research focuses on people, the vital facts of people, and their beliefs, opinions, attitudes, motivations, and behavior.61 Survey researchers seldom study whole populations.; rather, they study samples drawn from populations. According to Kerlinger, survey research has contributed 62 much to the methodology of the social sciences. One of the investigator's most important jobs is to specify and clarify the problem. To do this well, the researcher should not ask people what they think about a specific subject, but should also have specific questions to gain opinions that are targeted at the various facets of the problem. Survey research is perhaps best adapted to obtaining personal and social facts including beliefs and attitudes. Survey research has the advantage in that a great amount of information can be obtained from a large population. For example, a large population can be studied with much less time and cost than that incurred by a census. Another advantage of survey research is that the information obtained is accurate, within sampling 63 error. Kerlinger identifies a few disadvantages of survey re-search. First, survey information generally does not penetrate very deep below the surface. This was taken into account in this research effort by making extensive use of open ended comment

PAGE 74

59 questions. Such questions enables the individuals to explore critical issues which may not have been identified in forced choice questions. In order to analyze .these extensive comment questions a content analysis approach was used to integrate subjective comments with objective responses to survey questions. Another weakness is that survey research is typically demanding of both, time and money. Although sampling error is commonly thought to be a weakness of survey research, the survey information is 64 normally accurate. The Survey Instrument In order to gain the perceptions of key individuals involved in the management of the Orbiter DDT&E project, a survey instrument was prepared to obtain data regarding the perceived effectiveness of the award fee type of contract for the Orbiter DDT&E project. In addition, the instrument was to obtain information regarding the perceived reasons for the main causes of cost growth of the Orbiter development project. A copy of the survey instrument, which can be found in Appendix A, was sent to 36 individuals occupying key positions in the award fee evaluation process. These people included project management personnel, procurement managers, budget managers and legal advisors. The sample, although small, covers ,11 major personnel involved in the Shuttle Orbiter project, the object of this study. A total of 32 responses were received, or a response rate of approximately 90 percent.

PAGE 75

The instrument was designed to obtain opinions from the Orbiter management team regarding the effectiveness of the award fee contract management process. These propositions were: 1. Contribution of the award fee type of contract to the technical success of the Orbiter project. 60 2. How the award fee arrangement -contributed to minimizing cost growth. 3. Relations and decision-making between the government and the contractor. 4. Appropriateness of evaluation criteria and frequency of evaluations. 5. Strengths and weaknesses of the award fee type of contract for major R&D acquisitions. While a primary purpose of the instrument was to obtain the respondents perceptions regarding the use of the award fee type of contract for major R&D acquisitions, participants were asked to indicate what they believed to be the best form of contracting for several different acquisition situations. This provided another dimension to the study. An analysis was then made correlating the recommended type of contract with the job category of the survey participants. A computer program was developed which permitted analyses of the responses by job category, length of government service, and educational level of the survey respondents. These data provided for insights into the appropriateness of not only the award fee

PAGE 76

61 type of contract for R&D acquisitions, and specifically the Orbiter project, but also as analysis of the evaluation and fee determination process. Recognizing that the award fee process results is a "report card" to the contractor's management, the award fee arrangement is a powerful tool at the government's disposal in accomplishing the objectives of the contract. Thus, the computer analyses, which are extensively evaluated in Chapters 3 and 4, provided tools to evaluate the award fee and cost growth areas. Cost growth questions on the survey instrum$nt were designed to obtain the respondents perceptions regarding the major causes of cost growth for the Orbiter project. In addition, survey respondents were asked to suggest ways that they believed NASA could have reduced the cost growth on the Orbiter project. These questions, along with the strengths and weaknesses of the award fee method of contracting, were evaluated utilizing the content method of analysis discussed below. Due to the abundant amount of literature supporting three main causes of cost growth for R&D projects, i.e., inadequate or instable funding, technical uncertainty, and technical advancement, the survey instrument requested the respondents to rank the importance of these causes of cost growth as they saw them affecting the cost growth of the Orbiter project. Analyses were then conducted of the responses permitting assessments and relationships and comparative evaluations to be made.

PAGE 77

62 Content Analysis As mentioned, the extensive use of comments on the survey necessitated the use of content analysis to integrate this data with other forced choice question responses. Content analysis is a method of studying and analyzing communications in a systematic, objective, and quantitative manner to measure variables. Content analysis permits an objective and quantitative procedure for assignment of various types of verbal and other data categories. Generally, content analysis has not been done to measure variables, but rather to determine the relative emphasis of frequency of various communication phenomena, including trends, styles, and changes in content. Content analysis is a method of observation as 65 well as a method of analysis. Revolutionary evaluation of content analysis has been brought about by the computer, permitting a large number of variables to be measured. Another important attribute of content analysis is that it can be used to validate 66 other methods of observation and measurement. In terms of methodology, content analysis is basically an operation of coding. \ In content analysis, "coding" is used to describe the translation of question responses and respondent information to specific categories for purposes of analyses. Comments are coded or classi-fied in terms of some conceptual framework. For example, political statements can be coded as conservative or liberal. It is essen-tial that the coding operation be amenable to data processing if a

PAGE 78

63 quantitative analysis of the content data is required. First, the end product of the coding operation must be numerical. For ex-ample, if one is counting the frequency of appearance of certain comments, a numerical assignment must be made. Second, it is critical that the recording-keeping distinguish between the units of analysis and the units of observations. Third, when counting, it is important to record the base from which the counting is done. For example, it would not be of much use to know the number of a specific comment without knowing the number of the population in 67 total. Perhaps the best advantage of content analysis is its economy in terms of both time and money. It might be possible for an individual to undertake a content analysis, whereas undertaking a complete survey might not be feasible. An additional strength of content analysis has to do with historical research. Content analysts can study past periods of history and make various sons over time. Content analysis also has the advantage of being unobtrusive, that is, the content analyst seldom has any effect on 68 that which is being studied. Content analysis has some disadvantages. For example, content analysis is limited to the review and examination of recorded information. Although solidness of materials studied in content analysis strengthens the likelihood of reliability, problems 69 of validity can arise. Even with these disadvantages, content

PAGE 79

64 analysis provides means to evaluate qualitative data from a quantitative standpoint. Content analysis was used to evaluate the strengths and weaknesses of the award fee method of contracting. Positive and negative viewpoints were quantified to permit determination of frequency of occurrence and by respondents. Further, the comments were analyzed by the job categories, length of service, and educational level of the respondents making the specific comment. Results of the content analyses are presented in Chapter 3. Content analysis was utilized to evaluate the major reasons of cost growth, as perceived by the survey participants. Like the content analysis in the award fee area, the content analysis in dealing with the cost growth questions permitted the various comments by job categories, length of service, and educational level of the respondents to be compared and analyzed. Chapter 4 contains the results of these content analyses. Comparative Evaluation of Cost Growth and Changes In evaluating the cost growth of the Orbiter project, the Rand reports, GAO reports, and other material provided for comparative analyses to be performed. Primary data collected by the researcher provided the specific types of cost growth experienced on the Orbiter project. Causes of cost growth were grouped into three major categories: government-directed contract changes; impact of government-directed funding limitations; and overrun.

PAGE 80

Breaking the cost growth down into these categories permitted a comparison of the Orbiter cost growth with the cost growth of other comparable programs. As mentioned earlier, a computer program was utilized in evaluating the reasons for cost growth in response to the qualitative questions, thus, permitting an integrated analysis with the forced choice questions. 65 In assessing the extent and significance of the 1,468 contract changes issued on the Orbiter contract, the changes were grouped by various dollar levels with the aid of a computer program. Comparative analyses were made between the Orbiter project and another major spacecraft project. Specifically, Robert McGlashlan's dissertation provided an excellent data base to make comparative of the contract changes on the Orbiter project. For example, the data from McGlashlan's study provided for assessments to be made regarding the number of changes by certain dollar categories, as well as the causes or types of changes. This included comparative assessments of the dollar impact to the two programs, the Orbiter and Lambda projects, caused by inadequate funding, overrun, and contract changes. Overview The initial discussion of the environment of federal procurement proyides the setting for the analysis of the complexities of the federal acquisition process. This research effort will provide a comprehensive discussion and evaluation of the various types of contracts in the context of an extensive

PAGE 81

66 evaluation of the cost-plus-award-fee contract used for the acquisi-tion of the Orbiter spacecraft, and comprehensive of the cost growth and contract changes for the Orbiter development program. To accomplish those objectives, this study will be organized as follows. Chapter 2, the literature review, presents a comprehensive literature review of the three main facets of this research: award fee contracting for major R&D systems; cost growth of R&D projects; and, contract changes on major R&D projects. Professor Hunt's extensive work in researching the award fee method of contracting I in the Department of Defense provided an excellent background and theoretical framework for the use of award fee contracts for major R&D undertakings. Rand's extensive research and publications regarding cost growth of major weapons systems provided an excellent historical basis in order that the Orbiter cost growth analysis presented in Chapter 4 could be placed in proper context. Reports by the General Accounting Office (GAO) and selected articles and publica-tions provided additional insight into the major reasons for cost growth of major R&D systems. Selected case studies were also reviewed. In addition to the material regarding contract changes in the Rand reports and GAO reports, Professor Fox's publication provided excellent insight into contract change management and control based on his research on Department of Defense programs.

PAGE 82

67 Chapter 3 covers the utilization of the cost-plus-award-fee (CPAF) type of contract for acquisition of the Orbiter spacecraft. Background is given regarding the selection of the CPAF contract. Extensive treatment is presented on the procedures employed in establishing the award fee criteria, the operation of the award fee evaluation board, and the determination of the actual amount of fee earned. A survey instrument was developed and sent to key government personnel having key roles in the award fee evaluation process. These perceptions provided an abundant amount of significant data which were extensively analyzed and reported in Chapter 3. An extensive coverage and assessment of the cost growth of the Orbiter project is provided in Chapter 4. Included are an analyses of cost growth caused by changes, inadequate funding impacts, and overrun. A survey instrument was employed to obtain the perceptions of key government managers on the Orbiter regarding their assessment of the reasons for the cost growth and the ways NASA could have reduced the cost growth. A comparative analysis of the Orbiter cost growth was made with other major R&D programs. Findings are presented that relate the major causes of the cost growth of the Orbiter project in comparison with other projects. Chapter 5 relates the origin and significance of contract changes on the Orbiter project during the design development, test, and evaluation (DDT&E) phase of the Orbiter project. Ananalysis

PAGE 83

68 is presented evaluating the primary causes of the changes. Additionally, the changes are categorized by dollar range of significance. Further, comparative analyses of the Orbiter changes are made with another major spacecraft project allowing pertinent relationships to be drawn. Changes are required for all major R&D projects, the significance of which varies, since major R&D projects can typically be defined only in terms of objectives and performance specifications. It is not possible to define in advance of initiation of the contract work all contract specifications. Proper and sound management of contract changes is highly critical in R&D undertakings. Included in this report are seven appendicies containing the survey instrument in collecting data on award fee and cost growth, qualitative analyses of the strengths and weaknesses of the use of the award fee contract for the Orbiter and major R&D projects, survey participants' preferred type of contract for given acquisition situations, and analyses of perceptions regarding the causes of cost growth of the' Orbiter project. Each of. the evaluation chapters, namely Chapters 3, 4, and 5, present the analyses and interpretation of the data contained in the appendicies. The last chapter, Chapter 6, presents the summary of the research findings, conclusions, and for further stuay. Pertinent findings and conclusions stemming from the

PAGE 84

69 evaluation of the previous chapters dealing with the award fee type of contract for the Orbiter project, comparative analysis of the Orbiter contract cost growth, and analysis of the contract changes on the Orbiter project, respectively.

PAGE 85

NOTES -CHAPTER I 1 Merton J. Peck and Frederick M. Scherer, The Weapons Acquisition Process: An Economic Analysis (Boston: Harvard University Press, 1962), p.15. 2 Herman 0. Stekler, The Structure and Performance of the Aerospace Industry (Los Angeles: University of California Press, 1965), p. 47. 3 National Security Industrial Association, Defense Acquisition Study (Washington, D. C.: n. p., 1970), p. 22. 4 Ibid., p. 24. 5 The information regarding the F-14 Aircraft Program was obtained in an interview by the writer with David Novick, former staff member of the Rand Corporation, February 10, 1984. 6 Based on interview with David Novick, February 10, 1984. 7J. Ronald Fox, Arming America: How the U. S. Buys Weapons (Boston: Harvard University Press, 1974), p. 456. 8Richard J. Lorette, "Major Acquisition Problems, Policy and Research," National Contract Management Journal 10 (Winter 1976-77): 2. 9 The most complete documented assessment of the acquisition policies of the 1970's was done by the Rand Corporation in their report "Acquisition Policy Effectiveness, Department of Defense Experience in the 1970's." Therefore, I drew heavily on that excellent piece of work prepared specifically for the Office of the Under Secretary of Defense for Research and Engineering. The complete reference is: Edmund Dews and Giles K. Smith, Acquisition Policy Effectiveness: Department of Defense Experience in the 1970's, Report No.R-2516-DR7E, Santa Monica, Calif.: The Rand Corporation, 1979), pp. 1-15. 10Ibid., p. 1. 11Ibid., p. 2. 12Ibid., pp. 17-24. 13Ibid., p. 24.

PAGE 86

14 U. S., Department of Defense, Office of the Deputy Secretary of Defense, Improving the Acquisition Process, Memorandum, by Frank C. Carlucci, April 30, 1981. 15 "Why the Carlucci Initiatives Aren't Working," Government Executive, (August 1982): 30-2. 16Ibid. 17 u. S., Office of Management and Budget, Major System Acquisition, Circular No. A-109, April 5, 1976. 1 8"systems Acquisition: How A-109 Can Help Shorten the Process", Government Executive, (October 1977): 10. 19 National Aeronautics and Space Administration, Major System Acquisition, Management Instruction 7100, 14A, (April 19, 1978): 2. 20 U.S., Commission on Government Procurement, Summary 71 Report, (Washington, D.C.: U.S. Government Printing Office, 1972), p. iii. 21Ibid., p. 1. 22 U. S., Office of Management and Budget, Proposal for a Uniform Federal Procurement System, February 1982, p. 1. 23Ibid. 24Ibid. 25Ibid. 26u. S., Presidential Executive Order 12352, Federal Procurement Reforms, March 17, 1982. 27u. S., Office of Management and Budget, Proposal for a Uniformed Federal Procurement System, February 1982, p. v. 28 Ibid., pp. 8-9. 29Presidential Executive Order 12352. 30 U.S. General Accounting Office, Progress of Federal Procurement Reform Under Executive Order 12352, PLRD-83-88, (Washington, D.C.: U.S. Government Printing Office, June 17, 1983).

PAGE 87

45. 45. 31Ibid. 32 Commission on Government Procurement, Summary Report, p. 33 Peck and Scherer, Weapons Acquisition Process, p. 25. 34c . G P S R on overnment rocurement, ummary eport, p. 35u. s., General Accounting Office, Multiyear Authorizations for Research and Development, PAD-81-61, (Washington, D.C.: U.S. Government' Printing Office, June 3, 1981), p. 3. 36 National Aeronautics and Space Administration, Annual ReportSpinoff 1981, (Washington, D. C.: U.S. Government Printing Office, April 1981), p. 14. 37 Richard R. Nelson and Merton J. Peck, Technology, Economic Growth, and Public Policy, (Washington: The Brookings Institute, 1967), p. 83. 38 G. A. Smith, A-109: It's Purpose and Implications in Contracting Costs of Major System Acquisition", National Contract Management Journal 14 (Winter 1980): pp 33-4. 39 Ralph C. Nash, Jr. and John Cibinic, Jr., Federal Procurement Law, 3rd ed. (Washington, D. C.: The George Washington University Press, 1977), p. 4. 40 U. S., Congress, Senate, Report of the Committee on Governmental Affairs, S. Rept. 95-715, 95th Cong., 2d sess., 1978, pp. 6-9. 41u. s., Office of Management and Budget, Proposal for a Uniform Federal Procurement System, February 1982, p. 133. 42Ibid. 43Ibid. 44Armed Services Procurement Act, U.S. Code, vol. 5, sees. 219b, 412 b; vol sees. 151-161 (1948). 45 Federal Property and Administrative Services Act, U.S. Code, vol. 5, sec. 630; vol. 40, sees. 471-475; vol. 44, sec. 391 (1949). 72

PAGE 88

46 U. S., Office of Management and Budget, Proposal for a Uniform Federal Procurement System, pp. 8-9. 47Truth in Negotiation Act, U.S. Code, vol. 10, sec. 137 (1962). 7. 48 commission on Government Procurement, Summary Report, p. 49Nash and Cibinic, Federal Procurement Law, p. 38 50rbid., p. 40. 51Ibid., p. 222. 52 National Aeronautics and Space Administration, Procurement Regulation, NHB 5100.2C, (Washington, D. C.: U.S. Government Printing Office, 1981), p. 3-2:1. 53 Nash and Cibinic, Federal Procurement Law, p. 317. 54 U. S., General Accounting Office, Government Contract Principles, 2nd ed. (Washington, D. C.: U.S. Government Printing Office, August 1978), p. 4-3. 55 National Aeronautics and Space Administration, Procurement Regulation, p. 3-8:10. 56rbid. 57 National Security Industrial Association, Defense Acquisition Study, p. 19. 58 National Aeronautics and Space Administration, Procurement Regulation, Part 15. 59 National Aeronautics and Space Administration, Space Shuttle, SP-407, (Washington, D.C.: NASA Headquarters 1976). 60 The primary source material for this section was obtained from the official NASA procurement files for the Shuttle Program, Office of Shuttle Procurement, Johnson Space Center. 61 Fred N. Kerlinger, Foundations of Behavioral Research, 2nd ed. (New York: Holt, Rinehart and Winston, Inc., 1973), p. 411. 73

PAGE 89

62Ibid., p. 415. 63Ibid., p. 422. 64Ibid., pp. 422-3. 65Ibid., p. 525. 66Ibid., pp. 530-2. 67 Earl R. Babbie, The Practice of Social Research, 2nd ed. (Belmont, Calif: Wadsworth Publishing Co., Inc., 1979), p. 242. 68Ibid., p. 252. 69Ibid., pp. 252-3. 74

PAGE 90

CHAPTER II LITERATURE REVIEW The Space Shuttle Orbiter project was a research and development (R&D) activity contracted with the private sector by the National Aeronautics and Space Administration (NASA). To understand the full dimension of the procurement management approach undertaken by NASA, it is important to focus on the._ specific contracting arrangement, the results of performance, schedule, and cost, and the impact of contract changes. All of these three areas to be addressed in this study for the Space Shuttle Orbiter have also been subjects of the procurement literature. As indicated, this research effort focuses on the cost-plus-award-fee contract which has been used by the Department of Defense (DOD) and was used extensively by the National Aeronautics and Space Administration for the development of the Shuttle Orbiter. While little has been written on NASA's use of CPAF contracting, the literature on the Defense Department's usage of CPAF contracting is considerable. The Importance of Contract Type Profit, generally, is the primary motive of business enterprises. Recognition of this fundamental assumption can be seen in the NASA Procurement Regulations, which state that both the government and its contractors should be concerned with /

PAGE 91

harnessing this motive to work for the effective and economical 1 contract performance required in the national interest. To this end, both parties should seek to negotiate and use the contract type best calculated to stimulate outstanding performance. An 76 overriding goal should be to insure that outstanding performance is met by high profits, mediocre performance by mediocre profits, and poor performance is met by low profits or losses. Selecting the appropriate contract type is generally a matter for negotiation and requires the exercise of sound judgment. The objective is to negotiate a contract type and price that will result in reasonable contractor risk and provide the contractor with the greatest incentive for efficient and economical perfor-2 mance. Many factors need to be considered in selecting and negotiating the appropriate contract type, including price competi-tion, cost analysis, type and complexity of the requirement, urgency of the requirement, period of performance of the work, and the contractor's capability and financial responsibility. As previously explained in the introductory chapter, contracts fall within two basic families of contracts: fixed-price and cost-reimbursement. Figure 2-1 describes these two families of contracts the description of each type with the elements of each.3 It can be noted that contractor cost responsibility {risks) shifts from full contractor responsibility under the firm-fixed-price arrangement to virtually no cost responsibility under the cost-plus-fixed-fee type of contract. Meneely summarizes the key differences between these two broad

PAGE 92

PIRM II"IJCI!D PRICE .,, .. Gov.,.fll'l"'mt PIYI prict hCft II ftOI ILIDIKt to anr l'llllt f'Mftl ,.,.,.,, ..... centrector' l c01t prri,.,Ct Placn '"I Ilium rlala ., contractor Z Contracter "'' 0 gretf'lt inCf'ftliYI IO control c0111 L Minimul"ft ed mifttlffatiwt DUtdf'fl D:' Oft Datll'l u Ill Prf"frrttcl cUI'Irct w 0 L .. ertort: Pey mrnt illbiMCI on tort Pf'ntiPd rathpr than '"'"''' achPwta conrr.ctor proviO.t JOKfH tllort o""' atattd Pf"OCIIor fiapd pricf' L..._ ------Figure 2-1 TYPES OF CONTRACTS FIXED-PRICE FAMILY 1 COST-REIMBURSEMENT FAMILY INCREASING CONTRACTOR COST RESPONSIBILITY DECREASING FIXED.. RICE FIXED-PRICE PRICE COSTPLUS COST .. LUS COSTPLUS COST AND COST WITH I!CONOMIC INCENTIVE REDETEA INCENTIVE AWAADFEE PIXEDFEE SHARING PRICE CFPII MINATION FEE CCPAFI CCPFFI ADJUSTMENT CCPIFI CIIJ ... E .. AI Thl' prict Plld by ttw Firm Tar .. t: Gotrn Pros.-ctlwt; Govrrn Gower""""' PIYI Govrnmtnt PIYI PIIYI Celt: GowtrnmMI govf't'n,.,..nt mer be tnMI .,.,. .,ict tftet ml'tll Dltl htd prlc IIIOWIGI CUll lrtd 1ft IIIGWiblt CDII. boaM COli lncl al..,.ebleceat. rtwiNCI up.erd or 11 aum of fint1 for DbOdl or Mrvlcn ttntlvrfH. '" '"" ... ,., '" ll .. diM. IWIM. dDwnerd it MOOfllfl'd cftl 1n11 tor 1 gh,rn PfPiDCI. P,ut conu,...ncin occur. tlnal profit ptiU It IUbiKI ta lftCI'ftliv. '" .. ttr COfttr actor ,.,,.. 1 ... _.__ .... Celt-rlftl: -.'"I ptfoflt cftotrrrr11nfod ltr ''"on 1t1trd mined ltilr comparu"g bl .. 1ft hCh dOtl warr wtm actual rnt'rlt NYI 01"1' I poor ProwldH for prlct ld campar.ngtinal lms during Pf'rform athJel cost to tergpt not warr .,,,,. c,..r COlli lion (If IIIDWibll COlt iu''"'"t lo protKI ntODtiati'CII cMt to anc of contract cost end lldtultng formanu and all or 11 mutually a gr..,.. to I partl'l against tero cost end ed targpt fH lAW t., ad Dart of 1ft IWId IH Fiwl'd file may bllct by._ Nrfil'& COft Jitnifcant t'
PAGE 93

4 types of contracts. Under the cost-reimbursement type of contract, the contractor promises to provide their "best efforts" and the government promises to pay all of the contractor's allowable and allocable costs. Assuming the completion form (requires contractor to deliver end product) of contract, the contractor is obligated to continue performance as long as the government is willing to pay the bill. Under a fixed-price contract the con-tractor promises to deliver a product or provide a service at a pre-determined price, regardless of the actual cost outcome. Figure 2-2 further illustrates the fixed-price and cost-78 reimbursement families of contracts by indicating the applications and limitations of the various types of contracts. Of particular significance is that some form of cost-reimbursement type of contract is typically used for major research and development projects. Cost-reimbursement type contracts are usually preferred (in fact, this is suggested by the NASA Procurement Regulations) for development efforts and particularly for major aerospace and defense systems. This preference is based on the realization that since the cost data available usually does not permit sufficiently realistic estimate of the probable cost of performance or un-certainties surrounding the contract performance cannot be sufficiently identified to evaluate their impact on price. Fixed-price type of arrangements are generally not considered until the risks have been reduced which is typically when a program has reached the

PAGE 94

Figure 2-2 TYPES OF CONTRACTS FIXED-PRICE FAMILY .I COST-REIMBURSEMENT FAMILY INCREASING CONTRACTOR COST RESPONSIBILITY-----,>ECRl:I.<,ING FIRM FIXED FIXEDPRICE FIXEOPRICE PRICE COSTPLUS COSTPLUS COSTPLUS COST AND COST PRICE WITH ECONOMIC INCENTIVE REDETER INCENTIVE AWARDFEE FIXEDFEE SHARING CFFPI PRICE CFPII MINATION FEE ICPAF) CCPFFI ADJUSTMENT CCPIFI CFPEPAI ------Whrn r ana Wht'n conlnurncrl Whfrl' aU\Impt.on ol PrOlplchvr: Quanhlr Ocvrlopmrnt and It'll Lrwf'l otrtlort ton RrN>arch. Coat: Non prohl .n rt.nonbtc pr.u tin "'"'""" tram I di'Qft'P Ol Ull prOGliCIIOn Of UfVI(U whrrr a prohl '"(f"' lr Atll tor un11Cf'l llilutOn'I/OfQllftiiO 1M' t\UUllllhfd II un1tablt' m1r11e-1 or rrsponbl.rv llf con wrun a ana l1wP IS hltot'l'f' IO pro Vllrhl'ft' o!IC hfVI'rTlt'OI Prl'hl'l'lnar't' plora honl ana tchlirl outut labor condIDn\ can tractor\ ..,,u provdt' rra,onablr pr '' r can vdt' motavat.on tor mull bt I"IOIUIIII'O lon or tludf conlr.acl\ z Dt' dtnfll"d and ncrnllvr lor rllrclwr bf nf"QOIaiPd' lor '" rnor rflt'c hwr \Ublf"tlv"ly Perllcularlr IUIeblr cavrrra bt a srDaratr co11 control hal Pf'' OCI bul no I rn managemrnt Orvrtopmrnt .and 1r1t Coli IPianng: R & 0 0 tor 11anoaro or pr.c adru\tmrnt t.rr r.onu,ur Od Vthrrr lon.te pprtorm whrrt CPIF not pr1c rllorl\ '" '"'r clauu Can combnr wlh in anu OOIPCII\Il'!l lan tical. prolll nr non profl crnlwt\ on tJt'rform rtra.aclvr Whrn not tHo t'Siablo\ht-O n contractort <( ltm\ tor wr'lich IOUnd anu .and \Chf'l:llollt loJr and rr,u.ondblr ro u DFU!o can bt FFP cannot bC' actual p('rlorm.ancr .J dt'vtloped nr..,ohtiii'CI o'lnCJ 10.,... v.aluf' or 1hort pt-rod Awi'lrd trr mllly br g,: Lent of E.llorl A&C ol pN lor m.tnc r ulttd in con.unt t.on Q. oR'Iolt'\IOi!lhOn IIUd'l' rrnelr" olhrr 1 wtlh Olhtr ltPl'\o of "' mpract.cablc cnntracu. Lint ol Ellarl: U\td AdrQUJIIIP CD!ol or pr1C Prf'IPt&:ll\olt FFP RDI Ff'l" lmll "' FM 1hall not t&:Hd Colt llllflftl. Nor ap only whf'n work r..an 1ng a.altt must bt 1(111\bl(', prcnQ CPFF. "' ot r1hmtll'd COli phcablr tor rllort not br ctrarly dttntd ""etablt to rslabti!oh potrod' conform to lor A&.O or u--.. Dl IPKiliPd by gowtrn Dul f'IIOrl drlrrCJ can targrt\, S.Oir purpo'r contr'acro,'!l account tttimated coli tur mtnt or whch Pin br evrfi'O upon c annol bP to shll CO!. I "9 \rs.trm and Frt' htnlt producton contr.ach. only minor rtrvnce Ill rr\pons.bhlt to auur'.tnl r lho'll prcf' Uml' n CPFF to commrcal ac z QO'Wtrn,rnt. ri'Qurr1 tt>d4'1Nmnaton wll Pr.u ot AlE contract hwitn ot tne conuac 0 lmullanf'QoS agrrr br takrn promptlf Wf'Qhrra gudf"hnn lhall not uurd o-. ot tor mrnt on ,all rlrmrnts lfor arttrm.n.nu prot elhmaltd co11 of lhr 1-ol pr.cnu 1tructure Al'lro.achvl': 11 obirct.vrl 'II'UIII not pubhc won or uttllly "' RPas.onablt anuranct bt apphi'CI prOjiCI. 1-lhoJI pru rrllrtrr mnaton wll tit IAII.tn Sl'lall not br u!.rel '" prompt I., and rr r.ru ol CPFF or CPIF qu'"' HCA epprovat. whtn Obtrr.lnrr .J mnurrml'nl 11 teni oo. ,_,ruatalrd Proturrmrnl\ Onlr Nt;oliett-d Procurtmtnll Only N'tiOiilltd PriKurrmenrt Only AdrQualt' C.ontrector Cosl Auount.nQ .. Aelequelr Conine tor Cot A
PAGE 95

80 final stages of development and technical risks are minimal.5 Selection of contract type is generally directly related to antici-pated cost risks, and hence, expected profits. Demong and Strayer, two .other respected experts in the government procurement field, have concluded that it is essential to understand the multiple goals of the organization in selecting the contract type. Their evaluation of the goals of DOD's contractors--sales maximization, firm perpetuation, engineering staff and employee continuation, market share, research and development knowledge and expertise, technological innovation and base, risk reduction with long production runs, managerial job satisfaction, corporate image, employee relations, and supplier loyalty--; found that they will not necessarily raise the cost of 6 the acquisition process. Demong and Strayer's point is that by assuming that profit is the corporate manager's primary goal, DOD, in this case, may be overlooking other equally effective motivators that could prove to be less costly. The preceding discussion of contract selection provides the necessary background information to begin examination of move into the award fee contracting method. Origins of The Cost-Plus-Award-Fee Method of Contracting Development of management tools is similar in many respects to the development of a spacecraft system or a major weapons system --it is accomplished through the combined source of knowledge and

PAGE 96

81 performance of manY. groups and organizations. Variations of the CPAF concept were first tried in aircraft maintenance and overhaul contracts during the 1950's while the current incentive approach to after-the-fact evaluations resulted from the independent but concurrent ideas of several individuals during 1960 and 1961. 7 There were several independent planning actions by the Navy and NASA which considered the use of award fee contracting during 1961 and 1962; 8 The NaVy's first CPAF contract covered operation and maintenance of instrumentation systems and associated range facilities and was issued by the U.S. Navy Purchasing Office, Los Angeles in March 1964. Within NASA, the first negotiation for a CPAF contract covered operation, maintenance, and engineering services for the Mercury Manned Space Flight Network. Award of that contract was delayed, and NASA's first award of a CPAF arrangement was made by the Space Nuclear Propulsion Office in Cleveland for the research and development of a nuclear powered rocket engine. Originally approved for test by DOD for experimental use, the CPAF contract was envisioned for use only in level-of-effort types of contracts with the first five applications of the CPAF contract type under the test covered the procurement of engineering, technical, and support services. Use of CPAF contracts by the Navy during the two and one-half year period following the test approval was expanded to consider the procurement of research and deveiopment, naval architectural 9 design, and construction.

PAGE 97

82 NASA's experience with CPAF has spanned the Mercury, Gemini, Apollo, and Shuttle Programs, covering spacecraft develop-ments, mission and systems engineering, support services for laboratory and test facility support, technical services support, maintenance and operations, and launch support services. Use of the CPAF contract type is defined in the NASA cost-plus-award-fee 10 guide. A CPAF type of contract is defined as a cost-reimbursement type of contract which provides that the contractor's variable fee will be determined subjectively by designated, high-level, government personnel on the basis of periodic, after-the-. 11 fact evaluation of the contractor's performance. The CPAF contract normally has a base fee which is essentially guaranteed, and provides for additional fee to be earned based on the con-tractor's performance evaluated in accordance with criteria set forth in the contract. Award fee determinations are the subject of special checks and balances which provide procedural safeguards protecting the contractor from arbitrary or capricious evaluations, but are not subject to the conventional disputes clause procedures (to be discussed later in this section). The fee determination official is assisted by a performance evaluation board, technical and business evaluation coordinators, and specialists serving as evaluation monitors. Combination forms of CPAF and cost-plus-incentive-fee 12 (CPIF) contracts are often times employed. The CPIF arrange-ment provides for a fee which is adjusted by formula in accordance

PAGE 98

83 with the relationship which total allowable costs bear to target cost. Under the CPIF type of contract, there is negotiated initi-ally a target cost, a target fee, a minimum and maximum fee, and a fee adjustment formula to determine actual earnings. Combinations of CPIF and CPAF contracts are designed to provide incentives for maximum effort on the part of the contractor to 13 manage all aspects of the contract effectively. While the CPAF portions may involve certain formulas for measurement of performance achievement, the overall determination of award fee is generally based on subjective evaluations of performance and achievement, and the CPIF fee is based ori a fee adjustment formula applied to the fee available which is allocated to objective measurements of cost control. Raymond Hunt, generally acknowledged as one of the leading experts in the field of award fee contracting, indicates that the basic motive behind award fee concepts is the same as that for incentive contracting, namely, a desire to contract in ways that capitalize on the motivations of the contractor's profit with 14 variations in his actual performance. Award fee contracting alternatives resulted from recognition of problems of uncertainty associated with setting contract performance targets very far in advance of actual performance. Hunt further states that for its designers, award fee methodology undoubtedly represented a situationally constrained special case of incentive contracting, principally an alternative to cost-plus-fixed-fee (CPFF) contracts.

PAGE 99

84 At this point, the benefits of CPAF contracting should be de-lineated and evaluated. The NASA Award Fee Guide is quite specific regarding the benefits that should result from the use of CPAF contracts. A major advantage is better communication between technical and administrative personnel and within the contractor's firm, con-tractor personnel have experienced improved communication between 15 management and workers. Insofaras contractors are concerned, they have been afforded greater latitude in control of their personnel and activities. Better communication in turn should lead to improved performance. Another important aspect of the CPAF contract may be the preparation of well-defined work statements. A CPAF contract generally forces better work statement definitions than a CPFF contract. Clearly, the work statement and the award fee evaluation plan represent the basic documents about which all else revolves. Improved definition is seen as a key to structuring more meaningful CPAF contracts. According to the NASA CPAF Guide, indepth studies of CPAF experience have found that CPAF incentive contracts have generated 16 tangible motivations and beneficial response. Communication between government project offices, contracting offices, and contractors have improved, and vertical communication within the contractors corporate structure have improved. The CPAF concept demands and results in improved definition of tasks; both the

PAGE 100

85 government project managers and contractors management have used the CPAF procedures as management tools. to communicate to supervi-sory, administrative, and operating levels. At this point, it is appropriate to discuss the selection of evaluation criteria. Perhaps the greatest challenge with CPAF contracts lies in the quality of the evaluation criteria structure and the rating plan. It is critical to understand the distinction between subjec-tive and objective evaluation of contractor performance. Subjectivity relates to the inability to define requirements in a given performance situation with the precision of mathematical or similar objective measurement methods. Evaluation of contractor perfor-man'ce must be made solely on the basis of a subjective analysis which reflects the evaluator's opinions and impressions as to the level of quality achieved by the contractor. Objectivity, on the other hand, indicates that the parameters are well-defined, that they are mathematically or physically measurable, and that any two persons reviewing a given work effort would be able to equally determine the degree of performance achievement in some reasonably verifiable manner. Under the CPAF contracting approach, the broad criteria against which the contractor's ultimate performance is evaluated generally include performance of operations, technical management, 18 business management, and utilization of resources. While the broad criteria are included in the contract, subfactors to be evaluated and emphasized are made known to the contractor in

PAGE 101

writing prior to the beginning of each evaluation period. The intent of the criteria should be as clear as possible so as to provide clear goals against which the contractor works. Con-tractors are given an opportunity to submit pertinent information in connection with each evaluation period; there is a decidely unilateral aspect to award fee determination. Final determination by the government of the amount of award fee earned by the contractor is unilateral, and binding on 19 the contractor. Award fee determinations are not subject to the "Disputes" clause of the contract because of the highly judgmental character of the determination. 20 Contractors are normally given an opportunity after the fee recommendation for the 86 particular evaluation period to submit any additional data on their performance for that period. However, having considered the contractor's final position, the .fee determination official will make the final award fee determination. It is extremely important that the integrity the evaluation system be maintained at all times. The award fee concept is more than a contract type--it is a powerful way of managing and should be discussed from that perspective. The Award Fee Contract as a Method of Management Professor R. G. Hunt of tpe State University of New York at Buffalo, has done perhaps the most extensive research in award fee contracting. According to Hunt, the award fee method of

PAGE 102

87 contracting is best regarded as a method of management, and not just "contracting". Hunt conducted a study of award fee contracting in the Air Force Systems Command and found that the award fee concept stimulates a responsiveness attitude among con-tractors and broadens the latitude within which government managers may influence the directions and outcomes of the programs for which 21 they are responsible. In addition, the award fee method of contracting gives indications of helping to discipline actions within the government project office and their relations with contractors. Award fee contracting, while significantly enhancing the power of government managers over contractor organizations, holds genuine promise according to Hunt, as a feasible innovation for implementing joint management of the federal R&D system and of public-private sector exchanges more generally. The joint management concept, or J-model, as it is referred to, will be discussed in the following paragraphs. Government-contractor relationships under award fee contracts imply choice of what Hunt refers to as a shared leader-ship or joint management, which he calls J-model, of program 22 management and system acquisition. A J-model acquisition is, in essence, an administrative exchange among organizational elements of the contracting parties, shared decision-making, and essentially an even distribution of power among the parties to the contract.

PAGE 103

88 To clarify the J-model concept, Hunt contrasts it with the customary notions about the nature of customer-supplier relations in government systems acquisition, called the F-model (formal relationship). It views the working relationship between a govern-ment agency and its contractors as an altogether formal, arms-length link-up of entirely separate parties who come together for the strictly expedient purpose of performing a task. This F-model assumes that the special terms of this relationship are fully and clearly specified in a formal contract, but that management deci-sions independently determine and control the actions of the parties. In the F-model, the government's program responsibilities are regulatory rather than managerial. While a J-model looks upon the government-contractor relationship quite differently in systems acquisition, it envisions a dynamic acquisition environment and recognizes a limited division of labor between the contracting 23 parties. However, it also assumes informal cooperation at operation interfaces, continuous explicit and implicit negotiation of the work to be done and informal resolution of disputes. Increasingly, conditions in the real world are not likely to correspond entirely to the assumptions of either the F-model or J-model. However, the areas of research and development of major systems acquisition unquestionably correspond more closely to the J-model concept than to those in the F-model. Customer-contractor relationship is basically a single performance unit shaped by the

PAGE 104

89 separate properties of contractors and customers, as well as by the nature of their working relationship and by the technological and political framework within which they carry on their working relationship and by the technological _and political framework within which they carry on their business. Hunt's research supports the finding that contracts play an important part in forming and guiding those relationships and their outcomes, but extra contractual factors significantly influence the relationship 24 between the two parties. Hunt's extensive research in award fee contracting in-dicated that the award fee process assists the government in dealing with two management imperatives in R&D program management: coping with uncertainty and active participation.25 He has developed the "hallmarks" of award fee acquisition which are shown 26 in Figure 2-3. Hunt's research supports the contention that in R&D acquisitions, the government needs a strategy that can adapt to the uncertainty in R&D procurements, help reduce the uncertainty, and safeguard the government's interests by allowing more partici-pation in.management of the acquisition process. In his extensive work with the United States Air Force, Hunt surveyed a number of Air Force contractors who have performed work under CPAF contracts and reported that contractors are typi-cally highly responsive when performing under award fee contracts.

PAGE 105

90 Figure 2-3 HALLMARKS OF AWARD FEE FOR RESEARCH AND DEVELOPMENT ACQUISITIONS, 1980 1. Encourages government-contractor cooperation. 2. Assures an active role for government managers. 3. Recognizes limitations on top management's ability to control operations. 4. Stimulates formal and informal communication. 5. Recognizes variability of motivations. 6. Leaves to contractors the task of motivating their own personnel. 7. Views the acquisition process as dynamic. 8. Is flexible and provides for human judgment. 9. Simplifies contractual provisions. ,10. Helps assure that profits are earned. SOURCE: Raymond G. Hunt, "Use of the Award Fee in Air Force System and Subsystem Acquisition," Final Report to Air Force Business Management Research Center, Wright-Patterson AFB, Ohio, March 1980, pp. 21.

PAGE 106

Hunt's findings were that contractors possess a highly responsive attitude and high rates of formal and informal communication when working in an award fee environment.27 His research also indi-cated that contractors seem to be well informed on the award fee 91 evaluation plans with contractors generally having the opportunity to make suggestions on the plan and also to provide a self-appraisal at the conclusion of the award fee period prior to the government determining the amount of fee earned. While Hunt found that the award fee method was a "potent motivator" and clearly created a high tension environment, contractors perceived them-28 selves as very vulnerable. Even so, most contractors judged the award fee system.to be fair in actual practice. In summary, the literature surveyed in study strongly supports that the award fee concepts "works" in R&D acquisitions. Although the CPAF contract requires attention to details for effective administration, the after-the-fact subjective evaluations permit the government to reward or penalize the contractor based on the quality of contract performance. Award fee contracting en-hances the government-contractor working relationship which is highly critical to high quality performance in R&D acquisitions. Evaluation criteria can be tailored to the needs of the specific program. An apt conclusion the literature review of award fee con-tracting, is the following quotation from Hunt's article entitled "R&D Management and Award Fee Contracting":

PAGE 107

92 Needed for the development of sound procurement policy is a thorough realization of the intimacy of relations among program planning, contracting and program management. Today's era of the new Federalism calls for a shift of focus from the mere mechanics of contracting to the dynamics of planning, negotiationand administration. The award fee approach is conceptually consistent with such a broadened perspective and offers promise asa means of it into practicable procedures for R&D procurement. It is appropriate, at this point, to review the literature regarding cost growth of major research.and development projects. Cost Growth There is an immense amount of literature with cost growth of major systems. A main focus of this study deals with an emphasis on comparative analyses of cost growth; thus, the literature review will be presented in three parts: selected studies of the Rand Corporation; pertinent General Accounting Office (GAO) reports; and other sources such as books and articles, that present case studies of actual situations. Due to the many facets and organization of subject matter in the reports, the review of the Rand reports, as well as the GAO reports, will be presented in chronological order. The Rand Reports In the late 1940's the Rand Corporation initiated their first study dealing with weapons systems analysis primarily as an out-growth of their work assisting in development of a program budget for the federal government. Completed in 1949, Rand's first weapons system study also introduced social, political, and

PAGE 108

30 economic factors into the analysis. Following this initial effort, Rand has engaged in numerous studies dealing with the acquisition of major systems, including several studies on cost growth of major systems. This literature survey covers pertinent Rand studies conducted over the past 20 years. Rand's first report dealing with the cost of advance weapons was written by David Novick in 1962. Novick found that complexity and lack of reliability were two major causes of the high cost of new weapons. It was Novick's belief then that the 93 high cost of advanced weapons was essentially due to the nature of the weapons themselves. Novick identified the categories of cost of new weapons systems as: research, development, test and evalua-tion; initial investment required to introduce a new capability into the operational defense force; and annual operating costs ) which includes expenditures for maintaining and operating a system. 31 In his study, Novick found that the increased emphasis on research and development activities has a significant bearing on advanced weapon systems costs. For example, each succeeding generation of military aircraft has had two items in common--a higherperformance level than previous aircraft, and second, more complexity, hence more expense. With the higher performance and complexity characteristics, coupled with the new environmental conditions in which the new weapons operated, demand for increased

PAGE 109

reliability followed. The result has been increased emphasis on reliability requiring elaborate test programs.32 In addressing the question "How can expenditures remain more or less level and the cost of advanced weapons continue?", Novick suggested that the answer lies in the increased effective-ness of the new weapons, thus necessitating fewer units of new 94 systems to perform the same role that large numbers did during, for example, World War II. No longer will there be a need of procuring thousands of bombers and fighters. In addition to increased effectiveness, the rapid rate of, technological change and the recognition of the complementary nature of weapons and armed forces 33 serve to reduce large quantities of weapons. In placing the problem of cost growth and cost estimating in its proper perspective, Novick evaluated former Secretary of Defense Robert S. McNamara's philosophy during the Defense Department's first attempt to develop a program budget during 1962. According to Novick, the former Secretary of Defense's belief was that due to the great technical complexity of modern-day weapons, their lengthy period of development, the tremendous combat power and enormous cost, sound choices of major weapon systems in relation to military tasks and missions have become the key decisions around which much else of the Defense Department 34 revolves. Novick followed the previous mentioned report with a brief report dealing with the cost of new weapon systems, titled "Costing

PAGE 110

35 Tomorrow's Weapon System." Again, Novick found that there are people who prefer not to know the real cost of a weapon system in advance because large costs have a tendency to prejudice the chances that system has for survival. Estimating the cost of new weapon systems is a difficult and unreliable process but it is essential for budget preparation and for major program decisions. Novick's research indicated that the primary cause of bad cost estimates was neither incompetence nor deliberate understatement but rather a basic inability to understand the complexity of the job to be done or just being too optimistic. As one of Novick's classic statements indicates: but in the matter of cost estimates, it may be that the incentives to estimate low are much greater than the penalties, if indeed there are any penalties."36 Following Novick's work, Irving N. Fisher of the Rand 95 Corporation, in 1968 published the results of his research dealing with cost growth under incentive contracts. Fisher's works support the contention that cost overruns have been far less frequent and less substantial under incentive contracts than under cost-plus-fixed-fee (CPFF) arrangements. Defense Department officials have concluded this outcome as evidence that a con-tractor's performance under incentive contracts is typically more efficient than under CPFF contracts. This study also cites former Secretary of Defense McNamara; this time, in regard to evaluating the impact of incentive contracts on weapon systems costs. The former Secretary of Defense believed that resulting contracts are

PAGE 111

at least ten percent lower than they would have been under CPFF 37 pricing arrangements. An optimistic sound target cost is the key to effective incentive contracts, according to Fisher.38 In order to have meaningful incentives, it is required that the target cost be a realistic estimate of the contractor's expected cost outcome. So long as prices are determined competitively, there can be little 96 chance of awarding contracts with targets that significantly exceed contractor's anticipated costs. The market forces which operate in competitive environments tend to nullify the possibility of obtain-ing excessive targets. In the present defense acquisition environment, however, target costs for the majority of the incentive contracts awarded for major weapon systems are negotiated without the benefit of competition. These resultant contracts may fail to provide any real incentive for cost reduction and efficiency. Fisher concludes that there is no question that incentive contracts make both the government and contractors more cost conscious than non-incentive contracts such as CPFF contracts.39 In a separate 1968 report, Fisher reviewed how incentive contracts serve to reduce cost growth on weapon systems con40 tracts. Defense procurement policy during the 1960's relied heavily on the use of incentive contracts to provide contractors with motivation through increased profits to improve control of costs. Fisher found that contracts on CPFF arrangement do not

PAGE 112

97 provide motivation for contractors to minimize costs, and further, there is evidence that CPFF contracts may actually motivate con-41 tractors to increase costs. By increasing the total profit as actual costs are reduced below a target, incentive contracts place greater financial risks on contractors to achieve cost underruns since the government no longer completely absorbs cost overruns. In this report, as the previous report, Fisher makes the point that unless target costs are determined competitively, there is little chance of obtaining targets that significantly vary from contractors' anticipated .costs. Target costs for most incentive contracts awarded for major systems are negotiated without the benefit of competition. Fisher observed that incentive contracts often seem to be regarded as a substitute for competition since DOD normally awards production contracts to the development contractor without competition from other firms. Effective price competition, therefore, usually exists only at the development stage (the first 42 stage). It is commonly believed, according to Fisher, that incen-tive contracts provide considerable motivation for increased 43 efficiency and improved cost control. Because numerous incen-tive contracts continue to be awarded without real effective price competition, there can be no guarantee that the negotiated targets are sufficiently close to contractors' expected costs to provide meaningful incentives for increased efficiency. Thus, Fisher

PAGE 113

98 suggests that future gains in incen.tive contracting are likely to come through better ways of negotiating meaningful cost targets, increased competition, and improved methods of estimating costs.44 In 1969, Robert L. Perry conducted an extensive study of the Defense Department's system acquisition experience. This Rand study describes the of an analysis of 21 Army, Navy,and Air Force major system acquisition programs of the 1960's which 45 together cost more than 19 billion dollars over a 10-year period. Perry deals with the impact of program changes to the cost of the systems. Data that one might typically expect to be readily available in unambiguous form are not. Perry's study suggests that program managers and reviewers often do not know what data are relevant to understanding or even recognizing program change trends. Cost escalation causes were n.ot obvious, and there was no reasonable way of discovering whether poor estimating, inadequate cost control, or simple misunderstanding of program objectives were responsible for cost outcome, that differed significantly from 46 initial cost predictions. Large variances between anticipated and actual system cost and system performance were typical of the systems studied in this survey. Cost growths of the systems surveyed averaged 40 percent. Most of the cost of changes were adjustments to contract targets rather than being related to contractor overruns. Therefore, the government ultimately accepted responsibility for most of the 47 financial consequences arising in changes of program scope.

PAGE 114

99 Because changes in program scope account for most deviations between projected and actual program outcomes, it may be that current pricing and costing techniques are essentially valid. As found in Perry's study, most program change appears in the form of target cost adjustments, indicating that changes are generated largely from altered requirements or project specifications. Perry's 1969 study indicated that factors outside the control of the contractor typically caused most of the differences between the projected cost of the original program and the actual cost of the 48 program. Rand's studies during the 1960's revealed that due to increased emphasis in research and development, complexity of I systems, and increased reliability requirements, cost growth of major weapon systems significantly increased. Major increases to the cost of weapons systems were due to the fact that the 1960's was a period of rapid growth in technological development with defense weapons being of a highly sophisticated nature necessi-tating longer periods of development. Other reasons contributing to cost growth were bad cost estimating and overly optimistic program planning. An interesting finding was that contractors operating under incentive contracts had far less substantial and less frequent cost growth than cost-plus-fixed-fee contracts. This led to the conclusion by Rand that both the government and con-tractors are most cost conscious when incentive contracts are employed.

PAGE 115

100 In a brief 1970 memorandum report, David'Novick placed the cost growth problem with military systems in perspective with nondefense projects, and identified two issues contributing to the problem of cost overrun: the derivation of the original estimate of the cost of the system; and the control of cost when the project 49 is under way. Novick illustrates this fact that cost growth is neither unique to government or to DOD programs as proven by a variety of experiences. One example used by Novick dealt with construction of a nuclear power plant. A contract awarded by a public utility called for delivery of the nuclear core within four years for $55 million. On the original delivery the con-tractor had run out of funds and'was not able to make delivery. Delivery was actually made several years later and the cost overrun was 200 percent of the original price. Another example Novick cites is a 400 percent overrun for a large communication company when it introduced a new transmission technology. A final example used by Novick was the cost overruns on the Rayburn Annex to the 50 United States House of Representatives Office Building. In conclusion, Novick mentions that in the design and acquisition of future systems, errors will no doubt be made, whether in the procurement of new spacecraft for government use, the introduction of new power plants by privately owned utility companies, or in the purchase of new office buildings and homes by individuals.51 Rand issued a comprehensive report in 1970, by Alvin J. Harman describing a methodology for comparing and predicting cost

PAGE 116

101 experiences of weapon system procurements, using cost factors as a 52 basis. Cost factors are the ratios of actual costs to origi-nally predicted costs. Past experience has shown that in the typical case actual costs exceeded predicted costs. Cost increases can be caused by inaccurate cost estimates or by cost growth arising from such factors as scope change (changing of goals or specifications subsequent to the start of a development program), technological uncertainty, or various inefficiencies. Cost factors \ alone do not, according to Harman, provide sufficient information 53 to distinguish among such contributors. Cost factors are sufficient in themselves for making cost comparisons, since various types of systems may be more or less too complex to estimate accurately. Harman uses an example where one weapon system may have a cost factor of 1.2 and another a cost factor of 1.4. If the first was a short program involving mainly off-the-shelf technology and the second required a major advance in technology and its cost had to be predicted very early in the conceptualization stage of the program the apparent 40 percent increase in costs of the second might well be, in retrospect, considered funds well spent. The 20 percent increase of the first could be considered as evidence of unacceptable bad estimating or inefficient management. An effort to make adequate cost comparisons must take into account the influence on costs of other aspects of procurement, including program length and the degree of

PAGE 117

102 technological difficulty encountered during development.54 Harman developed a model which considers a number of the more significant influences on the size of a cost factor. One influence is program duration. For example, a longer program could result in a high cost factor because of the increased time during which the management of the program is inefficient, or because the cost projection was based on a yearly-design of the system. Another influence on the cost factor is the requirement for advanced performance of the system leading to a significant technological advance and therefore a greater concentrated effort in development of the system. This could result in a high cost factor due to the increased probability of development complexities or because of optimistic cost projections that such difficulties will no longer be encountered. In employing the model, Harman uses two sets of data, the first essentially composed of systems developed in the 1950's and the second from the 1960's. Various structures for the model all indicated that both long-duration projects and significant technological advances usually lead to high cost factors. For programs generally comparable in length and complexity, Harman found that the 1960's procurements would have resulted in actual costs exceeding estimates in essentially the same proportion as with the 1950's programs. Harman found that, on the average, development program duration seemed to have been somewhat shorter and the size of

PAGE 118

103 technological advance sought had been kept somewhat lower than in the 1950's.55 Harman's results only suggest that t.he range of possible cost uncertainty is based on past experience. It must be kept in mind that the government has yet to deal with the fact that weapon systems involving requirements for technological advance will continue to be part of the United States defense structure. Acquisition of such systems will, no doubt, involve unavoidable cost uncertainty. If this uncertainty is to be decreased, there must first be a significant improvement in the effectiveness of procurement strategies, which could require basic changes in organization and decision-making as well as in contract incentives and management systems. Only with confidence in information on schedule, performance, and future threat would cost factors be the 56 sole determinant of cost effectiveness comparisons. In 1971, Robert L. Perry's research was published dealing 57 with system acquisition strategies. This Rand report stated that efforts were made during the 1960's to improve the outcome of major system acquisition programs by changing contractural approaches and by introducing a number of different management reforms, even though typical programs continued to exhibit an average cost growth of about 40 percent (corrected for quantity changes and inflation), a schedule slip of about 15 percent, and final system performance that was likely to deviate by 30 or 40

PAGE 119

104 percent from the original specification. Cost growth development phases of programs were difficult to establish, but generally averaged 50 to 100 percent. Perry's study examines the basic factors contributing to cost growth and suggests two avenues for improvement: cost estimation process could be improved so that cost projections made early in a program would more nearly approxi-mate the final cost outcome; and more fundamental improvements could likely be achieved through some fundamental changes in the 58 acquisition process. An important step in cost estimating improvement, according to Perry, would be to incorporate into the estimating relationship a measure of the technical achievement sought in the program, since an analysis of past programs indicate a strong relation between the degree of advance sought and the subsequent cost growth. Also, Perry observed that the outcome of some European weapon system programs and some United States weapon system programs which were conducted outside the normal Department of Defense procurement channels suggest that two other basic changes are worth consider-59 ing. Both represent an attempt to deal with the significant uncertainties and inevitable changes that occur in major military systems after development has.begun. A separation of the develop-ment phase from the subsequent production phase, both sequentially and contractually, is the first change. Suggested as the second change was conducting the first phase of the development program in a very conservative manner, concentrating first on demonstrating

PAGE 120

105 system performance while delaying the more expensive tasks of detailed production design and demonstration of reliability. Perry found that cost increases appeared to have been accepted in order to meet performance and schedule goals. A product of his analysis was the conclusion that the prediction and control of system acquisition programs had not significantly improved in the 1960's over the 1950's, even though results tended to deviate less from program projections in the 1960's due to lower technological advances and shorter programs which were characteristic of that 60 decade. Three broad candidates for cost growth responsibility were identified by Perry: technical uncertainty; changes in scope; and cost estimating error. Approximately one third of observed cost growth and much of the deviation of system performance from that initially anticipated were found to be attributable to technical uncertainty, as well as some small part of the observed schedule slippage. However, approximately one half of the cost growth and an additional part of schedule slip appeared to be directly related to scope change (changes in program objectives). Residual cost 61 growth was attributed to estimating inaccuracies. Typically in the 1960's, _the total funding requirements projected at the beginning of a program had been slightly exceeded, while development costs and unit costs were found to be higher than anticipated. Increased total funding was frequently avoided by a

PAGE 121

106 reduction in production quantities. Of the sources of cost growth cited above, estimating inaccuracies had the least impact, with only about 15 percent of the observed cost growth due to the 62 imprecision of cost estimating procedures. A report dealing with acquisition policy effectiveness was released by Rand in 1979. In this report, Edmund Dews reviewed the performance, schedule, and cost experience of 31 DOD major programs, and included an indepth review of 26 of the 31 6l programs. While the report concentrates mainly on the Defense Department's experience in the 1970's, there are some comparisons to DOD's major programs in the 1960's. Programs involving substantial hardware competition during or before the start of full-scale development were characterized by substantially lower cost growth than the sample without hardware competition (cost growth ranged from 16 percent to 53 percent). In addition, those projects with hardware competition also did better in terms of program schedules and system performance goals. Ignoring inflation and changes in quantity, the major factors influencing cost growth for the programs of the 1970's were engineering changes, schedule changes, and estimating errors. For the entire 31 programs in the cost analysis sample, schedule changes alone contributed 64 .approximately 40 percent of the total cost growth. Causes of cost growth were considered to be somewhat different in the two periods. For the 1960's, changes in scope

PAGE 122

107 (increased system performance) were found to be largest factor. While inflation was not a problem in the 1960's, it became a significant problem in the 1970's. Schedule slippage was not identified as a significant factor in the 1960's, and cost esti-mating errors were considered to be of minor significance. Dews pointed out that no major acquisition program can be planned and managed with a high degree of efficiency with frequent and unpredictable changes in funding. Schedule slippage and cost growth are closely related and mutual reinforcing effects of program funding instability. Funding instability, externally generated, appeared to be a experience for major programs. In Dew's report, five main causes of cost growth are dealt 65 with and will be briefly reviewed here. Inadequate annual funding was the most frequent root cause for schedule slippage in major DOD weapon systems programs. This reason was given in more than one third of the 31 programs examined by Dews. Approximately 50 percent of the programs at least three years past the decision to proceed to full-scale development had schedule slippage due to inadequate funding. Underfunding was found; in many cases, caused by increasing performance above that called for in the original specification usually results in higher costs. Similarly, the funding shortfall may be traced to an overly optimistic baseline cost estimate, or to unexcepted technical difficulties. With a rise in unit cost and no compensating increase in the program's

PAGE 123

108 annual funding, the most obvious solution to the funding reduction is a cut in the production rate. Funding cuts are often made by Congress because, in their view, there has been inadequate program justification. Unexpected technical difficulties were the second cause 66 according to Rand. Engineering variance was found to result from two basic causes. The first relates _to the additional effort that is required to meet the original requirements. Unanticipated technical difficulties were acknowledged in 11 of the acquisition programs in the 1979 Rand study. Sixty percent of the programs at least three years beyond the decision to proceed to full-scale development indicated unexpected development difficulties as a major cause of cost growth. Changed performance was the third cause of cost growth 67 mentioned. Engineering changes, a change in the performance requirements of major systems, consisted of major restructuring of the programs, adding equipment, providing for increased reliability and maintainability over that originally planned, and a continual upgrading or improvement of systems. These latter cost changes stemming from improved performance were factors in the cost growth of 12 of 31 programs in the Rand study. Cost variance due to estimating errors, the fourth cause of cost growth, was very apparent in six of the acquisition programs in Rand's 31 program sample. It was a significant cause of cost growth in almost one-

PAGE 124

109 third of the programs at least three years the initiation of full-scale development. While some estimating errors were mistakes in estimator judgment, another frequently noted.source of cost growth was the initial omission of expensive system elements such as training or depot equipment. Some new acquisition programs have no previous counterparts to provide a firm basis to make sound cost estimates. Also, military hardware is frequently at the frontier of new technology. Unpredictability was the last area of cost growth identified in this Rand report. Two Air Force programs blamed a part of their cost growth on circumstances that could not be predicted at the time of initiation of full-scale development. These were the F-15 Aircraft Engine Program and the original A-10 Aircraft Program which did not allow for a full demonstration against the A-7 aircraft (i.e., compare the aircraft after each has successfully flown). Shown in Figure 2-4 are the cost growth comparisons of 13 programs in the 1960's. Figure 2-5 depicts the cost growth comparison of 17 programs of the 1970's. It is noted that in the 1970's cost growth was less from both, the mean ratios as well as the dollar weighted mean ratios. Figure 2-6 is an attempt by Dews to categorize the causes of cost growth to the five main areas discussed above, with the 31 programs in his sample. It can be observed that the largest cost growth frequencies were due to

PAGE 125

.. c: l. Figure 2-4 COST 1960's SA!-1PLE 13 PROGRAMS Mtlen retio 1.44 Doller-i9tted meen retio 1.47 Medien ratio .. 1.16 1.2 1.4 1.8 1.8 2.0 Colt1rowtt1 fetor I retio of rwult to pi 1 Figure 2-5 1970's SMIPLE 17 PROGRA.."1S Mtlen ratio 1.34 Doller __ i.,Dd rneen relio 1.20 Medi1n relio 1.24 Calt-11'DWVI fetor I retio of rasult to pi I 110 2.4 SOURCE; Rand Report No. R-2516-DR&E, October 1979, p. 55

PAGE 126

Figure 2-6 CATEGORIES OF COST GROWTH FOR 31 DEFENSE PROGRAMS. 1960-1979 Unexpected Inadequate Technical Changed Estimating Program Funding Difficulties Performance Errors ARMY Patriot s 8 r s Hellfire 8 s UH-60 YAH-64 8 IFV L L L XM-1 L Roland L Copperhead (CLGP) DIVAD Gun M-198 Howitzer I L NAVY F-18 LAMPS lii s L s Aegis s s CAPTOR s L L Harpoon I L Side'lll;inder (AIM-9L) L L L Tomahawk Guided Projectile 8-in. Guided Projectile SURTASS L L TACT AS s Condor s 8 r r AIR FORCE A-10 L 8 B-1 8 s & F-15 8 s 8 F-16 s s E-3A (AWACS) L 8 PLSS DSCS III ALCM GLCM Ke): L = cause of li.rg-e increase. 8 = cause of small increase. r = cause of small reduction. SOURCE: Rand Report No. R-2516-DR&E, October 1979, p. 111 Unpredictable 8 8 95

PAGE 127

112 changed performance and estimating errors. Concluding the review of this Rand report, it is appropriate to include Dew's brief analysis of cost growth in non-defense programs. Cost growth ratios for non-defense programs also appeared to be in the same ballpark as the defense programs. Non-defense programs also involved new technology or other substantial uncertainties. Figure 2-7 depicts these non-defense projects, and as can be observed, the most significant cost growth was experienced in the Trans-Alaska Oil Pipeline project which reached 325 percent. While finding many of the same causes for cost growth that existed during the 1960's, Rand's research efforts during the 1970's reveals other significant factors. These included technical uncertainty and development difficulties. A major cause of cost growth in the 1970's was inflation--specifically, the several years of double digit inflation experienced from the mid-to-late 1970's. Weapons systems cost increased significantly due to inflation, while in the same timeframe, funding did not keep pace with inflation. As a result, inadequate funding caused schedule slips and unit cost increases. During the 1970's, estimating errors also continued to be a prime contributor to cost growth. Rand also found that programs with hardware competition before full-scale development had substantially lower cost growth. In addition, those programs with competition before full-scale

PAGE 128

Figure 2-7 THE CORPORATION STUDIES OF COST GRm-lTH IN MAJOR U.S. NONDEFENSE CONSTRUCTION PROJECTS, 1956-1977 Project and Date of Final Cost Ratio of Final Cost Initial Estimate ($ Millions) to Initial Estimate& Trans-Alaska Oil Pipeline. 1970 7700 4.25 New Orleans Superdome. 1967 178 3.22 Cooper Nuclear Station. Nebr. Pub. Power Dist.. 1966 395 1.75 Dulles Airport, Washington, D.C. 1959 108 1.49 Toledo Edison's Davis-Besse nuclear power plant. Ohio, 1971 466 1.40 Rayburn Office Building, Washington. D.C .. 1956 98 1.34 Rancho Seco Nuclear Unit No. 1. Sacramento, 1967 347 1.24 Frying Pan-Arkansas River Project, Colorado, 1962 54 1.24 Second Chesapeake Bay Bridge, 1968 120 1.10 Bay Area Rapid Transit Authority, 1962 1640 1.04 MEDIAN 1.37 SOURCE: W. J. Mead et al., Transporting Natural Ga.s from the Arctie, American Enterprise Institute for Public: Policy Research, Washington, D.C., 1977. pp. 88-89, quoted in Edward R. Merrow. Stephen W. Chapel, and Christopher Worthing, A Rel:iew of Cost Estimation in New Technologia: lmplicaticns for Enerw Proceu PlafiU, The Rand Corporation, R-2481-DOE, July 1979, p. 38. a After c:orrection to remove effects of inflation and change of project seope. SECONDARY SOURCE: Rand Report No. R-2516-DR&E, October 1979, p. 35. 113

PAGE 129

114 development performed better in terms of program schedules and system performance. To understand betterthe historical developments regarding cost growth of major weapons systems, the results of an interview with David Novick is presented in the following section. Interview with David Novick On February 10, 1984, an interview was conducted with David Novick, a long-time associate of the Rand Corporation, and one of the pioneers in the cost estimating and cost growth analysis 68 areas. When asked what he saw as the major causes of cost growth in research and development projects, he responded with "lack of realism" in estimating the job. He stated that the engineers and scientists grossly tend to underestimate anything that they want. Novick also reaffirmed that major causes of cost growth for major systems were technical uncertainty, funding instability, and system or technological improvement. Novick also revealed that there is a tendency for the government planners to say a specific project is state-of-the-art when it really is not. In relating technical uncertainty and the desire to improve the system, Novick sees these as feeding each other. For example, the desire to improve often leads to technical uncertainty, and technical uncertainty causes the planners to search for alternatives which is an opportunity to go after desired improvement. Based on his studies, Novick believes that the

PAGE 130

115 military contributed to the cost growth of major systems with their habit of always wanting something better. There is a tendency for the military departments to get the available funding as large as possible and fit a optimistic program to that funding level. In that situation, cost growth will typically develop. In dealing with the issue of contractor business motives in the defense and aerospace work, Novick does not believe that profit maximization is paramount. He sees other contractor motivations having higher rank over profit maximization, including, the desire to maintain staffing, keeping the business base high to cover overhead costs, and being in a position to be more competitive in capturing new business opportunities. Novick made the point that defense and aerospace firms are not in business for the shortterm, therefore these firms will forego short-term profit and opt for long-term business stability. This is a very interesting qualification for the generally accepted profit maxim. In concluding his discussions on causes for cost growth for major systems, Novick added that "lead time" (the period of time from contract authorization to delivery) and inflation were difficult to deal with. Many of the materials utilized in weapons and aerospace systems, such as titanium sheet metal and stainless steel ball bearings, have a long manufacturing process. In regard to inflation, Novick believes no one can really predict inflation accurately. The longer the lead-time for a system the more

PAGE 131

116 uncertainty is placed into the cost of the system. While the Rand Corporation has done many extensive studies on the R&D acquisition and cost process, there are other sources to be considered. The General Accounting Office Reports This portion of the literature review covers the results of an analysis of pertinent U.S. General Accounting Office reports deemed pertinent to the cost growth aspects of major systems. The U.S. GAO is the investigative arm of the U.S. Congress and performs investigations on behalf of the Congress and executive agencies. As with the Rand reports, due to the organization of the subject matter in each GAO report, it was not practical to deal with the reports in any manner other than chronologically. However, the reader will find considerable consistency in the findings of the reports in regard to the reasons for cost growth on major systems. A GAO report of 1973, covering cost growth in major weapon systems for the 1960's, found significant cost increases. In analyzing the cause, this report revealed that the major reasons for cost growth were demands for greater capability of new systems, increased system complexity in order to obtain greater capability, inflation, and management of the system development and produc-69 tion. Findings in this report indicated that the military services constantly demanded that the performance and capabilities of new systems exceed those to be replaced, thereby increasing

PAGE 132

complexity of the system. Complex management organizations were frequently created to monitor these programs in detail regarding 70 cost, schedule, and performance objectives. 117 Based on an analysis of .45 systems in the above mentioned GAO study, it was found that inaccurate estimating by both contractors and government, represented 25 percent of the cost growth; inflation accounted for 30 percent of the cost growth; and changes in specifications, quantities, and delivery schedules 71 accounted for 45 percent of the cost growth. These areas of cost growth are generally consistent with the findings of the Rand studies. In 1976, the GAO issued a report covering the financial status of major acquisitions. This report includes financial data on 753 civilian and military major acquisitions estimated to cost $452 billion at completion, an increase of $176-billion, or 64 percent, over initial estimates for these systems.72 Acquisi-tions included in the report were past the planning stage and in development, test, production or construction phases. The Depart-ment of Defense (DOD) defines major programs as those with an estimated research, development, test, and evaluation (RDT&E) cost in excess of $50 million or an estimated production cost in excess of $200 million. Most civil agencies generally have not estab-lished criteria for defining major acquisitions; however, GAO used a uniform threshold of $25 million to define civil agencies' major

PAGE 133

118 acquisitions. For 201 projects, costs increased by $151 billion. Of that amount, $71 billion, or 47 percent was attributed to inflationary (economics) increases. Other increases were attri-buted to changes in: quantities ordered; system engineering characteristics; spare parts or similar types of support needs; delivery dates; and estimating corrections.73 Analyses of 148 civil acquisitions having 100 percent or more cost growth indicated that economic change, primarily infla-tion, accounted for 47 percent of the total cost growth of $90 billion, and was the principal cause of increased project costs. Analyses of the DOD data on 53 major acquisitions indicate that cost growth amounted to $61 billion or SO percent of the initial estimate of $121 billion for those 53 programs. As with the civilian projects, the major cause of the cost growth was economic change, mainly inflation, accounting for 46 percent of the total growth. Schedule or delivery changes accounted for only 18 percent of the cost growth for DOD's programs and had a negligible impact 74 on the civilian projects. According to the General Accounting Office (GAO), cost growth of major federal acquisitions has been a long standing problem. Their 1982 report dealing with the need for better cost reporting for major acquisitions reflected data collected on 802 projects, 486 which were acquisitions for federal ownership and 316 75 were grant acquisitions. This report reflects a GAO finding

PAGE 134

119 the cost of 376 projects increased a total of $318 billion over the congressional budget estimates, that is, total estimated cost of projects provided the Congress in initial budget justifications. In addition, the cost of 465 projects increased by $258 billion over baseline estimates, which is the initial congressional budget estimate adjusted for changes in scope. From a schedule standpoint, the data provided showed that 139 projects out of 170 slipped their completion dates by more than six months. Based on the data provided from the federal agencies, the GAO could not determine the extent which the schedule slippage contributed to the cost growth. Based on the quantity data provided to the GAO from the agencies, 57 projects out of 146 showed variances in excess of 25 percent. Here again, the extent to which these quantity variances contributed to cost growth could 76 not be determined by the GAO. Reasons cited for the cost growth in major acquisitions, which were consistent with prior GAO reports, included increases in requirements and scope, failure to provide for adequate inflation, failure to include all costs of the projects poor estimating practices, use of optimistic completion schedules, stretched-out procurement schedules, and delays in funding by the Congress, the 77 Office of Management and Budget or by agency management. While cost growth has been and is a continuing problem in federal acquisitions, GAO believes that established reporting

PAGE 135

120 systems, like DOD's selected acquisition reporting system, would provide a mechanism for periodically reporting on project status and progress and specific causes of cost, schedule, and performance variances. Such data are not readily available on civil programs and those DOD programs not on selected acquisition reporting. In order to provide a first step for measuring progress and early identification of real and potential problems, the GAO concludes that Congress could help minimize cost growth by requiring the agencies to establish a reporting system for major civil acquisi-. 78 tion similar to DOD. A 1983 GAO study dealt with defense spending in relation-ship to the federal budget covers historical trends in defense outlays and total obligational authority, defense policy and its relationship to the budget, long-term factors which influence defense policy and spending, and questions for use when considering 79 public policy and corresponding budget decisions. One of the issues raised in this report deals with the cost growth problem of defense systems. Restructuring the United States defense forces investment has focused congressional attention on the persistent problem of weapon system cost growth. Based on GAO's analyses, the major factors which contribute to cost growth include inflation, cost estimates, high risk system design, program stretch-out, changes in specifications, budgeting for future cost, and lead

PAGE 136

121 times (time from contract award to delivery of the article). Although cost growth for defense systems is not a new problem, 'it became more visible due to the high inflation rates of the 1970's and early 1980's. Inflation during that period was greater for many types of defense systems than for the economy as a whole. This study indicates that the DOD, as well as other government agencies, were required by the Office of Management and Budget to use prescribed inflation rates in budget submittals to Congress, and in many instances, the actual inflation rate was 100 percent greater than those rates used in budget submissions. This, of course, substantially contributed to the cost growth of many of the 80 weapon systems. Funding instability, technical complexity, and technical advancement occurred in over 50 percent of the major acquisitions and contributed significantly to the cost growth problem according to the GAO 1983 study. In addition, the GAO found that there was a tendency to make technology improvements to upgrade and improve the weapons systems capacity in order to meet redefined mission objec-tives, or just simply to put the latest equipment in a particular system. An example cited by GAO is the M60 battle tank in which the per unit cost has risen from $580,000 to $1,292,000 (both figures are expressed in 1982 constant dollars). Technological 81 improvements account for the entire amount of the increase shown. A comprehensive report was issued by the GAO in 1983 reflecting a

PAGE 137

122 summary of recent GAO reports.82 This report is a summary of issues and recommendations of previous reviews the GAO had made of 17 selected weapon systems. GAO believes that had those recommendations been implemented, they would have helped to minimize development risk and en.sure effectiveness, improve disclosure to the Congress, reduce cost, and improve management of programs. Although the report does not deal sp-ecifically with the causes of cost growth in major weapon systems, the GAO did recommend that, due to the rising cost of acquiring such systems, increased attention needed to be directed at identifying -opportunities for reducing development and acquisition costs. Of particular significance, is the GAO's observation that because Congress has to make decisions and allocate national resources among government programs, it was essential that the most accurate and complete information available be provided by DOD on the cost, schedule, and performance of weapon systems. One of the specific examples cited by the GAO was the exclusions of certain known cost items, such as simulators for pilot training, when DOD submitted the cost estimates for the B-1 bomber to Congress. These types of omissions (total omissions were in excess of $1 billion), obscure congressional visibility of the acquisition, noted the GAo.83 Conflicting guidelines and instructions for reporting program cost estimates were found in the Office of Secretary of Defense and the Air Force. In regard to cost growth impact, the

PAGE 138

123 GAO concluded that it was essential that accurate and informative data on the status and progress of weapon system programs be made available to the Congress and DOD's top level management in order for them to make informed decisions. Without accurate data, incomplete, misleading, or inaccurate status reporting could result in congressional and DOD decisions that would not otherwise be made. 84 In summary, one can conclude that the findings of the GAO reports, reference the major reasons for cost growth are largely consistent with the findings in the Rand studies. For example, GAO found that demands for greater system capability and increased system complexity played a large role in increasing the cost of major systems. Changes to specifications, changes, and changes observed by the GAO were that major weapons systems typically are high design systems, thus, being susceptible to significant cost growth. Playing a large role in cost increases for the past 20 years was inflation. To cope with the inflationary increases, without additional funding, the military services were forced to stretch-out delivery schedules; thus, driving up unit cost. Long lead time for military systems was also mentioned as a contributor to cost growth. Lastly, consistent with the Rand reports, the GAO found inaccurate estimating by both the contractor and the government, which contributed to cost growth of major systems.

PAGE 139

124 Case Studies on Cost Growth A third segment of cost growth literature covers three pertinent case studies on cost growth of major defense weapon 85 systems conducted in 1981 by the Committee on Armed Services. This three case study included the Black Hawk Helicopter Program, the Patriot Missile Program, and the Air Launched Cruise Missile (ALCM) Program. A primary reason the panel selected the Black Hawk Program as a case study was because of its history of significant unanticipated cost growth due primarily to poor cost estimating. Selection of the ALCM Program provided an opportunity to evaluate the impact of competition in the procurement process which appeared to account for the ALCM Program's low cost growth. An unusually long development period that the .. Patriot Missile Program had encountered accounted for a significant amount of this program's cost growth, and hence, the reason for selection of that program. Field hearing testimony indicated that the manufacturer of the Black Hawk helicopter had made poor estimates of initial production requirements which was a major factor in the resulting cost growth of 237 percent. Other factors, were unrealistic inflation estimates, program extensions, and the lack of recent production experience early in the program. According to this Congressional report, one of the contractor's representatives told the panel that his company did not fully understand the complexi-ties of manufacturing the number of helicopters requested by the 86 U.S. Government.

PAGE 140

125 Testimony on the Patriot Missile Program indicated that the reasons for cost growth were program stretch-outs, scheduling changes, unrealistic inflation estimates, changes in performance testing requirements, increases in the cost of government-furnished 87 equipment, poor cost estimating, and design changes. An analysis of the finding relating to the ALCM case study indicated that the program had experienced development and production cost increases of 26 percent and 8 percent, respectively. Some of the major contributors to cost growth were poor cost estimating, inefficient production rates, and changing performance requirements. The ALCM was the only case of the three studied in which the concept of dual-sourcing, where more than one major contractor was involved in production, was introduced early in the acquisition process. Dual-sourcing was utilized for the inertial navigation system, the engine, and the missile radar altimeter.88 The congressional panel's conclusion was that there were numerous factors that contribute to unanticipated weapon systems cost growth. These include: poor cost estimates by the government and contractors; the use of unrealistic inflation rates; program stretch-outs by the Congress and DOD; unstable and inadequate funding; changes in mission requirements and technical specifi-cations; high risk design; poor management; and the lack of competition particularly during the production phase of the acquisi-tion process. A conclusion by the congressional panel was that

PAGE 141

DOD's policies and procedures controlling competition should be improved to encourage competition, where practical, during all phases of the acquisition process for major weapon systems--not just during the development phase.89 126 Again, the reasons for cost growth of major weapon systems found in these case studies were largely consistent with the causes found in the analyses of the Rand reports and the GAO studies. Other Reference Material on Cost Growth In their work on cost growth of major defense systems in the 1950's, Marshall and Meckling found that the reasons for large cost growth was a function of attempting to make firm projections 90 prior to the program becoming well defined. They also noted a correlation between cost growth and the lack of incentives in the contracts, including the absence of competition. In addition, they found that high degrees of technical advances in the state-of-the-91 art could lead to significant cost growth. Based on a sample of 12 development weapons programs, Peck 92 and Scherer found cost growth averaged 220 percent. Their analysis indicated that a premium was typically placed upon tech-nical performance goals, usually at the expense of cost and, to some degree, schedule objectives. They postulated that the unpre-dictability of defense system program results manifested itself mainly on the cost dimension. 'Peck and Scherer found that the most significant causes for cost growth were unexpected technical

PAGE 142

127 uncertainties. They also noted that contractors have recognized that achieving technical excellence and meeting schedules were much more important in securing additional business than cost control. 93 A review of the work of Scherer dealing with cost growth, indicates that competitive optimism at the beginning of a development program encourages innovation and firms tended to submit excessively optimistic predictions of the proposed program's technical expectations, schedules, and cost estimates. In the past, several factors that have encouraged "buying in" to new development programs with unreasonable optimistic predictions of the cost were indicated. Financial penalties for this type of approach have been practically nonexistent; rather, the competing firms could gain most through being exceedingly optimistic. For example, according to Scherer, since military officials typically place emphasis on projected technical advancement in selecting the contractor for a new program, companies have a significant motiva-tion to be very optimistic in their proposals. Scherer postulates that an optimistic type of an approach is a major reason that most weapon system development programs of the 1940's and 1950's began with overly optimistic technical, schedule, and cost goals. 94 Scherer's case studies consistently revealed that enhancing weapon system quality and reducing development schedules had a considerable higher priority by contractor and DOD officials than

PAGE 143

128 minimizing development cost. Cost considerations typically have a vital role only when quality factors are satisfied.95 Fox's extensive study of the defense acquisition system, contains some significant implications regarding cost growth of 96 major weapon systems. Fox begins with comments by former Deputy Secretary of Defense David Packard. In 1970, Mr. Packard cited cost growth on major defense programs were due partly to the fact that there had been bad management of several defense programs in the past--bad management in the defense industry and the govern-ment. Fox further points out that senior military officers are given no incentives to trim the size of their organizations, or to reduce programs that experience large cost increases. Rather, military personnel are motivated to support continual increases in 97 budgets for new weapons. In his research on estimating the cost of large development programs, Fox conducted extensive interviews with Pentagon officials. He noted that one Pentagon official commented that if program cost estimates are too high, everyone would be the loser because the chances for the program being approved diminished substantially. However, if the estimate was based on rather optimistic situations and is lower than the facts would justify, the chances for the program being approved would be much higher due to the lesser impact on DOD's budget. Even if the actual cost of such programs turned out to be significantly higher, the responsi-bility would likely be obscured due to many things, such as, the

PAGE 144

129 rotation of program managers, unforeseen technical problems, changes in the requirements, or funding plan changes. Fox further points out that on large, complex advanced technology programs, only those individuals involved with the program and its technical complexities will really understand the cost implications of such 98 programs. Other evidence pointing to optimistic cost projections is mentioned by Fox, based on another interview with a senior military 99 officer. In essence, the officer commented that when one estimates the cost of a program which is believed to be needed by the country, one has a tendency to be very optimistic and to estimate the cost of the project based on current technical state-of-the-art. Then, when design decisions are made, everyone wants to include the technology. If one is completely objective in making cost estimates, there is a risk that the program would be terminated.100 Another real problem contributing to cost growth, according to Fox, is that contractors typically submit proposals based on the size of the government agency's budget for a given program. Fox observes that on most multihundred million dollar programs, the amounts proposed by contractors typically within a few percent 101 of each other and also the government cost estimate. A major reason for cost growth on major research and development contracts relates to contract changes, and therefore, much of the literature

PAGE 145

review contained under the cost growth section of this chapter applies to contract changes. Contract Changes 130 This section of the literature review covers the contract changes aspect of government contracts. A primary purpose of contracted research and development (R&D) programs is to advance scientific and technical knowledge and apply that knowledge to the extent necessary to achieve agency and national goals. Unlike contracts for supplies and services, most R&D contracts are directed toward objectives for which the work or methods cannot be precisely described in advance. Therefore, in major R&D projects for defense and aerospace systems, there are usually large numbers 102 of contract changes that add to program costs. A major reason for having a changes clause is to provide the government with the required flexibility which it needs during performance of the contract. While allowing the contracting officer to issue changes to the drawings and specifications which alter the physical character of the product, the clause also allows the contracting officer to order changes in the method and manner of performance of the work. Changes can be ordered by the government in a unilateral manner within the general scope of the contract. It does not matter whether contractors agree or disagree with the ordered change, they are contractually obligated to

PAGE 146

131 proceed with the work. The termsof the clause guarantee the contractor that an equitable adjustment will be made in the con-. 103 tract price and delivery schedule. Not all changes clauses are precisely the same, even though most of them include the same general characteristics. Shown below is the changes clause, as it appears in the NASA Procurement Regulation, and is included in all NASA R&D contracts (essentially the same clause is used for DOD contracts): Changes Clause The contracting officer may at any time, by a written order and without notice to the sureties, if any, make changes, within the general scope of this contract, in any one or more of the following: (i) drawings, designs, or specifications; (ii) method of shipment or packing; (iii) place of inspection, delivery, or acceptance; and (iv) the the amount of government-furnished property. If any such change causes an increase or decrease in the estimated cost of, or the time required for, performance of this contract, or otherwise affects any other provision of this contract, whether changed or not changed by any such order, an equitable adjustment shall be made (i) in the estimated cost of delivery schedule, or both; (ii) in the amount of any fee to be paid to the Contractor; (iii) in such other provisions of the contract as may be so affected, and the contract shall be modified in writing accordingly. Any claim by the Contractor for adjustment under this clause must be asserted within sixty (60) days from the date of receipt by the Contractor of the notification of change; provided, however, that the Contracting Officer, if he decides that the facts justify such action, may receive and act upon anysuch claim asserted at any time prior to final payment under this contract. Failure to agree to any adjustment shall be a dispute concerning a question of fact within the meaning of the clause of this contract entitled "Disputes." However, nothing in this clause shall excuse the Contractor from proceeding with the contract as changed. In the foregoing

PAGE 147

132 clause, the period of "sixty (60) days" within which any claim for adjustment must be asserted may be reduced to a period of not less than "thirty (30) days." In accordance with 10 U.S.C. 2306 (f), prior to the pricing of any contract change or modification that is expected to exceed $500,000, except where the price is based on adequate price competition, established catalog or market prices of commercial items sold in substantial quantities to the general public, or prices set by law or regulation, the contracting officer shall require the contractor to furnish a Certificate of Current Cost or Pricing Data and shall assure that the contract includey0gr is modified to include a Defective Pricing Data clause. Fox classifies changes in weapon system contracts as con-figuration changes, task changes, or program changes. Configura-tion changes refers to alterations to the design or configuration of a system being manufactured for delivery to the government. These types of changes include alterations to components or any of the various parts of, for example, spacecraft or aircraft. It is common practice in aircraft and spacecraft R&D projects to upgrade the system to the latest state-of-the-art components and this 105 increases the amount of changes. Task changes typically are not hardware changes but rather include additional testing, laboratory analyses, special studies, and items of that nature. Program changes generally involve very significant and costly changes, such as, changes in delivery schedules, rates of funding, and major technical perfor-mance enhancements. Significance of changes can also be evaluated by their impact to program cost growth. Fox's 1964 study of 139

PAGE 148

weapon system contracts indicated that cost growth attributed to changes amounted to 50 percent, or $7.6 billion compared with original program prices totaling $15.3 billion. 106 Another example is that during the height of the F-111 Aircraft Program, 1,500 changes were issued amounting to over $1.5 billion.107 133 In his work of change management for major weapon systems, Reece found that advocates of better change cost management usually offer as their foremost proposal a reduction in the number of 108 changes. One would conclude from this suggestion that un-necessary changes are being incorporated in the various weapon system programs. Reece proposed that to reduce the number of changes, steps such as extending the development process, extending prototype development, and stop development-production overlap should be taken. In his research experience, he found that the typical managers responsible for program costs strongly believe that a significant portion of contract changes are unnecessary, or just nice-to-have modifications. Reece concludes that reducing the number of changes is not really managing change. costs, but rather, 109 is avoiding change. One of the major problems in controlling the cost of changes, according to Fox, is the fact that all change orders are issued on a sole-source basis (to the firm who holds the prime contract). Because it is not feasible to transfer the work contemplated by change orders to other firms, this places the

PAGE 149

134 contractor in a particularly strong position to negotiate. It i$ often a long time span between issuance of the change and negotia-tion of the change and, many times, a substantial portion of the work is often completed when negotiations take place. When negotia-tions are delayed until the work authorized by the change order is substantially complete, the contractor is essentially assured to recover all actual costs and has nothing to gain by attempting to control or reduce those costs.110 There is clear evidence from the literature that change management and cost control of changes can be significantly enhanced by controlling only a few changes that typically constitute the majority of the cost of changes. For example, Fox found that 10 percent of the contract changes on a major aircraft development program represented over 80 percent of the cost of the 111 changes. Consistent with Fox's findings, Reece's research of a major weapon system development indicated that only nine percent of the changes represented over 90 percent of the total cost 112 increase for changes. McGlashan conducted an extensive research of cost growth on a major spacecraft development project and found that approximately five percent of the changes repre-113 sented over 80 percent of the cost of changes. This dimension of contract changes will be examined specifically in the Shuttle Orbiter case in Chapter 5.

PAGE 150

135 Summary of the Literature Review This analysis of the literature has formed a major conceptual framework and background for this study. The literature surveyed in this research supports the case of the award fee type of contract for major R&D acquisition. Professor Hunt's extensive research in award fee contracting provided a comprehensive theoretical framework for the use of award fee contracts in major R&D undertakings. A review of the cost growth literature was made of numerous publications of the Rand Corporation, General Accounting Office (GAO) reports, and selected books, articles, and case studies. Rand's research indicated that cost growth problems during the 1960's was largely contributed to by increased system performance, while during the 1970's, other causes of cost growth were technical uncertainty, development difficulties, and inflation. From reviewing the GAO reports, it was found that their assessment of the major reasons for cost growth were generally consistent with the findings in the Rand studies. Case studies reviewed in this research were basically consistent with the findings of the Rand and GAO reports. There was clear evidence in the literature review of contract change management that the management and cost control of changes can be greatly enhanced by controlling only a small number of changes that typically represent the significant part of the cost of changes. This literature

PAGE 151

136 review chapter has provided the foundation and background necessary to proceed to the research evaluation chapters dealing with the award fee type of contract for the Orbiter project, cost growth of the Orbiter project, and contract changes on the Orbiter contract.

PAGE 152

NOTES -CHAPTER II 1 National Aeronautics and Space Administration, Procurement Regulation, NHB 5100.2C, (Washington, D.C.: u.s. Government Printing Office, 1981), p. 3-4:1-2. 2Ibid. 3 Frank T. Meneely, "Determining the Appropriate Contract Type," Concepts, The Journal of Defense Systems Acquisition Management (Summer 1982), pp. 44-9. 4Ibid. 5National Aeronautics and Space Administration, Procurement Regulation, Part 3.402. 6 Richard F. Demong and Daniel E. Strayer, "The Underlying Theory of Incentive Contracting," Defense Management Journal (1st Quarter, 1981), p. SO. 7 5. Ibid., p. 8 6. Ibid., p. 9 7. Ibid., P 10 National Aeronautics and Space Administration, Cost Plus Award Fee Contracting Guide, NHB 5104.4, (Washington, D.C.: U.S. Government Printing Office, 1967). 11Ibid. 12Ibid., p. 2. 13Ibid. 14 Raymond G. Hunt, "Concepts of Federal Procurement: The Award Fee Approach," Defense Management Journal (2nd Quarter, 1982), p. 14. 15National Aeronautics and Space Administration, Cost Plus Award Fee Contracting Guide, pp. 4-5. J p. 8. 17Ibid., pp. 13-6. 18Ibid., pp. 2-3.

PAGE 153

138 19 National Aeronautics and Space Administration, Procurement Regulation, pp. 3-4: 12-3. 20 The "Disputes" clause all,.ows for the contracting officer's decision to be appealed to the NASA Board of Contract Appeals, or for DOD to the Armed'Services Board of Contract Appeals, for resolution of disputes between the contracting officer and the contractor. Contractors may take matters directly into federal court if they so chose. 21 Raymond G. Hunt, "Managing R&D in a Federal Setting", paper presented to Academy of Management, Public Sector New York (August 18, pp. 13-4. 22 "Concepts of Federal Procurement: The Award Fee Approach, 11 pp. 14-5. 23Ibid. 24Ibid. 25Raymond G. Hunt, "Use of the Award Fee in Air Force System and Subsystem Acquisition," Final Report to Air Force Business Management Research Center, Wright-Patterson AFB, Ohio, March 1980, pp. 21. 26Ibid. 27 Raymond G. Hunt, "Contractor Responses to Award Fee Contracts," National Contract Management Journal 15 (Winter 1982), pp. 84-90. 28Ibid. 29Raymond G. Hunt, "R&D and Award Fee Contracting," The Journal of the Society of Research Administration:-6 (Summer 1974, p.39. 30 David Novick, The Cost of Advanced Weapons, Report No. P-2556, (Santa Monica, Calif.: The Rand Corporation, pp. 1-3. 31Ibid. 32Ibid., pp. 6-7. 33rbid., p. 17. 34rbid., pp. 20-1.

PAGE 154

139 35 David Novick, Costing Tomorrows Weapon Systems, Report No. RM-3170-PR, (Santa Monica, Calif.,: The Rand Corporation, 1962), pp. 1 and 11. 36 Ibid., p. 4. 37Irving N. Fisher, A Reappraisal of Incentive Contracting Experience, Report No. RM-5700-RR, (Santa Monica, Calif.: The Rand Corpor.ation, 1968), p. 3. 38Ibid., pp. 43-4. 39Ibid. 40 Irving N. Fisher, Improving the Effectiveness of Contracting, Report No. P-3870, (Santa Monica, Calif.,: Rand Corporation, 1968), pp. 1-2. 41Ibid. 42Ibid., p. 7. 43Ibid., pp. 10-1. 44Ibid. Incentive The 45 Robert L. Perry, System Acquisition Experience, Report No. RM-6072, (Santa Monica, Calif.: The Rand Corporation, 1969). 46Ibid. 47Ibid., pp. 39-40. 48Ibid., p. 42. 49navid Novick, Are Cost Overruns A Military-Industry-Complex Speciality?, Report No. P-4311, (Santa Monica, Calif.: The Rand Corporation, 1970), p. 4. 50 Ibid. p. 6. 51 Ibid. 52 Alvin J. Harman, A Methodology for Cost Factor Comparison and Prediction, Report No. RM-6269, (Santa Monica, Calif.: The Rand Corporation, 1970), p. v. 53rbid.

PAGE 155

54Ibid. 55rbid., pp. vi-vii. 56 Ibid. p. vii. 57 Robert L. Perry, System Acquisition Strategies, Report No. R-733-PR, (Santa Monica, Calif.: The Rand Corporation, 1971), p. v. 58rbid. 59Ibid., v-vi. 60Ibid., p. 14. 61Ibid., p. 16. 62Ibid. 63 Edmund Dews, et. al., Acquisition Policy Effectiveness: Department of Defense Experience in the 1970's, Report No. R-2516-DR&E, (Santa Monica, Calif: The Rand Corporation, 1979), pp. vii-viii. 64Ibid. 65Ibid., pp. 92-4. 66Ibid. 67 Ibid. 68rnterview with David Novick, former staff member of the Rand Corporation, February 10, 1984. 69u. S. General Accounting Office, Cost Growth in Major Weapon Systems, B-163058, (Washington, D.C.: U.S. Government Printing Office, March 26, 1973), pp. 1-14, 26-31. 70Ibid. 71Ibid. 140 72u. S. General Accounting Office, Financial Status of Major Acquisition PSAD-77-62, (Washington, D.C.: U. S. Government Printing Office, June 30, 1976), pp. 2-3. 73Ibid.

PAGE 156

74 Ibid., pp. 2-3, 93-95. 75u. s. General Accounting Office, Status of Major Acquisitions as of September 30, 1981: Better Reporting Essential to Controlling Cost Growth, MASAD-82-24, (Washington, D.C.: U.S. Government Printing Office, April 22, 1982), p. ii. 76rbid. 77Ibid., pp. 5, 9. 78rbid., p. 14. 141 79u. S. General Accounting Office, Defense Spending and Its Relationship to the Federal Budget PLRD-83-80, (Washington, D.C.,: U.S. Government Printing Office, June 9, 1983), pp. 36-7. 80rbid. 81Ibid., p. 45. 82 U.S. General Accounting Office, Weapons Systems Overview: A Summary of Recent GAO Reports; Observations and Recommendations on Major Weapon Systems, NSIAD-83-7, (Washington, D.C.: u.s. Government Printing Office, September 30, 1983), pp. 7, 72-3. 83rbid. 84rbid. 85 U.S. Congress, House of Representatives, Report of the = Special Panel on Defense Procurement Procedures of the Committee on Armed Services "Weapons Acquisition Policy and Procedures: Curbing Cost Growth", 97th Cong., 1st sess., 1982, p. 2. 86rbid. 87Ibid., pp. 2-3. 88rbid. 89rbid., p. 11. 90 A. W. Marshall and W. H. Meckling, Predictability of the Costs, Time, and Success of Development, Report No. P-1821, (Santa Monica, Calif.: The Rand Corporation, 1969).

PAGE 157

91Ibid. J. Peck and Frederic M. Scherer, The Weapons Acquisition Process: An Economic Analysis (Boston: Harvard University Press, 1962), pp. 19-24. 93 Ibid., pp. 593-94. 94 Frederic M. Scherer, The Weapons Acquisition Process: 142 Economic Incentives (Boston: Harvard University Press, 1962), pp. 22-9. 95 Ibid., pp. 33-4. 96J. Ronald Fox, Arming America: How the U.S. Buys Weapons (Boston: Harvard University Press, 1974), pp. 81-2. 97Ibid. 98Ibid., pp. 159-60. 99Ibid., p. 161. 100Ibid. 101Ibid., pp. 165-67. 102 Department of Defense, General Services Administration, and National Aeronautics and Space Administration, Federal Acquisition Regulation, (Washington, D.C.: U.S. Government Printing Office 1984), Part 35.002. 103Ralph c. Nash, Jr., Government Contract Changes (Washington, D.C.: Federal Publications, Inc., 1975), pp. 35-9. 104National Aeronautics and Space Administration, Procurement Regulation, Part 7.404-1. 105 Fox, p. 363. 106Ibid., pp. 364-66. 107Ibid. 108JamesS. Reece, "The Management Change: A Catchword or an Opportunity," National Contract Management Journal (Spring 1971), pp. 127-28. 109Ibid.

PAGE 158

110 Fox, pp. 376-79. 111Ibid., p. 366. 112 Reece, 134. 113 Robert McGlashan, Jr., "An Analysis of Cost Estimate Growth on a Complex Development Project," (Ph.D. Dissertation, University of Texas at Austin, 1969), pp. 42-8. 143

PAGE 159

CHAPTER III THE AWARD '.FEE TYPE OF CONTRACT FOR THE ORBITER PROJECT In research and development (R&D) procurement, the type of contract has a major impact on contract results. In this case, selection of a cost-plus-award fee (CPAF) type of contract for the Space Shuttle Orbiter project was geared specifically to the R&D nature of the project. The technical complexities and uncertainties that are typically present in such a major R&D undertaking were contemplated for the Orbiter project, thus, lending additional credence to the use of the award fee arrangement. There was general agreement among all National Aeronautics and Space Administration (NASA) officials that the CPAF type of contract would be the most appropriate instrument for the Orbiter project. A CPAF arrangement was considered to be the most appropriate type of contract to motivate the contractor to strive for excellence in management, technical, schedule, and cost performance. Equally important, NASA desired to have a contractual instrument that would provide the flexibility necessary to permit changing areas of emphasis as the evolved; thereby facilitating the placement of proper manage ment attention on those elements of cost, performance, and schedule. The specific elements of the Orbiter award fee arrangement and process are discussed below.

PAGE 160

145 The Orbiter Contract Award Fee Plan An important element of the award fee contract is the criteria upon which the contractor will be evaluated to determine the amount of fee (profit) earned. Evaluation criteria for the Orbiter contract were establi'shed covering technical and cost performance. Below are the general criteria with subcriteria for periodically assessing the contractor's performance: 1. Performance Award Fee Criteria a) Timely management visibility into potential and actual problem areas to enable contractor and NASA management to achieve Space Shuttle Program goals in an efficient manner and the attainment of these goals by the contractor. b) Application of effective problem-solving techniques. c) Operational, technical, and general management practices which result in timely delivery of quality hardware and services. 2. Cost Award Fee Criteria a) Effective cost control for the current fiscal year. b) Management effectiveness in keeping total DDT&E cost within target cost. c) Effectiveness in maintaining a proper balance between DDT&E costs and the estimated impact on production cost and operational cost-per-flight. While the above broad criteria established the general areas

PAGE 161

146 of performance emphasis, detailed areas of emphasis within the parameters of the broad criteria were established prior to the beginning of each six month evaluation period. While it was NASA's responsibility to identify the areas of emphasis within the criteria, the contractor was afforded the opportunity to submit to NASA recom mended areas of emphasis and more detailed evaluation criteria for the ensuing evaluation period. After consideration of the contractor's recommendations, if any, NASA provided the contractor with the specific areas of work to be emphasized and the detailed criteria to be used for the specific evaluation period. Consistent with the contract terms, the contractor was notified of the detailed evaluation criteria prior to the beginning of the evaluation period. After conclusion of each evaluation period, the contractor's performance was evaluated by an award fee evaluation board (AFEB). This evaluation was conducted in a manner to evaluate the contractor's performance in relation to the evaluation criteria established for the specific period. A.-fee determination official (FDO), who was the Director of the Johnson Space Center, and who was not a member of the AFEB, determined the actual amount of fee earned by the contractor. Subsequent to each evaluation period, and prior to the FDO's fee determination, the contractor was afforded the opportunity to submit a self-appraisal report to the government covering the contractor's evaluation of its performance during that period. After consideration of the contractor's self-appraisal, the AFEB submitted its final report to the FDO with a recommended fee.

PAGE 162

147 At the same time evaluations, conclusions, and fee recommendation were submitted to the FDO, the contractor was furnished a copy of the AFEB's evaluations and fee recommendation. Within 15 days after receiving the AFEB report, the contractor had an opportunity to submit for reconsideration of the FDO proposed evaluations or conclusions, or exceptions to the evaluations, conclusions, or fee recommendations of the AFEB. After consideration of the AFEB recommendations, any recommendations or exceptions from the contractor, and any other pertinent information available, the FDO made a fee determination. As stated earlier in this study, award fee decisions of the FDO are not subject to the "disputes" clause of the contract. However, the contractor was afforded some measure of due process by having an opportunity to submit a self-appraisal report and by receiving an opportunity to comment directly to the FDO on the AFEB report. In making its evaluation, the AFEB chose one of five adjective ratings, each of which has a numerical range of values. These adjectives ranging from: excellent, with a range of over 90 through 100; good, with a range over 80 through 90; satisfactory, with a range of over 70 through 80; marginal with a range of over 60 through 70; to unsatisfactory for 60 and below. Figure 3-1 depicts the official contractual definition for each of the adjective ratings. The midpoint of the satisfact'ory range represents a basic point of departure in making the periodic evaluation since this point is

PAGE 163

Adjective -Grade Range Figure 3-1 DEFINITION OF ADJECTIVE RATING TERMS FOR AWARD FEE CONTRACTS SHUTTLE ORBITER CONTRACT 1972 Excellent Performance ranges from substantially better than average to outstanding. In this range, the contractor has improved all facets of his operation beyond that level described in the range below. The contractor has greatly exceeded the average performance standard which would be expected of a qualified Aerospace contractor. Areas of deficiency are few and, overall, are considered relatively unimportant. Good The contractor in this good range has exceeded average performance expected of a qualified Aerospace contractor. Areas of less-than-good performance are few and are more than offset by areas of above average or excellent performance. The. degree to which the contractor has exceeded the average performance standard which would be expected of qualified Aerospace contractor will determine the contractor's grade in this range. Satisfactory The top of the satisfactory range represents performance which is slightly above average performance expected of anyqualified aerospace firm. The midpoint of the satisfactory range represents average performance expected of any qualified aerospace firm and is the primary point of reference for this rating system. 148 Numerical Grade Range 90 + to 100 80 +to 90 75 +to 80 75

PAGE 164

Marginal Un _satisfactory Figure 3-1 Continued In this range, the contractor is below, but approaching average performance. The degree to which the contractor has slipped below average performance determines the grade in this range. Performance is significantly below the average performance expected of a qualified aerospace contractor. The contractor at this point is close to performance which would be considered unsatisfactory, and deficiencies are such that immediate steps must be taken to correct the situation. Performance is deficient in substantial areas of effort, and is sufficiently far below average performance as to be overall unsatisfactory. Immediate improvement is required in order to permit continuation of the contract. SOURCE: Office of Shuttle Procurement Johnson Space Center National Aeronautics and Space Administration August 1972 149 70 + to 75 60 + to 70 60 and below

PAGE 165

150 intended for average performance expected of a qualified aerospace firm. To give an idea of the depth and scope of the evaluation, each AFEB report was approximately 75 pages in length. In performing the evaluation, the contractor was assigned a specific adjective and numerical score for both technical performance and for cost performance, since the contract contained specific fee amounts available for cost performance as well as for technical performance, Actual award fee earned by the contractor was determined by converting the performance score to a percent of available award fee. Figure 3-2 graphically depicts the award fee rating curve. Each award fee rating point is equivalent to 2 and 1/2 percent of the available fee, resulting in a straight line from 60 to 100, representing the range of values whereby fee is earned. For example, an award fee rating of 80 yields 50 percent of the available fee, while a rating of 90 represents 75 percent of the available fee. Should the actual score not come out to be a whole number, interpolation is made to determine the precise percentage of available award fee. As indicated in the first chapter, some determination of the perceptions of government personnel involved in award fee evaluations is critical to evaluating its effectiveness. In this study a survey was conducted to obtain those perceptions, the results of which are covered in the ensuing section of this chapter. Analysis of Award Fee Survey Data In order to obtain the perceptions regarding the

PAGE 166

Figure 3-2 AWARD FEE CURVE 100 eo eo u 70 ... ... ... ... II eo c ... c > c ... 10 0 ... c c z ... u a: ... 111. 30 J I II J 20 II 10 J I II 0 10 20 40 AWARD FIE lllATINQ ONE AWARD FEI! RAnNO 2 1/2 PERCENT 01' AWARD I'I!E SOURCE: Office of Procurement Johnson Space Center 70 j II I v I 1/ I II j I I ao eo 100 National Aeronautics and Space Administration August 1972 151

PAGE 167

152 effectiveness of the use of cost-plus-award-fee (CPAF) type of contract, a questionnaire was developed and sent to government personnel occupying key positions in the award fee evaluation process. These personnel included: project management, procurement management, budget managers, and legal advisors. A total of 32 responses was received out of 36 solicited, or a response rate of approximately 90 percent. A copy of the survey instrument is included as Appendix A to this study. The survey instrument was designed to obtain perceptions from the gover?IDent personnel in regard to: the contribution of the award fee contract to the technical success of the Orbiter project; how the award fee arrangement contributed to minimizing cost growth; and, relations and decision-making between the government and the contractor. Appendix B to this report 'contains the statistical analysis of survey questions 1-11, all of which deal with award fee contracting. Figure 3-3 shows the statistical means and standard deviations for questions 1 through 11. The award fee type of contract was considered to have had a major contribution to the technical success of the Orbiter project, with 84 percent of the survey respondents indicating that the award fee had a positive effect. Significantly, none of the respondents considered the award fee contract to have a negative effect on technical outcome. Seventy-five percent of the respondents believed that the award fee type of contract contributed to minimizing cost growth of the project. This indicated a strong belief that the award fee

PAGE 168

Question Number Part A 1 2 3 4 5 6 7 8 I'nrt B 9 10 11 N c 32 Figure 3-3 MEAN AND STANDARD DEVIATION FOR SURVEY QUESTIONS 1 THROUGH 11 QUESTIONNAIRE FOR GOVERNMENT PERSONNEL AWARD FEE CONTRACTING, 1984 The Degree to Which Award Fee Contract Mean Standard Deviation Contributed to technical success 4.094 .641 Contributed to minimh:ing cost growth 3.969 .861 Affectt"d relations 3.531 1.077 Motivated contractor to be responsive 4.219 .975 Affected contractors nttentlon 4.531 .718 Represented key of concern 4.375 .660 f.vnluotlon frequenc:-y 4.094 .856 Affected decision-making 3.687 1.030 Evaluation should have been more objective 4.375 2.948 Process could have been improved 5.188 2.507 Fee Determination Official informed 7.437 2.539 NOTES: 1. Scale for questions 1 through 8 was 1 to 5. 2. Scale for questions 9 through 11 was 0-10. .... ln w

PAGE 169

154 contract served to reduce project cost. Of particular significance was that 88 percent of project managers supported that perception while all of.the budget managers believed that the award fee type of contract played an instrumental role in reducing project costs. Cost growth of the Orbiter project will be dealt with in a subsequent chapter. Relationships with the contractor did not appear to be affected by the award fee type of contract. Fifty percent of the respondents believed that award fee arrangement enhanced the relationship, while 41 percent considered this type of contract to have a neutral effect on relationships with the contractor, and the remainder believed the award fee type of contract had a negative effect on relationships. To place the contractor relationships factor in proper perspective it could be said that 91 percent of the respondents believed that the award fee type of contract was as effective or more effective than other types of contracts. The survey results strongly support .the finding in the following paragraph regarding a high degree of responsiveness by the contractor under the award fee contract. One of the most revealing results was that 91 percent of the respondents found the contractor responsive under the award fee arrangement. There was similar support for this contention in all job categories surveyed. Since contractor responsiveness is a vital prerequisite in major R&D projects, this high degree of

PAGE 170

155 responsiveness by the contractor is no doubt directly related to the fact that contractor responsiveness is interwoven throughout the evaluation process, therefore, significantly influencing the award fee earned. Management attention by the contractor to major issues and problems was also found to be strongly fostered by the award fee arrangement. Eighty-eight percent of the respondents believed that the award fee contract contributed to the contractor focusing on major problems of the Orbiter project. Project managers were unanimous in this contention as were all procurement managers except one. However, only 60 percent of the budget managers believed that the award fee contract contributed toward,increased contractor attention to problem areas. The mean response was 4.5, making this question the highest positive score of the first part of the survey. A unique feature of the award fee arrangement was that it required NASA to identify key areas of concern of the project and those areas where emphasis should be placed by the contractor for the ensuing evaluation period. Survey respondents overwhelming believed (97%) that the government had provided the contractor with the key areas of concern of NASA. On an agreement response scale with five representing the top score (strongly positive), the mean response was 4.4. This high positive response rate was observed across all the categories of managers surveyed. As stated earlier in this study the award fee evaluation

PAGE 171

156 determination frequency was each six months. Informal feedback on performance was provided the contractor on a quarterly basis. Among survey respondents, 75 percent believed that the above evaluation frequency was appropriate with a mean response of 4.1. Narrative comments from respondents indicated that evaluations more frequent than semi-annually would be too incomplete and counter-productive due to the time required to perform the evaluation and prepare the award fee report. Efficient and effective decision-making between the government and the contractor is always essential in R&D undertakings. A somewhat mixed response was received regarding the effects of the award fee type of contract on decision-making between the government and the contractor. While, overall, 56 percent of all respondents believed that generally the award type of contract had a positive effect on decision-making, 63 percent of the project management personnel did not believe the specific award fee arrangement enhanced decision-making of the project. Further, one third of the procurement managers did not believe the award fee type of contract enhanced decision-making any more than any other type of contract. Budget management, however believed that the award fee arrangement had a positive effect on decision-making with 80 percent of those respondents expressing that opinion. An analysis of the survey data indicated that the objectivity of the evaluation criteria was adequate. Respondents' mean score was 4.4 although the scale used here ranged from 0 to 10. This

PAGE 172

157 indicates that the award fee evaluation criteria was at an appropriate level of objectivity for .this type of contract. A review of the survey data from a job category standpoint, showed little consistency from project managers, with scores ranging from 0 to 8. This wide range of scoring was also the case for the procurement managers with scores ranging from 1 to 8. Budget managers generally believed that the evaluation criteria could have been more objective with scores in the 7 to 10 While overall, the data suggests the level of objectivity was adequate, the high standard deviation shows considerable variations in the response pattern. All respondents indicated improvements could be made in the award fee process with scores ranging from 2 to 10, again, on a scale of 1 to 10. In fact, 60 percent of the respondents indicated a score of 5 or higher, suggesting there were significant improvements that could be made. A subsequent section of this chapter will deal with survey participants perceived strengths and weaknesses of the Orbiter award fee arrangement. A majority of the respondents believed that the fee determination official (FDO) was fully informed regarding the award fee evaluations, with 87 percent of the survey participants scoring 5 or higher on a scale of 1 to 10. There is strong evidence from the perspective of all job categories that the FDO was fully informed, or informed to the degree necessary, in making the periodic award fee determination.

PAGE 173

158 Strengths and Weaknesses of Award Fee As part of the survey completed by government personnel occupying key managerial roles during the Shuttle Orbiter development, respondents were asked to state what they perceived to be the major strengths and weaknesses of the award fee type of contract for major research and development (R&D) acquisitions. These survey results are reviewed in the. ensuing paragraphs below. Strengths of Award Fee Survey participants were asked the following general question: "What do you consider to be the major strengths of award fee type of contract for major R&D acquisitions?" Table 3-4 contains a listing of the most frequent strength mentioned with the number of individuals responding with that comment. A complete summary analysis of the strengths of award fee contracting, as perceived by survey participants, can be found in Figure 3-5. Appendix C contains a computer analysis of the strengths of the award fee type of contract by job category of the respondents. As can be noted from Figure 3-4, 18 individuals, or 58 percent of those participating in the survey, indicated that the award fee type of contract stimulated frequent communication. Frequent communication was encouraged by the award fee type of contract due to flexibility afforded by the contractual relationship between the contractor and the government. Although a formal, binding contract exists, the nature of

PAGE 174

Figure 3-4 STRENGTHS OF AWARD FEE TYPE OF CONTRACT QUESTIONNAIRE FOR GOVERNMENT PERSONNEL AWARD FEE CONTRACTING, 1984 Strength Number Responding 1. Stimulates frequent communication. 2. Creates cooperation and relationship. 3. Timely and periodic feedback to contractor in regard to performance. 4. Motivator to contractor senior management; attention getter. 5. Subjectivity aliows for minimum penalty for near term cost growth if technically successfully. 6. Flexibility to accept changes in program without major contract pertubation; contractual provisions simplified. 7. Forces government management involvement frequently; award fee is good management tool. 8. Award fee was proper for the Orbiter DDT&E contract resulting in better performance and visibility by the government. N=31 18 8 6 12 3 10 7 10 159 Percent Responding 58.1% 25.8 19.4 38.7 9.7 32.3 22.6 32.3

PAGE 175

Figure 3-5 SUMMARY ANALYSIS OF STRENGTHS OF AWARD FEE CONTRACTING QUESTIONNAIRE FOR GOVERNMENT PERSONNEL AWARD FEE CONTRACTING_ 1984 CMT 1 CHT 2 CHT 3 CMT 5 CHT 6 f!!U Project a, 4 1 1 4 0 3 2 Management b. 50 12.5 12.5 50.0 0 37.5 25.0 c, 22 2.5 16.7 33.3 0 30.0 28.6 Procurement a. 7 4 3 3 2 4 2 Management b. 58.3 33.3 25.0 25.0 16.7 33.3 16.7 c. 38.9 50.0 50.0 25.0 66.7 40.0 28.6 Budget a. 3 2 2 1 1 2 2 Analyst b. 60.0 40,0 40.0 20.0 20.0 40.0 40.0 c. 16.7 25.0 33.3 8.3 33.3 20.0 28.6 Other a, 4 1 0 4 0 1 1 b. 66.7 16.7 0 66.7 0 16.7 16.7 c. 22.2 12.5 0 33.3 0 10.0 14.3 Total a. 18 8 6 12 3 10 7 d. 58.1 25.8 19.4 38.7 9.7 32.3 22.6 CHT: Comment NOTE: Comment numbers correspond to numbers in Appendix C. a. Number responding CHT 8 2 25.0 20.0 4 33.3 40.0 1 20.0 10.0 3 50.1 30.0 10 32.3 b. Percent responding in job category SOURCE: Computer Analysis of c. Percent of total column d. Percent of total survey participants Survey Data. See Appendix C .....
PAGE 176

Comment 1 2 3 4 5 6 7 8 161 Figure 3-5 Continued SUMMARY ANALYSIS OF STRENGTHS OF AWARD FEE CONTRACTING QUESTIONNAIRE FOR GOVERNMENT PERSONNEL No. AWARD FEE CONTRACTING 1984 Strength Stimulates frequent communication. Creates cooperation and team relationship. Timely and feedback to contractor in regard to performance. Motivator to contractor senior management; attention getter. Subjectivity for minimum penalty for near term cost growth if technically successfully. Flexibility to accept changes in program without major contract pertubation; contractual provisions simplified. Forces government management involvement frequently; award fee is good management tool; assures active role of Government managers. Award fee was proper for Orbiter DDT&E contract. Better performance resulted and better visibility.

PAGE 177

162 contract, i.e., cost-reimbursement and the earned award fee is dependent to a large degree of satisfying the needs and desires of the government managers, lends itself to an informal relationship. Frequent communication is no doubt thought by the majority of survey participants to be critical to the success of major R&D acquisitions. A review of the data in the computer analysis indicates there was a high degree of agreement among all job categories that frequent communication was stimulated by the award fee type of contract. The second most frequent mentioned strength was that the award fee type of contract was a motivator to the senior management of the contractor. Several of the respondents labeled this strength as an "attention getter." Twelve individuals responding with this strength which represented 39 percent of the respondents. Award fee evaluations were prepared in formal reports and a copy given to the contractor at the conclusion of each evaluation period and prior to actual fee determination. Corporate management of the contractor typically view award fee reports and the attendant fee earnings as a "report card" on the corporation and the firm's management.. Thus, the award fee procedure becomes a powerful motivator or an "attention getter." A review of the computer analysis indicated that project management personnel were much stronger on this strength than procurement management or the budget analysts. One half of the project management personnel responded with this strength while 25 percent that responded similarly in the other two categories. This

PAGE 178

163 high response by project management personnel was not unexpected since the project office has the responsibility for preparation of the award fee report and for reviewing the report with contractor management personnel. Ten of the survey respondents (over 32 percent), stated that the award fee type of contract was the most appropriate type of contract for the Orbiter DDT&E effort. Further, these individuals indicated that the award fee. arrangement resulted in better performance and better visibility. Adequate visibility is very critical during major R&D undertakings due to the long duration, i.e., several years, required for development of most major R&D systems. A perception of better performance no doubt ties in with the informal method of communications. Thus, current status of the project, problems, etc., is fostered and encouraged by the award fee type of contracting. Another major strength of the award fee type of contract, as viewed by the survey participants, was the flexibility to make changes in the project. While the government can order changes to any type of contract, the respondents are certainly referring to the contractor's willingness to accommodate government managers' suggestions without insisting on formal contract changes. Formal contract changes are, of course, required if a contract requirement is changed or modified. Contract statements of work and specifications are often not as specific in a cost-reimbursement contracts as is generally the case in fixed-price contracts. Cost reimbursement

PAGE 179

164 contracts typically contain performance requirements and performance goals as opposed to detailed specifications; hence, the ease to accommodate suggestions from goyernment technical managers suggesting different approaches to design, testing, and so on. Although the cost-plus-award-fee type of contract is not any more flexible than a cost-plus-fixed-fee type of contract, there is little question that the contractor would be more responsive and receptive under an award fee arrangement since fee are determined by the government. A review of the computer analysis showed all job categories of survey respondents being fairly consistent in terms of the percent of each category indicating this strength. Another strength frequently mentioned by survey respondents was that the award fee type of contract creates cooperation and team relationships. Twenty-six percent of the participants listed this as a major strength of award fee contracting. A surprising finding was that the procurement and budget managers responded higher on this strength than project managers. 'This was unexpected because procure-ment and budget managers are more apt to view the contractor relation-ship as adversal, as opposed to cooperative. However, this strength should be viewed in context with other strengths mentioned, e.g., stimulation of frequent communication. Those responding with the latter strength may have viewed frequent communication as evidence of cooperation with the contractor. : Numerous survey participants stated that the award fee type

PAGE 180

165 of contract forced government management involvement more frequently than, for example, cost-plus-fixed-fee contracts. This is through the periodic performance evaluations necessitating a written report assessing the contractor's performance against the award fee evaluation criteria. An inference to this strength is that the gove!nment management would be much less involved if the periodic award fee evaluations were not required. Timely and periodic feedback to the contractor regarding the contractor's performance was another strength of the award fee type of contract, as perceived by the survey participants. In this case, arrangements forced the government to identify to the contractor any weaknesses perceived by the government as well as the strengths. Unlike the cost-plus-fixed-fee contract, the award fee contract requires frequent feedback to the contractor regarding performance. It should be pointed out that this is a highly desirable feature under any type of cost-type contract from both, cost and technical standpoints. A final strength mentioned was that the subjectivity in determining the award fee earned allows for a minor or minimum fee penalty for near term cost growth if the additional cost incurred resulted increased technical performance. While the penalties for any type of cost growth, near term or long term, would obviously depend upon the evaluation criteria, the point is well made that fee penalties can be minimized for near-term cost growth.

PAGE 181

166 Weaknesses of Award Fee Along with the of the strengths of award fee contracting, survey participants were asked "What do you consider to be the major weaknesses of award fee type of contract for major R&D acquisitions." Respondents comments are summarized in Figure 3-6. A complete summary analysis of the weaknesses of award fee contracting, as perceived by survey participants, can be found in Figure 3-7. Appendix D contains a computer analysis of the weaknesses of the award fee type of contract by job category of the respondents. A large number of participants responded that a major weakness in award fee contracting is that the government evaluators are biased. In other words, evaluators perceive that they are grading themselves. This perception is somewhat justified since the evaluators are generally responsible for technically managing that area of the contract work that they are evaluating. Thus, if major deficiencies exist in the contractor's performance in areas where the government evaluator is responsible for managing, evaluators as would be expected to tend to be highly, defensive of their areas of responsibility. This points to the fact that an ideal evaluation should be conducted by unbiased personnel, i.e., by people who do not have day-to-day management responsibility for the areas under evaluation. From reviewing the computer analysis in Appendix D, the respondents indicating this area as a weakness were numerous in all job categories. For example, all job categories had a response in excess of

PAGE 182

Figure 3-6 WEAKNESS OF AWARD FEE CONTRACTING QUESTIONNAIRE FOR GOVERNMENT PERSONNEL AWARD FEE CONTRACTING, 1984 167 Weakness Number Responding Percent Responding 1. Individual evaluators are biased; they perceive they are grading themselves. 2. Time consuming and expensive administrative process. 3. Evaluations are too subjective. 4. Hard to keep award fee emphasis in balance with project; government does not shift emphasis project criteria enough. 5. Too much filtering from evaluators to award fee evaluator board and fee determination official. 6. Award fee evaluations are low priority to evaluators; members often replaced with alternates. 7. Contractor motivated to satisfy government managers' desires and not necessarily the best way. 8. Award fee not large enough to overcome other corporate goals, e.g.' maintain work force etc. N=31 17 54.8% 8 25.8 7 22.6 8 25.8 7 22.6 4 12.9 4 12.9 3 9.7

PAGE 183

Figure 3-7 SUMMARY ANALYSIS OF WEAKNESSES OF AWARD FEE CONTRACTING QUESTIONNAIRE FOR GOVERNMENT PERSONNEL AWARD FEE CONTRACTING CMT II Project a, 4 Management b. 50.0 c. 23.5 Procurement a. 7 b. 58.3 c. 4 (.. 2 Budget o. 2 Analy!lt h. 40.0 c. 11.8 a. 4 Other b. 66.7 c. 23.5 Total a. 17 d. 54.8 CMT: Comment CMT 12 I 12.5 12.5 3 25.0 37.5 2 40.0 25.0 2 33.3 25.0 8 25.8 1984 CMT 13 2 25.0 28.6 2 16.7 28.6 2 40.0 28.6 I 16.7 14.3 7 22.6 CMT 14 0 0 0 4 33.3 50.0 3 60.0 37.5 I 16.7 12.5 8 25.8 NOTE: Comment numbers correspond to numbers in Appendix D. CMT 15 1 12.5 14.3 2 16. 7 28.6 3 60.0 42.9 I 16.7 14.3 7 22.6 CMT 16 0 0 0 2 16.7 50.0 0 0 0 2 33.3 50.0 4 12.9 n. Number responding CMT 17 1 12.5 25.0 1 8.3 25.0 l 20.0 25.0 1 16.7 25.0 4 12.9 CMT 18 1 12.5 33.3 0 0 0 1 20.0 33.3 1 16.7 33.3 3 9.7 b. Percent responding in job category SOURCE: Computer Analysis of Survey Data. See Appendix D c. Percent of total column d. Percent of total survey participants 0\ Q)

PAGE 184

Figure 3-7 Continued SUMMARY ANALYSIS OF WEAKNESSES OF AWARD FEE CONTRACTING QUESTIONNAIRE FOR GOVERNMENT PERSONNEL AWARD FEE CONTRACTING 1984 Comment No. Weakness 169 11 Individual evaluators are biased; they perceive they are grading themselves. 12 Time consuming and expensive administrative process. 13 Evaluations are too subjective. 14 Hard to keep award fee emphasis in balance with project; government does not shift emphasis project criteria enough. 15 Too much filtering from evaluators to award fee evaluation board and fee determination official. 16 Award fee evaluations are low priority to evaluators; members often replaced with alternates. 17 Contractor motivated to satisfy government managers' desires and not necessarily the best way. 18 Award fee not large enough to overcome other corporate goals, e.g., maintain work force etc. NOTE: Comments are labeled 11 through 18 in the computer analysis in order to distinguish from computer analysis of strengths of award fee.

PAGE 185

170 40 percent, with 58 percent of the procurement managers listing the "grading yourself" concept as a major weakness. Fifty percent of the project management respondents weakness. Another major weakness g-iven was that the award fee method of contracting is a time consuming and expensive administrative process. This comment is referring to the time required to prepare the periodic evaluation reports. This is, of course, judgmental as to the worth of time spent preparing evaluation reports and briefing top government management and contra.ctor management. Obviously, the government believes that the process is, overall, cost effective. Comments such as "the award fee process is expensive and time consuming" leads one to believe that periodic evaluations would not be done, even though informal, in most cases for cost-reimbursement contracts. In any type of R&D contract, performance assessments should be made periodically and briefings made to government management and contractor management. It has been observed by some top government management personnel that award fee contracts ensure a periodic briefing on contract performance. Thus, the award fee process serves as a management information system. An interesting weakness cited by 25 percent of the survey participants was that the award fee evaluations are too subjective. Subjectivity, of course, is inhetent in the nature of award fee contracting. It is significant to note the subjectivity was listed as a major strength of award fee contracting. A review of the computer

PAGE 186

171 analysis, reveals that the budget analysts saw this as a weakness to a greater extent than project management and procurement management personnel. This analysis suggested that while the award fee type of contract fosters subjectivity in the evaluation process, there is concern that the evaluation criteria should be more objective. Having criteria that are more objective would make the periodic evaluations a little more straight-forward and less controversial for the evaluator to defend performance assessments. Another weakness cited by the survey participants was that it is hard to keep the award fee emphasis in with the program. Along this same line, others mentioned that the government does not shift the areas of emphasis under the award fee criteria enough. Areas of emphasis typically change or can be changed by the government at the beginning of each evaluation period. Evaluation criteria are usually fairly broad and generally do not need to be changed during the contract period of performance. Thus, areas of emphasis are essentially sub-criteria within the framework of the broad criteria. It is interesting to note, again reviewing the computer analysis, that this weakness was mentioned primarily by the procurement and budget managers and not by project management personnel. One interpretation is that this could be only a perceived weakness to some degree rather than a real weakness. It was not a surprise to this writer that project managers did not believe that to

PAGE 187

172 be a weakness, for the project manager is in a position to change the areas of emphasis if deemed appropriate. Another award fee weakness mentioned by seven survey respondents, was that there was too mqch screening from the government evaluators to the fee evaluation board (AFEB) and fee determination official (FDO). One major purpose of the fee determination process is to have checks and balances in the process to assure that only pertinent facts are considered during the fee determination and to filter out personal biases and frustrations not really germane to the award fee criteria. However, the point is clearly made by over 20 percent of those participating in the survey that there is too much unjustified filtering to the award fee evaluation board and fee determination official. A problem that often exists on many contracts is that the award fee evaluators are often replaced with their alternates. Schedule conflicts and other higher priority matters were typically given for not being able to part;cipate in the evaluation process. Those evaluators who have responsibility to prepare the evaluation reports often delegate that responsibility to subordinate personnel. Principal evaluators then often have their alternates participate in award fee committee and board activities. Cited also as a weakness was that the contractor, under the award fee type of contract, is motivated to satisfy the government manager's desires and not necessarily what the contractor may think

PAGE 188

173 is the best approach to, for example, problem areas. This is a very difficult and subjective area to deal with. First of all, it is the responsibility of the government managers to manage and monitorthe contract. Ideally, the best approach would be the same for the contractor as well as the government. However, due to the subjectivity of the award fee process, the contractor cannot ignore the desires of those individuals who are responsible for assessing the contractors performance. A delicate balancing is required by the contractor in the event of a conflict between what the contractor perceives as the best way and what government managers see as the best way. Usually, any issue in' this area can be resolved satisfactorily to both parties. A last weakness mentioned in award fee contracting was that the award fee dollars available were not usually large enough to overcome other corporate goals such as maintaining the size of the work force rather than trying to maximize profit. In all fairness, it should be pointed out that this weakness is not only a reflection of the award fee type of contract but the same weakness could be given for any type of cost-reimbursement contract, especially cost-plus-fixed-fee contracts. To a lesser degree, cost-plusincentive-fee contracts would fall within the same situation. There is little question in this writer's mind that as long as the government has resources available, contractors will be eager to maintain staffing at a level commensurating with those resources so long as

PAGE 189

174 the contractor's cost of operations is covered and a reasonable profit is made. This contention, i.e., profit maximization is subordinate to the corporate of staff retention, is supported in the literature in the context described above. Survey Respondents Recommended Contract Type The survey instrument solicited respondents' recommendations regarding the selection of an appropriate type of contract for three types of acquisitions: the deve'lopment phase of major R&D projects; limited production (3 to 10 units) of major systems; and, operational support of R&D systems. These responses are summarized in Figure 3-8. Appendix E contains a statistical analysis of the respondent recommendations by contract type. This Appendix contains the recommended contract type for each of the three types of acquisition by the survey group in total and also by the job categories of the group members. As can be noted in Figure 3-8, respondents generally preferred the cost-plus-award-fee (CPAF), completion form type of contract for the development of major R&D undertakings, with 21 of the 32 responses indicating that preference. Thus, approximately two-thirds of the survey participants believe that the CPAF type of contract was the most appropriate type for the Orbiter development activities. All of the project management personnel expressed the CPAF completion form contract as the preferred type. Procurement and budget management personnel had preferences of 50

PAGE 190

Figure 3-8 RECOMMENDED CONTRACT TYPES BY SURVEY RESPONDENTS FOR VARIOUS ACQUISITIONS QUESTIONNAIRE FOR GOVERNMENT PERSONNEL 1984 Responses Limited Major Production Contract Type R&D Projects (3-10 Units) Cost-Plus-Fixed-Fee 2 0 (Completion Form) Cost-Plus-Fixed-Fee 2 0 (Level-of-Effort) Cost-Plus-Award-Fee 21 10 (Completion Form) Cost-Plus-Award-Fee 3 2 (Level-of-Effort) Cost-Plus-Incentive-Fee 3 8 (Completion Form) Fixed Price Incentive 0 8 Firm Fixed Price 1 4 Total 32 32 175 Operational Support Of R&D Systems 1 4 3 18 4 1 1 32

PAGE 191

176 percent and 60 percent, respectively. Degrees of preference were very high, with 65 percent rating the degree of preference a 7 or higher on a scale of 1 to 10. It is concluded that the survey group had strong beliefs regarding their recommendations. In regard to limited production of major systems, the most frequently recommended type of contract was again, the CPAF completion form with approximately recommending this type of contract. However, 25 percent of the respondents recommended the cost-plus-incentive-fee type of contract and 25 percent recommended the fixed-price-incentive contract form. This data suggest that 50 percent of the survey participants would like to see more emphasis on cost control in a production environment. Although this study deals only with the development phase of a major R&D project, the above discussion of recommended contract types for the three identified acquisitions situations provides.the reader with a better perception of selection of contract type. Summary In conducting major research and development activities, it is necessary that the contractural arrangement provide for the flexibility necessary to permit government to have informal, as well as formal, involvement in directing the contract work. This study, using a special survey, has analyzed how the award fee type of contract fulfills these needs. In essence, award fee contracting

PAGE 192

177 provides a great degree of subjectivity in determining the amount of fee earned by a contractor. Based on the review of the nature of the Shuttle Orbiter design, development, test, and evaluation requirements, the CPAF type of contract was considered to have been the best type of contract for that project. This type of contract was also supported by the survey conducted by the researcher in conducting this study. Award fee contracting was found to provide a high degree of motivation for the contractor to strive for attainment of the goals of the government. Since the award fee is a subjective fee determination, the contractual arrangement for the Orbiter project provided adequate flexibility in making the subjective assessment of the degree of the attainment of the goals and objectives of the contract. Findings from this study are in-line with the results of Professor Hunt's award fee study for the U.S. Air Force. As reported in the literature review chapter, Hunt found that the award fee type of contract encouraged government-contractor relations, stimulated formal and informal communications, and provided for the necessary flexibility generally needed by the government and the contractor in accomplishing contract goals.

PAGE 193

NOTES -CHAPTER III 1The award fee criteria shown were taken from the Orbiter DDT&E contract, NAS9-14000, Schedule A, Appendix J.

PAGE 194

CHAPTER IV COST GROWTH OF THE ORBITER PROJECT Introduction Development of the Space Shuttle system was an outgrowth of the political pressures to develop a low cost space transportation system. As the National Aeronautics and Space Administration (NASA) attempted to formulate strategies for a post-Apollo Space Program, it found itself without the clear mandate that it enjoyed with the lunar landing mission. Starting with ambitious plans for an extensive program of manned spaceflight, which included a Space Station Program followed closely by a Space Shuttle Program, NASA was forced through a number of regressive moves that left the Shuttle as the major focus of its efforts and dropped the Space Station endeavor. As budgetary pressures from the administration intensified, NASA had to proceed with a cost-effective space transportation system. Even with focusing attention on the Shuttle, pressures.on NASA to reduce the post-Apollo budget continued throughout the 1970's. While this study deals only with the design, development, test and evaluation (DDT&E) phase of the Orbiter project, the total initial estimates of the entire Space Shuttle system will also be elaborated on in the succeeding paragraphs to place the Orbiter project in perspective as a total program. NASA was committed to the Congress and the Office of

PAGE 195

180 Management and Budget (OMB), to estimates for a total Shuttle DDT&E Program, including the Orbiter project, that were pledged to not exceed $5.150 billion in constant 1971 dollars.1 The use of constant 1971 dollars assumes that the value of the dollar remains constant; hence, no inflation or deflation. Figure 4-1 shows the initial cost estimates for the various Shuttle Program elements, which are referred to as projects. Although this research is limited only to the Orbiter DDT&E project, (the program estimates and actual outcomes for the remaining program elements will not be dealt with in this paper)., one can readily see that the Orbiter was almost 70 percent of the total Shuttle budget. As shown in Figure 4-1, the initial Orbiter DDT&E estimate was $3,493 million. This number was revised to $3,438 million to reflect program content adjustments. This revised amount was deve-loped essentially by deleting the air breathing jet engines which would have been used to ferry the Orbiter from landing site to launch site. Therefore, the $3,438 million estimate will be referred to as the NASA commitment for the Orbiter element of the Shuttle 2 Program. Project Cost Compared with Commitments to Congress and OMB While there was cost growth experienced for the Orbiter DDT&E effort, actual outcome was less than the commitment to OMB in 1971 value dollars and also less in real year dollars. Based on data compiled by the Shuttle Program Office at the Johnson Space Center,

PAGE 196

Figure 4-1 INITIAL SPACE SHUTTLE PROGRAM ESTIMATES DESIGN, DEVELOPMENT, TEST, AND EVALUATION PHASE IN 1971 CONSTANT-VALUE DOLLARS (MILLIONS OF DOLLARS) 181 Program Element Estimate Percent of Total Orbiter $3,493 Program Support .470 Space Shuttle Main Engine 580 Solid Rocket Booster 350 External Tank 257 Total $5,150 SOURCE: Space Shuttle Program Office Johnson Space Center National Aeronautics and Space Administration March 1980 67.8% 9.1 11.3 6.8 5.0 100.0

PAGE 197

182 the actual Orbiter DDT&E cost, including Orbiter support contracts and fee, amounted to $2,786 million in 1971 dollars compared with the $3,438 million commitment to OMB. In terms of real year dollars, the actual project cost was $4,942 compared with the OMB commitment of $5,142. As indicated in the preceding paragraph, the actual project outcome was less than the commitment to the Congress and OMB. This underrun was achieved while at the same time encountering technical difficulties, e.g., thermal protection system tile problems, and experiencing several years of double digit inflation rates. Further, the underrun was experienced after absorbing the impact of governmentdirected contract changes and funding impacts. In arriving at the $2,786 million amount, the actual project outcome ($4,942 million) was de-escalated using actual inflation rates. The initial commit ment of $3,438 million had a real year value of $5,142 million.4 A final cost outcome lower thaninitial estimate for a major R&D project of the magnitude of the Orbiter project, covering a 10-year period (1972-1982), was by NASA management to be a remarkable achievement especially in light of the budget problems and thermal protection system problems. The following sections of this chapter describe the cost growth,for the Orbiter spacecraft project, the subject matter of this research. While all Orbiter project contracts amounted to $4,942 million the Orbiter spacecraft project's actual cost was $4,680 million (including fee) or 95 percent of the total effort.

PAGE 198

183 Cost Growth Attributable to the Government Cost growth on the Orbiter amounting to 40 percent of total project cost, is defined as the amount of actual cost over the original negotiated cost. In this context, cost growth includes government-directed contract changes, funding impacts, schedule stretch-outs, and overrun. Overrun is defined as the contractor's inability to perform the work required by the contract within the negotiated amount. With the award of the basic DDT&E contract, the first orbital flight was scheduled for March 1979. Due to schedule slips mainly associated with the thermal protection system and program budget cuts, the first flight occurred in April 1981. Cost growth on the Orbiter project caused by the government consists of two categories: contract changes; and fiscal year funding constraints which necessitated slipping delivery schedules of the Orbiter spacecraft and associated equipment. government-directed contract changes amounted to approximately $581 million for the DDT&E phase of the project. Figure 4-2 depicts the basic contract value, the approximate value of contract changes and funding impacts, and the contract overrun on the Orbiter project. Contract Changes Government-directed contract changes, amounting to 13 percent of the total cost growth, consisted mainly of product improvements, increases to hardware testing, additional development verification

PAGE 199

Figure 4-2 ORBITER CONTRACT COST GROWTH 1984 Basic Contract Cost Growth Government Directed Changes Impact of Government Directed Funding Overrun TOTAL (Cost Without Fee) SOURCE: Contract Records Office of Shuttle Procurement Johnson Space Center Millions of Dollars $2,630 581 450 759 $4,420 National Aeronautics and Space Administration March 1984 184 Percent 59.5% 13.1 10.2 17.2 -100.0

PAGE 200

185 requirements, and spare parts. Chapter 5 of this report reviews the contract changes in depth. By and large, contract changes did not contribute to the schedule slippage. Schedule slips were due mainly to fiscal year funding shortfalls and the thermal protection system problem associated with preparation for the first flight of the Shuttle Orbiter. Contract changes for the Orbiter DDT&E phase were estimated at $581 million which represents slightly more than 13 percent of the total DDT&E estimate at completion. An interesting observation is that the excess cost to the government due to in-adequate funding had almost as much an impact on the project cost as the contract changes. Funding Impacts Commitments made by NASA to the Congress and OMB were based on the assumption that funding would be made available in reasonable amounts and in a timely manner as the program progressed. Expecta-tions regarding funding, however, were far different from the reality which ensued. Severe funding constraints were imposed at the outset of the program, representing a way of life for the Shuttle project. For example, the contractor's proposal for the first year of the project was based on $140 million funding, but the government made available only $70 million. Other significant variations between the funding required and the amount provided occurred throughout the project. As a result of the very tight funding environment, the cost profile of the Orbiter that had been negotiated never

PAGE 201

materialized. Constant re-phasing of project work content and schedules became a way of life for the project. 186 Cost control in those circumstances was frequently reduced to hard choices between deferring and deleting work that had been planned and considered necessary to the conduct of an orderly program containing a reasonable balance between resources and risk. With reductions in resources, risk increased, while at the same time the government and the contractor maintained success-oriented schedules and project plans. However, the effort of the funding shortages began to emerge when it was too late for the project to react without extremely large cost impacts. Clearly, the most significant example of those consequences was the cost growth of the thermal protection system. Although several causes contributed to the cost growth, there is no question that it was due in large part to the unavailability of funds in earlier years to perform required analyses and development testing. While the cost growth in the thermal protection system alone was large, the cost growth to the overall project schedule which resulted from the TPS problems was even greater. Cost growth stemming from the thermal protection system is but one example of failure impacts. Still neither this factor, nor a complete catalogue of all the other singular effects of funding constraints, can accurately portray the cumulative effect that funding shortfalls had on the program. They effectively changed it from one that was a planned expenditure of resources for

PAGE 202

187 reasonably scheduled milestones to one that was extremely uneven in terms of the timing and availability of resources necessary to do the job. This is clearly exemplified by the fact that more than 100 cost/schedule tradeoff options were performed by the contractor just during 1978-1979 to assess the effect of reduced funding.5 Accord-ingly, the funding shortfall contributed to an estimated $450 million increase to program cost as can be noted from Figure 4-2. Comparison of Orbiter Cost Growth to Cost Performance on Major R&D Programs An additional means that is relevant in assessing the control of program costs is a comparison of the 40 percent cost growth experienced on the Orbiter project with cost performance that has been experienced on other programs. In fact, this is a principal objective of the research. While comparisons of this kind can be imprecise because of variations in program complexity and the diffe-rent techniques that are used to account for cost growth, they nevertheless provide insight into the quality of a contractor's cost performance when reasonable assumptions are made regarding possible differences in the programs involved and the methods used to track and account for cost growth. Set forth in Figure 4-3 are data depicting the total cost growth of selected major R&D programs. Also, the amount of overrun is shown to permit visibility into the government-caused cost growth versus that cost growth due to the contractor's inability to perform

PAGE 203

Figure 4-3 PERCENT OF TOTAL COST GROWTH AND OVERRUN EXPRESSED AS A FuNCTION OF TOTAL COST, 1983 188 Program Total Cost Growth Overrun Orbiter 40% 17% F-111 Fighter/Bomber 71 10 Shuttle External Tank 79 5 F-16 Fighter 30 0 MX Missile 60 1 F-15 Fighter 7 1 F-18 Fighter 36 29 B-lA Bomber 50 35 E-3A (AWACS) 79 0 Average 50% 11% SOURCE: All data, Orbiter data, obtained directly from the Department of Defense in 1983, and from Rand Report R-2516-DR&E, "Acquisition Policy Effectiveness: Department of Defense Experience in the 1970's," October 1979. Orbiter data obtained from Office of Shuttle Procurement, Johnson Space Center, National Aeronautics and Space Administration, 1983.

PAGE 204

189 the work within the negotiated amount. Data in Figure 4-3 were obtained directly from the responsible program offices within the Department of Defense, and from a study by the Rand Corporation. Therefore, the data are considered to be highly reliable and certainly represent the best information available. As can be observed from Figure 4-3, the average cost growth was 50 percent, while the Orbiter total cost growth amounted to 40 percent of the total DDT&E project cost. Data for the other programs were also limited to the DDT&E phase of the respective program; hence, a fairly comparable analysis is possible. Perhaps the most accurate gauge of cost performance is total cost growth rather than drawing a distinction between contractor overrun and the remaining cost growth typically caused by additional government-generated requirements. Obviously, from a Congressional and OMB viewpoint, they are concerned with total cost growth and not generally a distinction between overrun and government-caused cost growth. Another factor that gives validity to analyzing total cost growth is the different philosophies that can be employed when negotiating contract changes. Many times, for example, the government may order a change to a contract because the contractor may be having difficulty meeting:a specification for a particular subsystem or component. In making the change, the government often times decides to upgrade or improve the system. When this happens, it is difficult, many times, to determine clearly the cost impact caused by government versus the impact caused by the contractor.

PAGE 205

190 Thus, when the change is negotiated, there are two perfectly legitimate and legal ways to negotiate an equitable adjustment. One method is not to agree to any $pecific amount that is caused by the con tractor, and hence, no overrun would be agreed to. However, in this case a lower than normal fee (profit) would be negotiated. A second approach would be for the government and the contractor to agree to two cost impact amounts: one, for the overrun; and two, for the product improvements. In this latter case, additional fee would be paid only for the product improvements. While the additional fee dollars should come out precisely the same under either of the above two approaches, the first method would not identify any overrun while the second approach would. Thus, one has to be somewhat careful in viewing elements of cost growth without keeping the total cost growth in proper perspective. Selected data from a report issued by the General Accounting Office (GAO) in September 1980, compares the cost growth of several major federal programs. Figure 4-4 depicts the cost growth of 24 classes or groups of federal programs. Consistent with the point made in a preceding paragraph, there is no distinction between government-caused cost growth and contractor overrun. One can observe from Figure 4-4 that the cost of the Orbiter program, in comparison with the Congressional commitment, is one of six programs that underran the Congressional commitment. Considering the complexity of such a major undertaking, the Orbiter DDT&E cost mance was in excellent posture in relation to the other programs.

PAGE 206

PERCENT 400 300 200 . 100 80 60 40 20 1-1-f-1-I2 --Figure 4-4 FINANCIAL STATUS OF MAJOR FEDERAL ACQUISITIONS ACCORDIN.G TO THE U.S. GENERAL ACCOUNTING OFFICE 19so'8-i9so's 6 1 8 7 :;:: 9 10 17 -16 4 3 19 18 15 ::r --14 13 BASELINE ESTIMATE f----.._ ___ ---,,_ 5 l r -40 22 11 124 :-ORBITER vs 23 COMMITMENT 21 1950s. 1960s 1970s 1980s -------SOURCE: U.S. General Accounting Office Report to the Congress, "Financial Status of Major Federal Acquisitions" as of September 30, 1980. 1-' \0 1-'

PAGE 207

Figure 4-4 Continued FINANCIAL STATUS OF FEDERAL ACQUISIT!OUS ACCORDING TO 'I'UE U.S. GENERAL ACCOUNTING OFFJCE 1 11 WEAPONI IYIUIII HAIIVAIID UNIVERSITY -AN ECONOMIC ANALYIIIII I U WEAPONS SYSTEMS RAND CORPORA liON PRDICJABILIJY Of COSTS, TIME I SUCCESS DEVELOPMENT 3 45 WEAPONS SYSlEMS 1172 GAD REPORT TO HOUSE Of REPRESENTATIVES 4 SENATOR PROXMIRE -DR. CURRIE CONGRESSIONAL HEARING, 1974 I 5 ORBitER NASA/OMB COMMITMENT I 8 DEPARTMENT OF .IUSIICE AliEN DOCUMENTATION, IDENTIFICATION I TELECOMMUNICATION, IMMIGRATION I NATURALIZATION SERVICE 'I 2&5 PROJECTS-DEPARTMENT Of TilE ARMY, CORPS OF ENGINEERS NAVIGAJUiN, FLOOD CONTROL, BEACH EROSION I POWER I DEPARTMENT Of OfFENSE, DEPARTMENT OF THE ARMY IMPROVED HAWK MISSILE II 13 PROJECTS tENNESSEE VALLEY AUTHORITY NUCLEAR PLANT UNITS, RESERVOIRS, DAMS 10 .ZO. PROJECJS-VETERANS ADMINISTRATION-MEDICAL CENTERS 11 NAIIONAL SCIENCE FOUNDATION VERY LARGE ARRAY RADIO TELESCOPE 1:1! 7 PROJECTS-DIStRICT Of COLUMBIA-UNIVERSITY, OFFICE BUILDINGS, WATER MAINS, WATER TREATMENT PLANT 13 DEPARTMENT Of DEFENSE, DEPARTMENT OF THE NAVY ADVANCE ASW TORPEDO 14 DEPARTMENT OF DEFENSE, DEPARTMENT OF THE NAVY SHIP COUNTERMEASURE MINE 11 OEPARJMENT OF TRANSPORTATION, FAA, RESEARCH I DEVELOPMENT VOICE SWITCHING CONTROL SYSTEM 11 ARCHITECT Of THE CAI'IlOL, LIBRARY Of CONGRESS JAMES IIADISON BUILDINI 17 DEPARTMENT OF DEFENSE, DEPARTMENT OF THE AIIMY LANCE MISSILE 11 DEPARTMENT OF ENERGY IMPROVE SlEAII PLANT Y-12, OAK RIDGE, TENNESSEE 11 NASA SPACE TELESCOPE 20 DEPARTMENT OF INTERIOR-NAVAJO INDIAN IRRIGATION PROJECT, NEW MEIIICO 21 ENVIRONMENTAL PROJECTION AGENCY, WASTEWATER TREATMENT WORK, HONOLULU, HAWAII 2Z ,DEPARTMENT OF THE ARMY, CORP OF THE ENGINEERS, BEACH EROSION -E. INLET TO ROCKAWAY INLET & JAMAICA IIAY, N.Y. 23 DEPARJMENJ OF THE ARMY, CORP OF ENGIIfEERS, DADE CO. BEliCH EROSION CONTROL, FLOIIIOA 24 HNERAL IIERVICEI ADMINISTRATION, COIIRllfDUS. rrllERIIL Rlllli!IND I PARKIND FACILITY SOURCE: U.S. General Accounting Office Report to the Congress, "Financial Status of Major Federal Acquisitions", as of September 30, 1980. ..... \0 N

PAGE 208

193 Cost Overrun As stated earlier, the "overrun" category of cost growth is cost growth attributable to the contractor's inability to complete contract requirements within the negotiated amounts. In cost-reimbursement contracts, the negotiated amounts are only estimates since the contractor will be reimbursed for the actual incurred cost. Thus, actual expenditures in excess of the negotiated value becomes "overrun", unless such over-expenditure is caused by governmentdirected new work, e.g., contract changes. This last dimension, contract changes, will be addressed in the next chapter. As stated earlier, the overrun for the Orbiter DDT&E project amounted to $759 million. There was a contract overrun even though the Orbiter project actual costs were less than commitments made to the Congress and OMB. Equating to approximately 17 percent of the total Orbiter DDT&E project cost, the $759 million overrun, or that amount of the expenditures the contractor did not receive any fee or profit for, compares very favorably with the other projects in Figure 4-4. While the average overrun was 11 percent, based on a simple arithmetic average, the Orbiter, being a spacecraft as opposed to an aircraft, is considered by the industry and government to be a much more complicated vehicle. It is difficult, if not impossible, to determine in long duration, cost-reimbursement type contracts when a overrun actually begins from a timing standpoint. Overrun is the difference between

PAGE 209

194 the estimated cost at completion and the negotiated contract target cost. Since the Orbiter contract was incrementally funded, and the periodic assessments as to the percentage completion of the work were, to a large degree, judgmental, it is extremely difficult to determine among incurred cost as to what is overrun versus what is baseline contract cost. One can however, be fairly accurate as to the total amount of overrun. An assessment was made by the Orbiter contractor as to how the $759 million was spread from a timing standpoint. That analysis is depicted in Figure 4-5. It can be observed that in the late 1970's the overrun began to be recognized and as the work progressed, the amount of overrun increased substantially. Inflation Contributing significantly to the cost growth of the Orbiter DDT&E project was inflation. Ac'tually, inflation amounted to considerable more than had been estimated at the time of negotiations of the contract. Figure 4-6 compares the contractor's proposed inflation rates for years 1975-1981 with the actual Consumer Price Index (CPI) for those years. Although the contract was awarded in 1972, the years 1972-1974 were omitted since the contract estimates were not negotiated until 1975 and included the actual cost for those years. Based on estimates of inflationary impact prepared by the contractor, the total additional inflation experienced under the

PAGE 210

5000 4000 en 3000 H :J :z: ... en 3 2000 0 R 1000 75 Figure 4-5 TARGET COST VERSUS ACTUAL COST ORBITER DDT&E 1984 76 77 78 79 80 81 82 76T ---}$7S8.6M 83 LEGEND Actual Cost Contract Target Cost SOURCE: NASA Form 533 Financial Report, May 1983. Orbiter Contractor Financial Report. 1-' \0 V1

PAGE 211

Year 1975 1976 1977 1978 1979 1980 1981 SOURCE: 196 Figure 4-6 COMPARISON OF CONTRAcroR PROPOSED INFLATION WITH ACTUAL CONSUMER PRICE INDEX (CPI) 1975-1981 Contractot Actual Proposal Price Index 7.2 9.1% 7.4 5.8 7.6 6.5 7.1 7.6 6.7 11.5 6.4 13.5 6.1 10.2 1 Office of Shuttle Procurement Johnson Space Center National Aeronautics and Space Administration 2u.s. Council of Economic Advisers, Economic Indicators; (Washington, D.C.: U.S. Government Printing Office, 1981), pp. 23-4.

PAGE 212

197 contract, in addition to the inflation priced into the basic contract, was in the $70 million to $80 million range. Double digit inflation created the funding problems as discussed above, which resulted in schedule changes to.the delivery dates of the Orbiter spacecraft. Cost Growth Assessments By Survey Respondents In the survey analyzed in Chapter 3, respondents were given questions on the suggestion that inadequate/instable funding, technical uncertainty/difficulty, and technical advancement are often the major causes of cost growth for major R&D projects. Respondents were asked to rate these three causes, on a scale of 1 to 10, as they saw those causes affecting or contributing to the cost growth of the Orbiter project. Appendix F contains a statistical analysis of the three categories of cost growth. Included are data by the survey group collectively and also by job category of the group members. Figure depicts the respondents average rating for the three categories of cost growth discussed above, along with the percent responding with a rating of 6 or higher on a scale of 1 to 10. As indicated, there is strong belief by the survey participants that inadequate funding, technical uncertainty, and technical advancement were major contributors to cost growth. These three causes of growth are elaborated on in the following paragraphs.

PAGE 213

Figure 4-7 SURVEY RESPONDENTS AVERAGE (MEAN) COST GROWTH RATING QUESTIONNAIRE FOR GOVERNMENT PERSONNEL, 1984 Cause of Cost Growth Inadequate and Instable Funding Average (Mean) Rating On Scale of 1 to 10 7.313 Technical Uncertainty/ Difficulty 6.281 Technical Advancement 4.844 N = 32 Standard Deviation 2.533 2.159 2.864 198 Percent Rating Cause 6 or Higher 84.4% 71.9 37.5

PAGE 214

199 Inadequate or Instable Funding This reason for cost growth, as stated earlier in this report, refers to the government's inability to provide funds to the contractor in accordance with the contractor's requirements. Based on a review of the data, 84 percent of the respondents rated this reason a 6 or higher on a scale of 1 to 10. Overall, the respondents collectively rated this reason for cost growth 7.3. Of particular significance is the fact that 100 percent o"f the procurement managers rated this cause of cost growth a 6 or higher. In fact, 83 percent of the procurement managers rated this reason 8 or higher. Budget managers rated this cause very high also, with 80 percent of them with a rating of 6 or higher. Project managers scored this area the lowest, with 62.5 percent responding with a rating of 6 or higher. Respondents, overall, had beliefs that inadequate funding contributed significantly to the Orbiter cost growth. As discussed earlier in this chapter, inadequate funding actually increased the cost of the Orbiter project by $450 million. Twenty-five percent of the cost growth was caused by inadequate funding--($450 million out of total growth of $1,790). Technical Uncertainty/Difficulty Technical uncertainty refers to the difficulty that is typically encountered during major R&D undertakings. Respondents' perceived technical uncertainty to be a major cause of cost growth, but not quite as significant as inadequate funding. Based on the

PAGE 215

200 responses, the overall average rating was 6.3 on a scale of 1 to 10, with 72 percent responding with a rating of 6 or higher. Budget managers responded with 80 percent scoring 6 or higher. Project management personnel responded with a 75 percent rating in this cost growth category with a 6 or higher, while 67 percent of the procure-ment managers responded with a rating of 6 or higher. As is typical with other major R&D projects, technical uncertainty was perceived to be a significant contributor to the cost growth of the Orbiter i project. Cost growth due to technical uncertainty was a prime con-tributor to overrun for the Orbiter project: Overrun, of course, is non-fee bearing effort and is the responsibility of the contractor, while cost growth due to the government's failure to fund properly is subject to an equitable fee adjustment to the contractor. Overrun on the Orbiter project amounted to $759 million and constituted 42.4 percent of the total cost growth. Technical Advancement Technical advancement typically refers to product improvements and a general upgrading of subsystems and components to the latest technology. While a fairly significant perceived reason for cost growth of the Orbiter project, this category was clearly in third place to the prior two categories. Overall, respondents rated technical advancement 4.8 on a scale of 1 to 10, with a 37.5 percent this category a 6 or higher. Procurement managers preceived

PAGE 216

201 this area to be a larger contributor to cost growth than other respondents, and responded with 50 percent rating this area a 6 or higher. Twenty-five percent of the project management personnel responded with a 6 or higher, while only 20 percent of the budget managers rated this area a 6 or higher. Survey participants were requested to indicate the major causes of cost growth from the perspective of their managerial roles during the DDT&E phase of the Orbiter project. Figure 4-8 lists the major causes of cost growth indicated by those participating in the survey. Appendix G contains a computer analysis of the major causes of cost growth by the job category of those individuals making a particular response. A summary of that computer analysis is presented in Figure 4-9. Representing the most prominent cause for cost growth of the Orbiter project was inadequate project funding. Twenty-three participants, or 74 percent of the total participating in the survey listed the project funding problem as a prime contributor to cost growth. From reviewing the data presented earlier in this.chapter, it can be observed that the impact of government-directed funding limitations increased project cost by $450 million. From reviewing the survey data, procurement, and budget personnel responded more frequently than project management personnel with inadequate funding as a major cause of cost growth. In fact, 80 percent of the budget and procurement personnel mentioned inadequate funding or funding limitations as a cause for cost growth while 50 percent of the

PAGE 217

Figure 4-8 MAJOR CAUSES OF COST GROWTH FOR ORBITER PROJECT QUESTIONNAIRE FOR GOVERNMENT PERSONNEL AWARD FEE CONTRACTING, 1984 202 Cause Number Responding Percent Responding 1. Inadequate funding/funding limitations, causing program replanning and rescheduling. 2. Technical uncertainty-development problems. 3. Thermal Protection System (tiles) specifically identified as t,echnical uncertainty. 4. Optimistic program plans and schedules. 5. Inflation. 6. Technical advancement product improvement and other contract changes. 7. Low initial estimates by the contractor. N = 31 23 74.2% 15 48.4 6 19.4 3 9.7 7 22.6 10 32.2 5 16.1

PAGE 218

Figure 4-9 SUMMARY ANALYSIS MAJOR CAUSES OF COST GROWTH QUESTIONNAIRE.FOR GOVERNMENT PERSONNEL, 1984 CMT 21 CMT 22 CMT 23 CMT 24 CMT 25 CMT 26 CMT 27 Project a. 4 4 3 1 1 2 1 Monngement b. 50.0 50.0 37.5 12.5 12.5 25.0 12.5 c. 17.4 26.7 50.0 33.3 14.3 20.0 20.0 Procurement a. 10 5 2 0 4 4 0 Mnnagement b. 83.3 41.7 16.7 0 33.3 33.3 0 c. 43.5 33.3 33.3 0 57.1 40.0 0 Budget a. 4 2 0 1 1 2 4 Analyst b. 80.0 40.0 0 20.0 20.0 40.0 80.0 c. 17.4 13.3 0 23.3 14.3 20.0 80.0 Other a. 5 4 1 1 1 2 0 b. 83.3 66.7 16.7 16.7 16.7 33.3 0 c. 21.7 26.7 16.7 33.3 14.3 20.0 0 Total a. 23 15 6 3 7 10 5 b. 74.2 48.4 19.4 9.7 22.6 32.3 16.1 CMT = Comment NOTE: Comment numbers correspond to numbers in Appendix G. a. Number responding b. Percent responding in job category SOURCE: Computer Analysis of c. Percent of total column Survey Data. d. Percent of total participants in study See Appendix G N 0 w

PAGE 219

Comment Number 21 22 23 24 25 26 27 Figure 4-9 Continued SUMMARY ANALYSIS MAJOR CAUSES OF COST GROWTH QUESTIONNAIRE FOR GOVERNMENT PERSONNEL, 1984 204 Inadequate funding/funding limations, causing program replanning and rescheduling. Technical uncertainty-development problems. Thermal Protection System (tiles) specifically identified as technical uncertainty. Optimistic program plans and schedules. Inflation. Technical advancement/product improvement and other contract changes. Low initial estimates by the contractor. NOTE: Comment numbers correspond to numbers in Appendix H. SOURCE: Computer Analysis of Survey Data See Appendix H

PAGE 220

205 project management personnel indicated that reason. Also, 80 percent of the respondents in the "other" job category, which included government attorneys, believed that inadequate funding significantly contributed to the cost growth. It was not surprising that the administrative personnel, i.e., procurement, budget, and legal, were more sensitive to the funding issues than project management personnel since administrative people are generally less sympathetic to technical changes. At any rate, the cost to the government for delaying funding actions proved to be quite severe. It should be noted, however, that the ability to properly and timely fund the Orbiter contract was beyond the control of NASA. The problem resided mainly with the Office of Management and Budget (OMB) and the Congress. Another frequently mentioned cause of cost growth was the technical problems encountered during the Orbiter development. A _total of 15 people responded with that reason for cost growth, representing almost 50 percent of those participating in the survey. One-half of the project management personnel believed technical uncertainty to be a major contributor to cost growth, while in excess of 40 percent of the procurement and budget managers responded with that reason. As noted in the literature review chapter, technical uncertainty and technical complexity have been found to be prime contributors to cost growth of major R&D projects. Six of the respondents specifically mentioned the thermal protection system on the Orbiter as a major development complexity.

PAGE 221

206 Representing perhaps the single most technical complexity that contributed to the schedule delay in the launch of the Space Shuttle from the Orbiter standpoint, the thermal protection system was highly critical to a safe reentry from space. In the context used here, the thermal protection system refers to the tiles on the fuselage and wings of the Orbiter. More than 30,000 individual tiles, most of which were of different sizes and shapes, were required to be bonded to the skin of the Orbiter spacecraft. Representing a significant technical challenge, never befor.e had any country developed a spacecraft the size of a DC-9 jet passenger plane to be flown in space and returned to Earth. Notwithstanding the thermal protection system problem, the government and its industrial contractors overcame the difficulties, and the Orbiter spacecraft has been successfully flown several times as of this writing, with no major problems. Technical advancement, product improvement, and contract changes were major contributors to the cost growth of the Orbiter development project as indicated by a third of the survey participants. This was not an unexpected finding in conducting this research project, since, as discussed in the literature review chapter, these areas were found to be major contributors to cost growth in R&D projects. In dealing with major defense and space systems it is not uncommon for the government to have a secondary objective of striving for improvements or advances to the state-of-the-art technology. Those who do not have real budget responsibility for a project

PAGE 222

207 often push for product improvements. Some of the survey respondents cited that "product improvement is the nature of the game when dealing with R&D". In dealing with the appropriateness of the cost of technical advancement, product improvement, and other types of contract changes, a logical way to review the cost is in relation with other programs. As elaborated on earlier in this chapter, the cost growth of the Orbiter compared very favorably with other complex R&D programs. If each of the many hundred changes issued on the Orbiter contract was to be evaluated individually, each would most likely be found to be mandatory to someone. Inflation was listed as a major contributor to cost growth by several of those surveyed. As elaborated on earlier in this chapter, the Orbiter project actual cost was less than the commitments to the Congress and OMB. However, there were inflation estimates negotiated into the basic Orbiter contract. Additionally, major increments of work were added to the contract, the estimates included projections for inflation. As noted earlier in this chapter, it has been estimated that unanticipated inflation amounted to $70 million to $80 million. This simply means the contractor did not get any fee on that amount, but the contractor recovered the actualcost incurred. Another major cause of cost growth, as mentioned by several survey participants, was low initial estimates from the contractor. In contracting, the only penalty associated with low initial estimates, assuming the firm was selected for contract

PAGE 223

208 award, is that the contractor does not receive any fee for cost growth that is not caused by the government. In fact, one school of thought suggests that some firms attempt to recover through the changes clause for low initial estimates. There was no evidence of such an approach at the beginning of the project or in the performance of the Orbiter contract. Still it must be pointed out that there is little incentive for firms to propose cost estimates significantly in excess of government estimates and estimates by outside reviewers such as OMB. Rather, past history would indicate that many federal programs are shaped to fit the available resources. That approach, in itself, is not all bad because many federal programs, especially major R&D programs, could be structured to literally consume unlimited resources. A last cause of cost growth indicated by the survey respondents was optimistic program plans and schedules. There are pros and cons associated with optimistic program planning and schedules. A major advantage of optimistic plans and schedules is that firm dates and plans that represent real challenges are set in advance. A disadvantage is that. such optimism may not allow for adequate time for certain testing. Moving on to further project milestones could mean the of resources prematurely. Thus, a delicate balance is needed between realism and optimism in establishing program plans and schedules.

PAGE 224

209 Methods NASA Could Have Used to Reduce Cost Growth In the conduct of the survey of government managers involved in the management of the Orbiter development contract, managers were asked to suggest ways they believe NASA could have reduced cost growth on the Orbiter project. Results of the survey data are contained in the following paragraphs. Again, one must remember that all comments analyzed in this section are from open-ended questions and have been categorized. Survey participants responded to the of ways NASA could have reduced cost growth by suggesting a number of ways that cost growth could have been minimized. Figure 4-10 depicts the suggestions along with the of people responding with that particular suggestion. As can be noted from Figure 4-10, many of these suggestions relate directly back to the reasons for cost growth as shown in Figure 4-8. Appendix H contains a computer analysis by job category of the respondents, of the responses given for ways NASA could have reduced cost growth. A summary of Appendix H is presented in Figure 4-11. Respondents overwhelmingly agreed that stability of funds and less optimistic budgeting would have reduced the cost growth of the Orbiter project. This perception was shared by over 50 percent of the respondents. Rationale for this response directly relates to the funding instability problem elaborated on earlier in this chapter as one of the reasons for cost growth, and need not be further elaborated on.

PAGE 225

210 Figure 4-10 METHODS NASA COULD HAVE USED TO REDUCE COST GROWTH OF ORBITER PROJECT QUESTIONNAIRE FOR GOVERNMENT PERSONNEL AWARD FEE CONTRACTING, 1984 Number Responding 1. Stability of funds and budgeting for 16 development contingencies. 2. Qualify development systems before 6 production/freeze design early. 3. Place more responsibility on contractor. 3 Less 4. Tighter control of contract changes. 8 5. Build all production Orbiters at same 1 time and to the same design. 6. Purchase spares concurrent with production. 2 7. Build a simpler spacecraft with less 1 capability and value. 8. Incentive contract would have been more 1 meaningful to reduce cost. 9. Better management of subcontracts. 1 10. Better estimates of project cost. 3 N = 31 Percent Responding 51.6% 19.4 9.7 25.8 3.2 6.4 3.2 3.2 3.2 9.7

PAGE 226

Figure 4-11 SUMMARY ANALYSIS METHODS NASA COULD HAVE USED TO REDUCE COST GROWTH OF ORBITER PROJECT QUESTIONNAIRE FOR GOVERNMENT PERSONNEL, 1984 CMT 31 CMT 32 CMT 33 CMT 34 CMT 35 CMT 36 CMT 37 CHT 38 CMT 39 CMT 40 --, Project a. 3 3 0 3 0 0 1 1 0 1 Management b. 37.5 37.5 0 37.5 0 0 12.5 12.5 0 12.5 c. 18.8 50.0 0 37.5 0 0 100.0 50.0 0 33.3 Procurement n. 7 2 2 0 1 2 0 1 1 1 Management b. 58.3 16.7 16.7 0 8.3 16.7 o 8.3 8.3 8.3 c. 43.8 33.3 66.7 0 100.0 100.0 0 o.o 100.0 33.3 Budget a. 2 0 0 3 0 0 0 0 0 1 Analyst b. 40.0 0 0 60.0 0 0 0 0 0 20.0 c. 12.5 0 0 37.5 0 0 0 0 0 33.3 Other a. 4 1 1 2 0 0 0 0 0 0 b. 66.7 16.7 16.7 33.3 0 0 0 0 0 0 c. 25.0 16.7 33.3 25.0 0 0 0 0 0 0 Total a. 16 6 3 8 1 2 1 2 1 3 d. 51.6 19.4 9.7 25.8 3.2 6.5 3.2 6.5 3.2 9.7 CMT "' COMMENT NOTE: Comment numbers correspond to number in Appendix H. a. Number responding b. Percent responding in job category SOURCE: Computer Analysis of c. Percent of total column survey Data d. Percent of total participants See Appendix H N ...... ......

PAGE 227

212 Figure 4-11 Continued SUMMARY ANALYSIS METHODS NASA COULD HAVE USED TO REDUCE COST GROWTH FOR ORBITER QUESTIONNAIRE FOR GOVERNMENT PERSONNEL, 1984 Comment No. 31 Stability of funds/budgeting for development contingencies/ less optimistic budgeting. 32 Qualify development systems before production/freeze design early. 33 Place more responsibility on contractor less government monitoring. 34 Tighter control of changes. 35 Build all production Orbiters at same time and to the same design. 36 Purchase spares concurrent with production. 37 Build a simpler spacecraft with less capability and value. 38 Incentive contract would have been more meaningful to reduce cost. 39 Better management of subcontracts. 40 Better estimates of project cost. NOTE: Comment numbers correspond to numbers in Appendix H. SOURCE: Computer Analyses of Survey Data See Appendix H

PAGE 228

213 Another way NASA could have reduced cost growth of the Orbiter project, as perceived by a number of survey participants, would have been to qualify development systems before flight hardware production, and to have a "design freeze" earlier in the project. A "design freeze" refers to having an approved design and not making any more changes unless mandatory. Almost 40.?ercent of the project management personnel participating in the survey mentioned that qualifying development systems before production of flight hardware, which would have led to a design freeze, would have reduced cost growth. However, with optimistic manufacture and launch schedules, it was not possible to qualify all development systems prior to proceeding with flight hardware manufacture. This was a risk inherent in the structure of the project. Placing more responsibility on the contractor and less government monitoring was mentioned as another way NASA could have reduced the cost growth of the Orbiter project. Comments of this nature relate directly to the management style employed by the government in managing the contract. While most would agree that placing more responsibility on the contractor was and is a good thing to do, it is very judgmental whether the effect of government monitoring contributed to cost growth of the Orbiter project. Tighter control of changes was stated by approximately 25 percent of the respondents as being a major contributor to cost growth. Evidence presented earlier in this chapter indicates that contract changes amounted to approximately $581 million for the ...

PAGE 229

214 Orbiter DDT&E project. It is beyond the scope of this research project to evaluate which changes were considered mandatory and which were product improvements. However, the point is well made that tighter control of changes would have most likely reduced project cost. One can also take the position that had some of the nonmandatory changes not have been made, the quality of the spacecraft would not have turned out as well as it did. An indepth evaluation of the reasons behind the contract changes and the associated cost would constitute a major and worthwhile research endeavor. Three ways were mentioned that NASA could have taken in order to reduce project cost related to each other: (1) build all production Orbiters at the same time and to the same design; (2) purchase spares concurrent with production; and, (3) build a simpler spacecraft with less value and capability. Dealing with the first point, there is little question that manufacturing all spacecraft concurrently would have.yielded less cost per spacecraft and lower total project cost. The fixed cost associated with a major project such as the Orbiter is very significant. However, funds were not available from the Congress and OMB to proceed with concurrent production. Design changes made to selected Orbiters were considered necessary to meet unique payload requirements. Procurement of spare parts concurrent with production would reduce the unit cost of spare parts. There were two extenuating circumstances precluding concurrent purchase of spare parts. First,

PAGE 230

215 NASA was not in a position to determine spare parts quantity, and second, funding was not available to buy any significant quantities of spares. In dealing with the comment that NASA could have built a spacecraft that was technically simpler and with less capability and value, one has to be cognizant of the purpose of the Orbiter spacecraft. There is no question that the cost of the Orbiter would have been less had NASA chosen to build a less sophisticated vehicle, however mission needs would not have been met. Other comments suggesting ways NASA could have reduced the cost growth of .the Orbiter included incentive contracting, better management of subcontracts, and better estimates of project cost. Incentive contracting refers to an objective fee formula, as described in Chapter 1, with a target cost and shareline for actual fee determination. Normally, in the design and development phase of major projects, it is not possible to establish firm target costs. Also, due to the flexibility typically desired by the government as well as the contractor, an incentive type of contract could be an inhibitor to a smooth working relationship between the two parties. Thus, the flexibility awarded by the award fee type of contract is normally more highly desired during developmental phases of work. Finally, better estimates of project cost were given as a way NASA could have reduced the cost growth for the Orbiter project. This is a valid point as discussed under the previous section of this chapter dealing with the major causes of cost growth and, therefore, will not be further elaborated on here.

PAGE 231

216 Summary This chapter presented an analysis of the cost growth of the Orbiter DDT&E project. Based on the data collected an analyzed, the cost growth of the Orbiter DDT&E project was not out-of-line with what one may have reasonably predicted the cost growth to be. While there was cost growth experienced under the contract, the cost outcome was less than the commitments made to the Congress and Office of Management and Budget (OMB). Data presented in this chapter indicated that the total cost growth, government-caused and contractorcaused, for the Orbiter project was 40 percent of the total project cost. Average total cost for the other major R&D projects presented was 50 percent. From analyzing the Orbiter DDT&E cost data in the official contract records, and from evaluating the results of the survey instrument, it was found that the most significant causes for cost growth were contract changes, instability of funds, and overrun. Notwithstanding the cost increases, total project cost as indicated above,,was less than the original commitments to the Congress and OMB. Major perceived reasons for cost growth for the Orbiter project by the survey participants were inadequate funding, technical uncertainty, inflation, and over optimistic program planning. These perceived reasons were basically in line with reality. Technical uncertainty caused many contract changes, and inflation and overoptimistic program planning contributed to the overruns on the Orbiter project.

PAGE 232

NOTES -CHAPTER IV 1 The original commitment for the total Shuttle Program DDT&E in Fiscal Year 1971 dollars was at the total program level and not in terms of project elements such as the Orbiter. A reference for this is NASA Headquarters Memorandum, dated March 17, 1981, Subject: Space Shuttle.Program Costs, Public Affairs Officer, Office of Space Transportation System. 2 The total Orbiter project commitment by NASA to the Office of Management and Budget was $3,438 million and included Orbiter support contracts in addition to the prime contract for the Orbiter spacecraft. As shown in Figure 4-2, the negotiated value of the basic Orbiter spacecraft contract was '$2,630 million and a final cost of $4,420 million excluding fee of 'approximately $260 million. Thus, the total value of the Orbiter DDT&E spacecraft project was $4,680 million compared with the total Orbiter project cost of $4,942 million. The $262 million difference between $4,942 million and $4,680 million was for support contracts. 3 The Orbiter DDT&E project actual cost versus commitment, including fee and Orbiter support contracts, based on data compiled for the Shuttle Program Office, Johnson Space Center, in report dated August 30, 1983, entitled "Orbit,er Actuals vs. Agency Commitment." 4Ibid. 5 Tbe number of cost and schedule tradeoff options, estimated to be in excess of 100 by the contractor, includes requests by NASA Headquarters as well as the Johnson Space Center. This information was based on information informally obtained by the writer from the prime contractor in August 1983.

PAGE 233

CHAPTER V CONTRACT CHANGES Introduction Contract changes consist of directives from the government contracting officer to "change" the work under contract. Typically, contract changes modify the contract by altering the contract state-ment of work in the areas of spe,cifications, test requirements, or 1 other elements of the contract. Although the "change", referred to as a contract change authorization (CCA), is unilaterally issued by the government, the contract provides that an equitable adjustment be made to the contract cost and fee. As previously mentioned, in major research and development (R&D) projects for defense and aero-space systems, there are usually large numbers of contract changes that add significantly to program costs. This is primarily. due to the fact that major R&D contracts are directed toward objectives for which the work cannot be precisely described at the time of initial contract award. This chapter consists of an analysis of the significance of contract changes on the Orbiter design, development, test, and evaluation (DDT&E) contract. Although the majority of the CCA's on the Orbiter DDT&E contract caused increases in the contract cost and fee, there were a small number of changes which resulted in decreases

PAGE 234

219 to the contract expenditures. Impact of Changes Figure 5-1 depicts the cost growth summary for the Orbiter DDT&E contract. As can be noted, the CCA's issued by the government contributed $581 million to the contract cost growth. From an aggregate standpoint, the CCA's amounted to 13.1 percent of total contract cost, while the impact of inadequate/untimely funding and overrun amounted to 10.2 percent and 17.2.percent, respectively. Over the period 1972 to 1983, there were a total of 1,468 changes issued under the Orbiter DDT&E contract. Of that number, 1,414 increased the contract cost while 54 changes resulted in decreases to contract cost, as indicated in Figure 5-2. It can be observed that 493 or 33.6 percent of the changes resulted in no effect to contract value. In large part, this was due to the contract threshold provision which required that, in order to change the contract value, the change had to impact (decrease or increase) the contract value by more than $100,000. Of course, the.contractor was paid the entire cost of the change since the contract was costreimbursement, and the essence of such a contract provision meant the contractor did not receive any additional fee for the change. Likewise, for changes decreasing contract value under $100,000, the government did not receive a fee credit. Each change is viewed separately and not cumulative to other changes for threshold consideration. For example, ten $90,000 changes would amount to

PAGE 235

Figure 5-1 SUMMARY OF CONTRACT COST GROWTH (MILLIONS OF DOLLARS) 1972-1983 Basic Contract Cost Growth Government Directed Changes Impact of Government Directed Funding Overrun Total (Cost without Fee) SOURCE: Office of Shuttle Procurement Johnson Space Center $581 450 759 220 $2,630 $4,420 National Aeronautics and Space Administration March 1984

PAGE 236

No Price Adjustment Figure 5-2 .SUMMARY OF .CONTRACT CHANGES ORBITER DDT&E (MILLIONS OF DOLLARS) 1972-1983 Number 493 Increases to Contract Value $0 to $99,999 298 100,000 to 499,999 391 500,000 and over 232 Subtotal 1,414 Decreases to Contract Value 54 Totals 1,468 SOURCE: Office of Shuttle Procurement Johnson Space Center National Aeronautics and Space Administration March 1984 221 Dollar Value 0 $13.7 91.0 580.9 --$685.6 (105 0 0) $580.6

PAGE 237

222 $900,000 but would be within the threshold they were individually less than $100,000. There were certain exclusions to the contract changes threshold for making equitable adjustments to the contract value. These included special studies, spare parts, ground support equipment, and a few other minor exceptions. Regardless of dollar value of those latter items, the contract cost and fee were adjusted. Hence, 298 changes in the $0 to $99,999 category consisted mainly of changes for special studies, spare parts, and ground support equipment. Notwithstanding the exclusions from the threshold, the threshold eliminated the necessity to negotiate approximately one-third of the changes. The dollar value of the 298 changes in the $0 to $99,999 category amounted to approximately 2 percent of the total value of the changes, while the number amounted to 20.3 percent. In total, 791 changes or 54 percent of the changes were valued at less than $100,000, while representing only 2 percent of the total dollar value of the changes. One can observe that had the threshold covered all changes under $100,000, with no exclusions, over one-half of the changes (791) would have fallen into that category. Therefore, the threshold functioned as a very meaningful method to eliminate a sizeable effort in terms of staffing requirements for the government and the contractor to negotiate those changes. Another major benefit to the government by having a large changes threshold is that the contractor is motivated to not

PAGE 238

223 recommend changes to the government that would fall within the threshold. This is due to the fact that the contractor would receive no fee for threshold changes. Hence, contractors are motivated to recommend changes that exceed the threshold so that they can receive fee for the changes. (Contractors' normally are motivated to recommend contract changes since changes represent additional sales and profit.) The observation can be made from Figure 5-2 that the majority of the changes, from a dollar value standpoint, were over $500,000 each. Collectively, those changes amounted to 84.7 percent of the total changes having an increase to contract value, and equaled the net value of all changes ($581 million) after offsetting for those changes which decreased contract value. From a volume standpoint, the changes in the over "$500,000" category amounted to only 15.8 percent. Thus, had the changes threshold been negotiated at a $500,000 level in lieu of $100,000, approximately 84 percent of the changes would not have required negotiation. A more comprehensive discussion of changes threshold is presented below. Changes Threshold Changes thresholds are typically established at the time of initial contract negotiations. A changes threshold cannot be dictated by the government. Rather, it is established by mutual

PAGE 239

224 agreement through the negotiation process. At the time of negotiation of a threshold, the contractor, as well as the government, projects the estimated value of changes within given threshold amounts, e.g., $100,000, $500,000, etc., and strives to negotiate such amounts into the initial contract value. Thus, for example, if a contractor and the government estimate that there will be $25 million in threshold changes in a billion dollar contract, the initial contract value would be increased to compensate for the anticipated threshold changes. Since thresholds for changes are generally found only in cost-reimbursement type contracts (even though fixed-price contracts occasionally have small thresholds for changes) there is no cost risk to the contractor. In other words, if the $25 million, from the prior example, did not materialize, the government would not pay any of the $25 million; however, to the extent that additional fee was paid the contractor for a threshold provision, the contractor would be paid fee. On the other hand, should the threshold changes amount to $50 million, the government would pay the $50 million with no additional fee. Hence, in that case, the contractor would have performed $25 million in extra work without receiving any additional fee. In most R&D contracts, it is not uncommon for the amount of threshold changes to substantially exceed the initial estimates for those types of changes. Also, in competitive new awards, it is not unusual for the government to get a sizeable threshold without having

PAGE 240

225 to increase contract value; thus, a clear advantage to the government. High thresholds save a significant amount of contract administrative effort on the part of the government and the contractor. For example, the contractor does not have to prepare formal, detailed cost proposals for threshold changes and the government avoids the administrative cost associated with performing a cost analysis of the change and conducting the ensuing negotiations. Only budgetary estimates are generally sufficient for threshold changes. Changes with Price Decreases Only 54 changes resulted in contract price decreases for the Orbiter DDT&E contract. These changes are commonly referred to as credit changes. As shown in Figure 5-3, the 54 credit changes, representing less than 4 percent of the total number of changes, amounted to $105 million. While 33 of those changes, amounting to $49 million, affected the spacecraft, one change, amounted to a $23 million credit which deleted a radar rendezvous system from the spacecraft. There were four changes totaling $19 million that consisted of deleting certain spare parts, while seven of the changes worth $37 million deleted selected testing requirements and testing hardware. All in all, the credits amounted to approximately 15 percent of the dollar value of the changes with .price increases.

PAGE 241

226 Figure 5-3 SUMMARY OF CONTRACT CHANGES WITH PRICE DECREASES ORBITER DESIGN, DEVELOPMENT, TEST, AND EVALUATION PROJECT 1984 Number Dollar Value Project Percent Spacecraft Changes 33 $49.0 million Deletion of Testing 7 37.0 and Test Hardware Deletion of Spare Parts 14 19.0 Total 54 $105.0 million SOURCE: Office of Shuttle Procurement Johnson Space Center National Aeronautics and Space Administration March 1984 46.7% 35.2 18.1 100.0%

PAGE 242

227 Orbiter Cost Profile In August 1972, the Orbiter DDT&E contract was awarded. Figure 5-4 presents the DDT&E contract cost expenditures by fiscal year. Those amounts include the dollar value of changes as well as baseline contract effort. It can be observed that the heavy expenditure years were 1975 through 1979. While the DDT&E contract required delivery of two Orbiter spacecraft, only one was capable of going into earth orbit. The other Orbiter was a test vehicle, used in the approach and landing test and entailed releasing the vehicle from the Boeing 747 carrier aircraft. First orbital flight was in April 1981 even though the spacecraft was delivered to the launch site in March 1979. When the vehicle was delivered to the launch site at Kennedy Space Center in Florida, there was still extensive work required to complete installation of the thermal protection system (mainly tiles on the wing and fuselage) prior to launch. This effort at Kennedy Space Center was not envisioned by the government nor the contractor to take as long as it did. For example, when the spacecraft was first delivered in March 1979, expectations were to launch in November 1979. However, as problems with the thermal protection system continued, the launch was incrementally postponed until March 1981. Figure 5-5 presents a breakout of the changes by year. Excluded from Figure 5-5 are the changes for spare parts, overhaul and repair, funding impacts caused by the government, and changes

PAGE 243

228 Figure 5-4 COST PROFILE SHUTTLE ORBITER DESIGN, DEVELOPMENT, TEST, AND EVALUATION PROJECT (MILLIONS OF DOLLARS) 1973-1983 Fiscal Year* Amount Percent 1973 $ 65.3 1.5% 1974 305.7 6.9 1975 563.0 12.7 1976 691.4 15.6 1976 186.8 4.2 1977 705.6 16.0 1978 592.5 13.4 1979 500.0 11.3 1980 343.3 7.8 1981 251.4 5.7 1982 171.0 3.9 1983 45.2 1.0 Total Cost $4,420.2 100.0% *Government Fiscal Year (FY). For FY's 1973-76, the FY covers from July 1 through June 30; 1976T represents the transition period (July 1, 1976September 30, 1976); and FY's 1977-1983 covers from October 1 to September 30 of the respective FY. SOURCE: Office of Shuttle Procurement Johnson Space Center National Aeronautics and Space Administration March 1984

PAGE 244

Year 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 Figure 5-5 CHANGES BY CALENDAR YEAR* ORBITER DESIGN, DEVELOPMENT, TEST, AND EVALUATION PROJECT (MILLIONS OF DOLLARS) 1973-1982 $100,000 $499,000 OVer $500,000 No. No. No. 10 $2.6 17 $46.1 27 21 5.5 15 31.1 36 29-6.5 20 58.4 49 38 9.7 34 117 0 2 72 57 12.9 15 76.3 72 55 12.7 21 84.4 76 47 11.2 10 17.0 57 44 9.5 15 33.7 59 31 6.5 12 21.0 43 19 3.6 5 5.9 24 Totals $48.7 36.6 64.9 126.9 89.2 97.1 28.2 43.2 27.5 9.5 ------Totals 351 $80.7 164 $491.1 515 $571.8 *Excludes changes under $100,000, spare parts, overhaul and repair, funding impact, and changes with price decreases. SOURCE: Office of Shuttle Procurement Johnson Space Center National Aeronautics and Space Administration March 1984 N N I.C

PAGE 245

with price decreases. As can be noted from Figure 5-5, changes continued at a fairly high rate through 1981, even though the dollar value dropped off sharply beginning in 1979 when the vehicle was shipped. Orbiter Development Milestones An analysis was made of the issuance of changes compared with the Orbiter development schedule. Figure 5-6 depicts the major milestones of the Orbiter development compared with the issue rate of changes. The changes correspond with the changes set for in Figure 5-5. Although changes were generated starting in 1973, the dollar value increased significantly after the preliminary design reviews (PDR's) in 1974. Changes continued at a high dollar value for one year after the orbital flight critical design review. Analysis of Significant Changes As discussed earlier in this chapter, 232 of the 1,468 changes to the Orbiter DDT&E contract constituted $581 million, or approximately 85 percent of the dollar value of all changes that increased the contract value. A further analysis was made of all changes over $5 million and it was found that 25 changes fell into that category. Those 25 changes, collectively, had a value of $305 million. In other words, 1.7 percent of the changes (25 out of 1,468) represented 52.6 percent of the net value of all changes, or

PAGE 246

Figure 5-6 SPACE SHUTTLE ORBITER DEVELOPMENT SCHEDULE 1972-1981 YEAR 1972 1973 1974 1975 1971i 1977 1978 1979 NO. C:G/\' s 0 'J.7 36 72 72 76 57 $ CCI\ 1 s 0 $1t9tl $37tl $1i5M $127M $89M $97M $2Rt! AATP 8/72 !::!. PRR I 1/72 I!. SRR 8/73 !::!. PDR 2/74-7/74 !::!. ALT PDR 11/74 l::!.FI.T PDR 3/75 l::!.Al.T CUR It/7ft l::!.ORD FJ.T CDR 10/77 !::!. FCF 5/77 1980 1981 1982 59 43 24 $4JM $:ZIIM $10M lnt ORB Del to Sfte !::!. 3/79 ORB FLT lt/81 ATP Alii hor:t ty tn Prncced I'RR Pror.rnm Review SRR Rcqufrcmcntn Rrvicw PllR Pre! lmln.ny Oc!!fr,n Rt'vfcw AJ.T llppronch :mtl L:mdfnr, Tent Fl. T Fll!lht (Orbftnl) SOURCE: Prepared by Researcher Based on Official Contract Records. Crfttcnl Review ORB Orbiter FCF -First Captive Flight with Carrier Aircrnft office of Shuttle Procurement Jo;t:mson .Space Cent:er National Aeronautics and Space Administration May 1984 N w ......

PAGE 247

232 44.5 percent of the value of all changes with price increases. This is a clear indication that change management by the government should be concentrated on the large dollar changes. Due to the magnitude of the 25 changes over $5 million, an analysis was made of each of those changes to determine the reason for issuing the change. The results of that evaluation are found in the succeeding paragraphs. In conducting the analyses of those changes over $5 million the changes were placed into six categories: additional testing requirements; additional system integration for the total Shuttle system; added vehicle capability; changes required to meet original vehicle requirements; improvements made to the spacecraft that would serve to reduce the cost of spacecraft maintenance; and, added training and simulation effort. Figure 5-7 depicts the number and dollar value of changes falling into each of the six categories. Most significant of all categories was the category of addi-' tiona! testing requirements. Not only was this category the most significant in terms of dollar value, but it was also in terms of numbers of changes. Thirty-five percent of the total value of the changes fell in the category of "additional testing requirements." Of the $107.2 million, $57 million related to testing of the Shuttle avionics system. A substantial part of the avionics effort was the operation of the Shuttle Avionics Integration Laboratory (SAIL) at the Johnson Space Center in Houston, Texas. Due to the criticality of the avionics system, it was considered necessary for the government to have a major integration laboratory at Houston in close

PAGE 248

Figure 5-7 CATEGORIES OF CHANGES OVER $5 MILLION 233 ORBITER DESIGN, DEVELOPMENT, TEST AND EVALUATION PROJECT (MILLIONS OF DOLLARS) 1982-1983 Category Number Percent Additional Testing Requirements 9 $107.2 35.1% Additional Systems Integration 4 62.4 20.4 for Total Shuttle System Added Vehicle Capability 5 58.5 19.2 To Meet Original Vehicle 4 52.9 17.3 Requirements Cost Effective Maintenance 2 15.2 5.0 Added Training and Simulation 1 9.0 3.0 Totals 25 $305.2 100.0%

PAGE 249

234 promimity to the government technical managers responsible for the avionics system. Avionics, in a broad context, refers to the electrical and electronic systems used in aviation. There are several avionics subsystems in the Orbiter: guidance and control, communication, tracking, displays and controls, and electrical power distribution. Additional systems integration activity comprised the second most significant dollar amount of contract changes. This effort entailed the analysis of the total Shuttle system--Orbiter, external tank, Space Shuttle main engines, and the solid rocket booster. Included were tasks associated with propulsion testing, thermal analyses, vehicle performance, ground operations, and other areas associated with the operation of the total Shuttle system. This activity included the identification of single failure points together with an assessment of the associated risks. Actually, this category of changes, amounting to $62.4 million, did not contribute to the cost of Orbiter spacecraft. Rather, the systems integration activity contributed to the cost of the total Shuttle operations cost. Representing the third largest category of changes was added vehicle capability. This area amounted to $58.5 million. Changes in this category dealt mainly with modifying the Orbiter spacecraft to accommodate additional payload capability and mission duration. Constituting the fourth largest area of changes was

PAGE 250

235 modifications to the spacecraft to meet original vehicle requirements. The specific requirement for the work these changes authorized were not part of the original Orbiter contract. These changes, valued at approximately $53 million, represented modifications to spacecraft subsystems to improve the performance and reliability of the Orbiter spacecraft. For example, approximately one-half of the total value of these changes was to improve the reliability of the hydraulic system. As with the hydraulic changes and other changes in this category, the modifications no doubt also improved the safety of the spacecraft while at the same time improved vehicle performance. However, it is difficult to determine what portion of the cost was performance improvement versus safety. Cost effective maintenance was the fifth category of changes over $5 million. These changes, while not improving the function or operation of the Orbiter vehicle, related to reducing the maintenance cost of keeping spacecraft in an operational mode. Additional training and simulation activity represented the last category of changes over $5 million, and mainly consisted of running mission simulations in a crew station mockup for the benefit of training flight crews. As can be noted from the above, a very small percent of the changes constituted almost 50 percent of the value of all changes. Another observation that should be made is that a substantial portion of the cost of the changes was not hardware cost, but rather

PAGE 251

236 additional testing requirements and engineering effort related to flight operations. The remaining question is, how did the Orbiter project compare to other projects? The following section discusses the changes on the Orbiter project in comparison with a study of another major spacecraft project. Orbiter Project Changes Versus Project Lambda A comparison of the impact of changes to the Orbiter DDT&E contract was made with another project analyzed by McGlashan in his late 1960's study of cost growth and changes to a major spacecraft development project called project Lambda. 2 Lambda was a ficti-tious name given to a real project. In analyzing the contract change authorizations (CCA's), on a major spacecraft project, McGlashan reviewed indepth the CCA's over 3 $100,000. These 220 CCA's represented 24 percent of the total number of CCA's over $100,000, and amounted to 99 percent of the dollar value of all CCA's issued that resulted in price increases. He found that only 20 of the CCA's accounted for 80.7 percent of the 4 total proposed price increase. This compares with the Orbiter project where 25 of the CCA's accounted for 45 percent of the total dollar value of CCA's having increases to the contract value. In further analyzing McGlashan's research, it was observed that project schedule extensions associated with funding deficiencies accounted for 30.3 percent of the cost of CCA's (schedule slips were 5 treated as CCA's). This 30.3 percent was reduced to 26.7 percent of the cost growth when viewed against total cost growth, including

PAGE 252

237 overruns. For the Orbiter DDT&E project, the funding deficiencies accounted for 25.1 percent of the total cost growth, including overrun. Thus, the two projects, Orbiter and Lambda, had a high degree of relationship for the amount of cost growth caused by inadequate funding. Shown in Figure 5-8 is a comparison of the impact of changes, inadequate funding, and overrun on the Orbiter project compared with project Lambda. As stated in the preceding paragraph the amount of cost growth attributable to changes is approximately the same relationship to total cost growth' amounting to 25.1 percent and 26.7 percent for the Orbiter project and Lambda project, respectively. However, there is a significant difference between the two projects in the amount of cost growth due to changes. For example, the Lambda project experienced cost growth due to changes at twice the rate of the Orbiter project while the overrun rate for the Lambda project was less than one-third that of the Orbiter project. A further comparison was made of the Lambda and Orbiter changes by dollar category. Figure 5-9 depicts the changes for the two projects by four separate ranges. As can be observed, 45 percent of the Lambda project changes were in the "no price adjustment" category, while 35 percent of the Orbiter project changes were in that category. The distinct difference is that the Orbiter contract had a $100,000 change threshold, while the Lambda contract did not have a threshold. Thus, it appears that a substantial number of the

PAGE 253

Category 238 Figure 5-8 COMPARISON OF COST GROWTH ORBITER PROJECT VS LAMBDA PROJECT 1 Orbiter Project 2 Project Lambda Government Directed Changes 32.5% 60.7% Impact of Inadequate Funding 25.1 26.7 Overrun 42.4 12.6 Totals 100.0% 100.0% SOURCES: 1 2 Prepared by Researcher Unpublished doctoral dissertation "An Analysis of Cost Estimate Growth on a Complex Development Project," Robert McGlashan, Jr., University of Texas at Austin, 1969, pp. 47-51.

PAGE 254

Range Figure 5-9 COMPARISON OF CONTRACT CHANGES WITH PRICE INCREASES BY DOLLAR CATEGORY ORBITER PROJECT VS LAMBDA PROJECT (MILLIONS OF DOLLARS) 239 Orbiter Project 1 LAMBDA PROJECT2 No. of Dollar No. of Dollar Value Changes Value No Price Adjustment 493 $ 0 358 $ 0 $0 $99,999 298 13.7 219 6.9 $100,000 $499,000 391 91.0 110 26.8 $500,000 and over 232 580.9 110 787.8 Totals SOURCES: 1,414 $685.6 1office of Shuttle Procurement Johnson Space Center 797 $821.5 National Aeronautics and Space Administration 2 Unpublished doctoral dissertation "An Analysis of Cost Estimate Growth on a Complex Development Project," Robert McGlashan, Jr., University of Texas at Austin, 1969, pp. 47-51.

PAGE 255

240 Lambda changes could have been for minor changes to the contract specifications of the Lambda spacecraft, although the Lambda study does not indicate the reason for the "no dollar value" changes. In the under $100,000 range, the Orbiter project had 21 percent of the number of changes in that category while the Lambda project experienced 27 percent in that range; thus, an unusually close correlation. For the changes over $500,000, in the Lambda project ended up with approximately 14 percent falling in that category while the Orbiter contract had slightly over 16 percent in that range. Once again, there was a high degree of similarity between the two for contract changes in the over $500,000 range. One last comparison of the CCA's on the Orbiter and Lambda projects is the extent of CCA's which decreased contract value. While the Orbiter project experienced 54 CCA's valued at $105 million, the Lambda project had a total of 39 CCA's that decreased 6 contract value by $7.8 million. Even though the percent of the number of CCA's in relation to the total number of CCA's were very similar, 3.7 percent for the Orbiter and 4.7 percent for the Lambda, the dollar value of the differences as stated above, was quite significant. While Figure 5-3 gives visibility into the categories of changes with price decreases for the Orbiter project, such data were not available for the Lambda project.

PAGE 256

241 Summary Contract changes are inevitable for major R&D projects. This is due, in large part, to the fact that major R&D contracts are directed toward objectives for which the work cannot be precisely described in advance of awarding initial contracts. It is not possible nor practical to place estimates in the initial contract to cover all unknowns. This chapter analyzed the significance of the contract changes on the Orbiter DDT&E contract. Changes amounted to $581 million, constituting approximately one-third of the total cost growth of the contract. Overall, those changes amounted to 13 percent of total contract cost. A significant feature of the contractual arrangement was that contract changes with increases or decreases to contract value of $100,000 value or less did not require negotiation and the contract value was not adjusted. This contractual threshold was very meaningful in terms of efficiency and productivity, for the contractor as well as the government, in administering the contract. An analysis was made of all changes over $5 million, with the finding that 25 contract changes, or less than 2 percent of the total changes, had a value of $305 million or 45 percent of all changes with price increases. The comparison of the significance of Orbiter DDT&E changes with another major spacecraft development project, and revealed significant relationships. This analysis provided for a further comparison of the Orbiter changes evaluation.

PAGE 257

NOTES -CHAPTER V 1 National Aeronautics and Space Administration, Procurement Regulation, NHB 5100.2C., (Washington, D.C.: U.S. Government Printing Office, 1981), Part 7.401-1). 2 Robert McGlashan, Jr., "An Analysis of Cost Estimate Growth on a Complex Development Project," (Ph.D. Dissertation, University of Texas at Austin, 1969). 3 Ibid., pp 42-8. 4Ibid. 5rbid. 6Ibid.

PAGE 258

CHAPTER VI FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS This chapter presents the summary of the findings of this study, the conclusions, and recommendations for further research. A summary of procurement reform over the past 25 years is presented followed by the research findings, conclusions, and recommendations for additional study. Procurement reform for ma]or defense and aerospace systems has been suggested by nearly everr administration for several de-cades. In each decade the Defense Department took the lead in procurement reform. During the 1960's, cost growth in weapons systems was a significant concern by the Department of Defense (DOD) and the Congress. In the 1960's Secretary of Defense McNamara embarked on procurement reform by: moving DOD toward more incentive contracting; instilling higher degree of formality in contractor selections; improving methods of cost estimating and cost control; introducing the total package procurement concept; and, emphasizing the program evaluation review technique method of program control. Still during the 1960's, unit cost of weapons systems increased dramatically, in part due to problems during the 1960's which were influenced by the Vietnam conflict. A major acquisition policy change by NASA during the 1960's, consistent with DOD, included the movement toward incentive contracts, with incentives on cost,

PAGE 259

244 schedule, and performance. Changes in contract types within the DOD and NASA did not bring about the real answer to cost growth of major R&D systems. The leadership of procurement reform of the 1970's was taken by then Deputy Secretary of Defense David Packard. Secretary Packard's most significant emphasis was the need for early hardware testing. Another major Packard initiative was competitive development which entailed two parallel contracts during the develop-ment phase of major projects. A major acquisition policy change by NASA during the 1970's, as NASA entered the Shuttle Program, was movement toward the cost-plus-award-fee (CPAF) type of contract for major development and programmatic contracts. Finally, procurement reform in the early part of the 1980's was led by Deputy Secretary of Defense Frank Carlucci. Carlucci emphasized reducing acquisition cost of weapon systems, improving the efficiency of the acquisition process, and reducing acquisition lead time. Two acquisition policy changes NASA made during the early 1980's included the movement toward an increased use of incentive type contracts, and a movement toward consolidation of several prime contracts into single contracts with the objective of increasing efficiency and reducing operational cost of the Shuttle. From the period 1960 to 1984, few really substantial changes were made in the statutory and regulatory framework of federal procurement. Authorized by the Congress in 1969, the Commission on Government Procurement (COGP) embarked upon a study of the entire federal procurement process. This was the first major review of the

PAGE 260

245 federal procurement process, chartered by Congress, since the Hoover Commissions of 1949 and 1955. As a result of the COGP's recommendation, Congress enacted the Office of Federal Procurement Policy (OFPP) Act in 1974. A major recommendation of OFPP was the issuance of a uniform Federal Procurement Regulation. On April 1, 1984, ten years after creation of OFPP, the Federal Acquisition Regulation came into being for use by all federal agencies replacing the NASA Procurement Regulation, the Defense Acquisition Regulation, and the General Service Administration's Federal Procurement Regulation. It is hoped that significant efficiency will result within the federal government and industry as a result of implementation of the Federal Acquisition Regulation. While there are currently over 4,000 legislative provisions that affect the federal procurement process, the Armed Services Procurement Act of 1947 and the Federal Property and Administrative Act of 1949 still remain as the basic statutory framework of government procurement. Procurement reform impacted on NASA as well over the period 1960 to 1984. Previous acquisition experienced by NASA and the Defense Department were instrumental in developing the acquisition strategy for the Shuttle Program. Procurement planning for the Space Shuttle system resided with the Johnson Space Center, the lead Center for the Shuttle Program. Based on extensive feasibility studies of the Space Shuttle system conducted during the late 1960's, NASA issued the request for proposal (RFP) to industry for preliminary design definition studies

PAGE 261

246 in 1970. After the completion of five parallel definition contracts, NASA issued the RFP in March 1972 for the Orbiter project and systems integration phase of the Shuttle.Program. Four proposals were received and the selected contractor was awarded a cost-plus-award-fee contract in August 1972. Contracted for in two major increments, it was 1975 before the total Orbiter design, development, test and evaluation (DDT&E) effort was under contract. The DDT&E effort was completed in 1983 following four highly successful orbital flights. The Marshall Space Flight Center in Huntsville, Alabama had procurement responsibility for their project elements, i.e., the Space Shuttle main engines, external tank, and solid rocket boosters; however, the JSC Shuttle Program Managers participated in Marshall's procurement planning activity. Participation by NASA Headquarters was rather extensive in the source evaluation and selection process of all the major Shuttle elements. The next section presents the major findings of this study. Summary :of Findings Award Fee Contracting Professor Raymond G. Hunt has conducted the most extensive research in award fee contracting. Hunt's work, concentrating on the Department of Defense (DOD) acquisitions in award fee contracting, resulted in the development of what he termed "hallmarks" of award 1 fee contracting. The more significant hallmarks included that award

PAGE 262

247 fee contracting: encouraged government-contractor cooperation; allows for active role of government managers; stimulates formal and informal communication; is flexible and allows for human judgment; simplifies contractual provisions; and, recognizes variability of motivations. Hunt concluded that in R&D acquisitions, the government needs a strategy that can adapt to the uncertainty in R&D procurements, help minimize uncertainty and protect the government's interests by allowing government participation in management of the 2 acquisition process. In the acquisition of a major research and development R&D program, it is important that thecontract provide for the flexibility necessary to permit the government to have informal and formal relationships with the contractor. A team relationship is necessary in order to pursue optimum solutions and designs to the many issues confronting the contractor and the government. It is essential to have timely feedback from the government to the contractor while the work is underway. An award fee arrangement provides the necessary contractual flexibilities to permit the government and the contractor to have a relationship that provides motivation for the contractor to pursue the government's objectives, while at the same time allowing the government to participate in major decision-making activities with the contractor. In view of the fact that most major R&D projects, like the Shuttle Orbiter development effort, are of long duration, and high

PAGE 263

248 .risk technical achievements required to meet the government's goals, it is essential to have a contractual arrangement that will permit the government to provide motivation to the contractor to achieve the desired results. A method of contracting that permits the government to shift areas of contractual emphasis and provide for profits to accomplish those goals is needed. The cost-plus-award-fee (CPAF) type of contract provides that needed flexibility to accomplish those objectives. Flexibility is provided by the CPAF type of contract to provide motivation for the contractor management to direct the firms efforts to strive to achieve the government's objective. Therefore, the flexibility in the CPAF type of contract is an excellent means to provide the necessary motivation, through awards of profit or fee, to perform at an optimum level. Based on the survey results of this study, the "flexibility" provided by the CPAF contract scored as a major strength of award fee contracting. Award fee contracting allows for subjectivity in determining the amount of ree earned by a contractor. This subjectivity is, perhaps, the single most important: attribute of the CPAF type of contract. For example, the subjectivity in the CPAF contracting for the Orbiter provided for the government to evaluate the total contract performance and make meaningful and logical tradeoffs before penalizing the contractor from a fee standpoint. This means that, for example, a near-term expenditure that was cost effective over the long-term could be so recognized and the contractor not penalized

PAGE 264

249 from a fee standpoint. Also, technical and cost tradeoffs could be considered during the'fee determination process. These types of tradeoffs and the subjectivity afforded by the CPAF type of contract would not be possible under the cost-plus-incentive-fee type of contract. In utilizing the CPAF type of contract for major projects such as the Shuttle Orbiter project, care must be taken so as not to allow the subjectivity and flexibilities afforded by the CPAF type contract to create weaknesses or contracting "pitfalls". Some of the weaknesses perceived by the government evaluators of the Orbiter DDT&E contract were that individual evaluators in the survey used in this study became biased, and that the evaluations were too subjective. Evidence was also presented that it was hard to keep the fee areas of emphasis in proper balance with the total project; specifically, this study found that the government did not promptly shift emphasis in evaluation criteria. Additionally, it was perceived by government evaluators in the survey that the contractor was motivated to satisfy the government's managers' desires and not necessarily the best approach deemed by the contractor. While CPAF contracts can degenerate to this type of contractor motivation, extreme care must be taken by the government to minimize this type of situation. Data collected and analyzed by the writer in conducting this study provided abundant and significant evidence supporting the use of the CPAF type of contract for the Orbiter DDT&E activities.

PAGE 265

250 Significant findings of the study included that: 84 percent of survey participants believed that the award fee type of contract contributed to the technical success of the project; 75 percent of the respondents believed that the award fee contract contributed to minimizing cost growth; 91 percent of the survey participants found the contractor responsive under the award fee contract; and, 88 percent of the respondents believed that the contractor focused on the major problems of the Orbiter project. Thus, this evidence leaves little doubt that the award fee type of contract was the most appropriate type for the Orbiter project. Cost Growth of the Orbiter Project Studies by the Rand Corporation found that the major causes of cost growth for major weapons in the 1960's to be scope changes, technical uncertainty, and cost estimating errors, with scope changes (increased system performance) constituting the largest category. Major cost growth causes in the 1970's, according to the Rand reports, were engineering changes, schedule changes, estimating errors, and inflation. Inadequate funding was the most frequent root cause for schedule slippage, and the impacts to program cost due to schedule slippage were compounded due to the many years of double digit inflation. Programs with hardware competition before full scale development were found to have significantly lower cost growth. Based on the GAO reports reviewed, major causes of cost growth in the 1960's and 1970's were demands for greater capability

PAGE 266

251 of new systems, increased system complexity in order to obtain greater capability, inflation, and inaccurate estimating. In summary, the findings by the GAO were largely consistent with Rand studies. The GAO studies found that funding instability, technical advancement, and technical complexity occurred in over 50 percent of the major acquisitions. Peck and Scherer, based on their extensive research in R&D acquisition, found that the most significant cause of cost growth was 3 technical uncertainty. They also observed that emphasis seemed to be placed on achieving technical excellence. Peck and Scherer also found that competitive optimism at the beginning of a development program encourages innovation and firms tended to submit excessively optimistic projections of a program's technical expectations, sched-4 ules, and cost estimates. Optimism was indirectly encouraged due to the general lack of any financial penalties. Data were presented in this paper that indicated the final Orbiter DDT&E expenditures were less than the commitments to the Congress and OMB. Specifically, the total Orbiter DDT&E co.st was $4,942 million compared with the OMB commitment of $5,142 million. This was a rather a remarkable achievement by NASA and its contrac-tors, considering the cost growth experiences of other agencies. Covering a development period of 10 years, the Orbiter project is certaintly considered one of the most challenging R&D projects every undertaken. An analysis of the contract .cost growth indicates that

PAGE 267

252 57 percent was caused by the government. Government-directed changes, namely for product improvements, increased systems capability, reliability, and safety, amounted to 32 percent of the cost growth. Impacts resulting from inadequate and untimely funding amounted to 25 percent of the cost growth. Remaining cost growth, amounting to 43 percent of the total cost growth, was attributed to overrun by the contractor. Total cost growth for the Orbiter development effort, government-caused and contractor caused, was 40 percent of the total project cost. Average total cost growth for the other major R&D projects reviewed in this study was 50 percent. Perceptions by the government personnel participating in the survey conducted during this study were that the primecause of cost growth for the Orbiter DDT&E project was inadequate funding. As indicated, inadequate funding did actually account for 25 percent of the cost growth which could have been avoided with more funding certainty. Representing the second most perceived reason for cost growth was the technical uncertainty associated with the development of the Orbiter spacecraft. Survey respondents indicated that they believed technical advancement, product improvement, and other governmentdirected changes significantly contributed to the Orbiter's cost growth. As stated earlier in this summary section, governmentdirected changes did account for 32 percent of the cost growth; therefore, the perceptions by survey participants were, in fact,

PAGE 268

253 accurate. Survey participants also believed that inflation and over optimistic program planning contributed to the cost growth of the Orbiter development effort. Inflation did, in fact, contribute significantly to the Orbiter cost growth. Total Orbiter DDT&E project cost amounted to $4,420 million in real year dollars which equated to approximately $2,803 million in 1971 dollars. Thus, the difference between those two amounts is the inflationary impact of the dollar during the contract period of performance. The data presented in this study indicated that unanticipated inflation, i.e., that inflationary impact to the project not priced into the contract, was in the $70 to $80 million range. While the contractor was reimbursed the unanticipated inflation as part of the contract cost, the contractor received no additional fee or profit. There were a number of ways suggested that would have served to reduce the cost growth of the Orbiter DDT&E project. Stability of funds was suggested by survey participants to be the number one way NASA could have reduced the cost growth. Data presented support this contention; the cost impact resulting from inadequate and untimely funding was almost equal to the dollar value of all contract changes. Since many of the changes were mandatory, the funding impact was no doubt the single most "controllable" significant contributor to cost growth caused by the government. Other ways identified where NASA could have reduced the cost growth of the Orbiter DDT&E project were: tighter control.of

PAGE 269

contract changes; qualifying development system before hardware production; building a less complicated spacecraft; building all spacecraft concurrently; purchasing spare parts concurrent with spacecraft production; and use of a cost incentive contract. 254 In summary, the study results indicated that the most significant causes for cost growth of the Orbiter DDT&E project were contract changes, instability of funds, and overrun. Still notwithstanding the cost growth of 40 percent, final project cost outcome was less than the original commitments to the Congress and the Office of Management and Budget. Contract Changes A survey of the literature indicated that contract changes are normally included in the cost growth literature. As noted from the preceding paragraphs, contract changes were found to contribute significantly to cost growth in major R&D acquisitions. It was clear from reviewing the literature that a primary purpose of R&D programs was to advance scientific and technical knowledge and apply that knowledge to the degree necessary to achieve agency and national goals. Thus, it is not unexpected to find in major R&D projects for defense and aerospace systems, large numbers of contract changes that add significantly to program costs. Scherer's case studies as related in Chapter 2, revealed that contract changes to enhance weapon system quality and reducing development schedules had a considerable higher priority by contractors and DOD officials than

PAGE 270

255 minimizing development cost. Fox's extensive research in R&D procurement revealed that it was common practice in aircraft spacecraft R&D projects to upgrade the system to the latest state-of-the-art components.5 Fox relates a major problem in controlling the cost of changes which is the fact that contract changes are issued on a sole-source basis to the firm who holds the prime contract.6 There are typically little incentives or motivation for a contractor to resist changes; but rather quite the contrary, since contract changes equate to additional sales and profit for the contractor. It is clear from reviewing the literature that contract change management rests primarily with the government. Contractors recommend and seek changes, mandatory as well as non-mandatory changes. There is clear evidence from the literature that change management and cost control can be significantly enhanced by controlling only a few changes that typically constitute the majority of the cost of changes. Sound change management is essential to proper cost management of major projects. When major R&D projects for defense and aerospace systems are undertaken, the anticipated work usually can only be described in terms of performance requirements, goals, and objectives. It is not possible, nor practical, to price estimates for anticipated changes into the initial contract. Unusual large contingencies, if priced into the contract for changes, could result in profit windfalls to

PAGE 271

256 the contractor. Underestimating the value of changes could result in the contractor .absorbing large amounts of work without fee or profit. Thus, changes are typically handled on a case by case basis. Contract changes for the Orbiter DDT&E project amounted to $581 million and constituted approximately one-third of the total cost growth of the contract. Those changes amounted to 13 percent of total contract cost. There were a total of 1,468 contract change authorizations (CCA's) issued under the Orbiter DDT&E contract, of which 1,414 increased contract cost while the remainder resulted in decreases to contract cost. A total of 232 CCA's or 16 percent of the total number of changes, represented 85 percent of the dollar value of all the changes. A significant feature of the contractual agreement included a threshold whereby changes with increases or decreases to contract value of $100,000 or less did not require negotiation and the contract value was not adjusted. The contractor was reimbursed only for the actual cost of those changes and did not receive additional fee. For those changes with decreases, the government did not obtain a reduction in fee. There were 493 CCA's, or one-third of the total CCA's, that fell within the $100,000 threshold. Therefore, the contractual changes threshold was very meaningful in terms of efficiency and productivity for both, the government and the contractor, in administering the contract. It was found in this study of Orbiter contract changes that if the threshold for equitable adjustment had been placed at $500,000 rather than $100,000, such a

PAGE 272

257 feature would have-covered approximately 84 percent of the total volume of CCA's, representing 16 percent of the cost of all changes. Thus, a major finding of this study is that changes thresholds for major R&D projects should be established at high levels. Contract changes issued to the Orbiter DDT&E contract were evaluated to determine the time phasing of issuance of the changes in relation to the project schedule. Although the contract period of performance was for 10 years, the peak expenditure years were 1975-1979. This study indicated 'that the major cost impact changes were issued during the 1975-1978 timeframe. First flight of the Orbiter spacecraft occurred in April 1981. As a result of an analysis made of all changes over $5 million, it was found that 25 CCA's, or less than 2 percent of the total changes, had a value of $305 million or 45 percent of all CCA's with price increases. Over one-third of these changes, in terms of volume and dollar value, was due to additional testing requirements. Major categories of CCA's in the over $5 million range included additional Shuttle systems integration effort to support the total Shuttle system, added vehicle capability, additional spacecraft modifications to meet original vehicle requirements, changes for cost effective maintenance, and additional training and simulation requirements. Thus, a large portion of the dollar value of the CCA's in the over $5 million area were not hardware-related, but significant additional effort for testing, and systems engineering effort related to flight operations.

PAGE 273

258 A comparison was made of the significance of the Orbiter DDT&E changes with major spacecraft deve.lopment project, called, for research purposes, "Project Lambda." It was found that only 20 CCA's out of approximately 1,000, accounted for 81 percent of Project Lambda growth, while 25 Orbiter CCA's represented 45 percent of the Orbiter project's cost increase. Thus, in both projects a relatively small number of CCA's had a major impact on project cost. Another-significant observation between Project Lambda and the Orbiter DDT&E project was that the cost impact due to inadequate funding was almost identical from a relative standpoint. For Project Lambda, the funding constituted 26.7 percent of the total cost increase while the Orbiter's cost increase due to inadequate funding amounted to 25.1 percent of total cost growth. In conclusion, it is highly critical that contract changes be managed properly for major R&D projects. As indicated earlier, contract changes are inevitable for R&D undertakings and, although costly, typically result in better products. It is believed that a high change threshold significantly contributes to better change management. Conclusions This study was an extensive review and evaluation of the procurement management of the Shuttle Orbiter design, development, and evaluation (DDT&E) project. Specifically, an analysis was

PAGE 274

259 performed of the type of contract, cost growth, and contract changes of the Orbiter project. Included was an extensive review of the literature regarding the use of award fee contracting for major research and development (R&D) projects, cost growth of R&D projects, and the significance of contract changes for R&D projects. This research endeavor has produced several conclusions which are enumerated on in the following paragraphs. 1. Cost-plus-award-fee contracts are appropriate for major R&D projects. A major conclusion of this study is that the CPAF type of contract appears to be the best form of contract for major R&D projects. Providing an excellent means to motivate the contractor toward the government's objectives, the CPAF contract is an instrument that allows for shifting of award fee emphasis as the work progresses and the situation dictates. Award fee contracting fosters a high degree of frequent communication during performance of the contract work. In R&D activities, it is essential to have a team relationship between the government and the contractor. Award fee arrangements are designed to not only provide for, but encourage a cooperative, team relationship. Award fee contracting allows for a large degree of subjectivity in determining the amount of award fee earned. Unlike an objective incentive arrangement, the award fee contract provides for the government to make discretionary judgments in regard to the tradeoff's between technical and cost performance in determining the fee earned. Award fee contracting also forces more

PAGE 275

260 government management involvement in the program. Data presented in this study, including the literature review and the survey conducted, strongly supported the use of the award fee type of contract for major R&D projects. 2. Some form of cost incentive should be used for hardware production beyond the research and development phase. Based on the survey conducted in this study, almost 70 percent of the participants stated that an objective cost incentive should be used for hardware acquisitions beyond the development stage of acquisition. Survey respondents strongly believed that some objective form of motivation with monetary rewards should be employed in the acquisition of production hardware. 3. A major weakness of award fee contracting is evaluators are biased. A very significant .perceived weakness of the award fee procedure of the Orbiter project was that a significant number of the survey participants believed that award fee evaluators are biased. Evaluators believe that they are grading themselves. This points to the need for the government to examine critically the appointments of award fee evaluators to ensure that any bias is minimized. 4. Inadequate and untimely funding by the government significantly contributes to the cost growth of major R&D projects. Based on the evaluations of the Orbiter DDT&E project cost growth, a review of the literature, and from reviewing another major spacecraft

PAGE 276

261 project, the evidence is clear that untimely funding and inadequate funding by the government have major impacts to the cost of R&D projects. For the Orbiter project, inadequate and untimely funding amounted to $450 million additional program cost and represented 25 percent of the total cost growth. A review of spacecraft project Lambda revealed that inadequate funding increased that project's cost by 27 percent. Federal agencies should be more aggressive in pursu-ing full funding requirements from the Office of Management and Budget (OMB) and the Congress to avoid or minimize the severe cost penalty that inadequate funding causes. Inadequate funding is clearly a controllable cause of cost growth, and enough evidence exists that the Congress and OMB should be convinced of the import-ance of timely project funding. 5. Contract changes significantly contributes to cost growth of major R&D projects. A conclusion of this.study is that contract changes have a significant effect on the cost of major R&D activi-ties. Contract changes are typically for technical advancement, product improvement, and the desire by the government to I the most recent technology. Contractors generally recommend contract I changes since "changes" represent.additional sales, and this should be recognized by the government. A conclusion regarding the worth or value of contract changes to a given project could not be made. Further, much of the cost growth data reviewed did not distinguish between the various types of

PAGE 277

262 cost growth. However, this research found that the contract changes for the Orbiter amounted to approximately one-third of the total cost growth compared to slightly less'than two-thirds for project Lambda. Thus, the only real conclusion that could be drawn was that contract changes significantly contributed to the cost growth for both projects, and therefore government project managers should judiciously make cost and technical tradeoff's for all contract changes, especially those that have a high probability of significantly increasing project cost. 6. Technical uncertainty and development problems are major contributors to cost growth of R&D projects. Data from the survey conducted for this research effort indicated that technical uncertainty and development problews encountered were perceived to be major causes of cost growth for the Orbiter project. Those same causes of cost growth for the Orbiter were also found to be major causes of cost growth for Project Lambda. The literature review verified that technical uncertainty and development risks were major contributors to cost growth of major R&D projects. While it is impossible to completely eliminate technical uncertainty and development problems, there appears to be several factors that should be considered by the government and contractors. These factors are elaborated on in the following paragraphs. Optimistic program plans and schedules are generally a good attribute of project objectives; however, the plans and schedules

PAGE 278

263 must be realistic and achievable. Heavy expenditures of resources when it is known that'major milestones cannot be achieved can be very costly. Another consideration to minimize technical uncertainty indicated in the literature and mentioned by a significant number of survey participants is to qualify development systems before hardware I production. In essence, this would require freezing the design of the systems and not allow any more changes. Modifications to systems in production to incorporate major design changes are very costly and should be avoided unless absolutely mandatory. Thus, project man-agers should carefully evaluate project status prior to proceeding with hardware production. Finally, technical uncertainty is related close to technical advancement and product improvement. In fact, it is sometimes very difficult to distinguish between those two causes of cost growth. When the government proceeds to incorporate product improvements development problems often occur. 7. Contractors have a tendency to underestimate the real costs of major R&D projects. Low initial cost estimates were confirmed by the literature review to be a major cause of cost growth. Under estimating initial project costs was also mentioned by several survey respondents. This reason for cost growth was a conclusion in the Project Lambda study. This is a very difficult issue to deal with. Cost growth is expected for major R&D projects and it is not

PAGE 279

264 normally desired by the government to price into the initial contract large sums of money for development contingencies and technical advancement. Further, government agencies and contractors are, to a large degree, forced to structure programs in line with budgetary limitations from the Congress and OMB. 8. Contract changes thresholds should be established at high dollar levels. Government project managers and contracting officers should aggressively seek to establish contract changes thresholds in all major contracts. While the dollar level of the threshold is negotiable with the contractor, the level should be established to cover at least 30 percent of the anticipated volume of changes, and preferably, at a level that would cover approximately 50 percent of the anticipated changes. Contract changes thresholds provide for cost effectiveness and improved productivity. When thresholds are used the contractor is not motivated to recommend changes within the threshold amount since the firm would not receive additional profit or fee for those types of changes. Since many of the changes in major contracts are initiated by the contractor, the contractor would be motivated to recommend only those changes that were clearly outside of the threshold amount. Thus, the government would have fewer changes to deal with and it is presumed that a better cost and technical tradeoff could be done in deciding whether or not to approve the change. Another major cost savings and productivity enhancement of

PAGE 280

265 high dollar changes threshold is avoidance of the cost that would be incurred by the contractor in preparing a detailed cost proposal. In addition, the government would ayoid the expense associated with proposal evaluation and negotiation. A changes threshold at a level sufficient to cover a significant number of the changes should require less staffing by the government and the contractor for contract administration. The findings and conclusions drawn from this study have led to specific recommendations for future research. Recommendations for Future Research As with the conduct of any research effort, there are other areas of worthy inquiry that must be passed because of time and space limitations. This study identified several areas that seem worthy of further research. It is believed that an excellent research project would be to duplicate this study on the research and development phase of another major spacecraft or weapon system project. Results from another comparable study would enable further comparison and testing of the findings from this study. Another worthwhile research project would be to conduct a critical evaluation of the contract type, cost growth, and contract changes of the Orbiter production contract. A comparison of the results of such a study, with the findings of the development phase, may produce findings regarding certain actions taken during the

PAGE 281

266 development phase indicate what would lead to better programmatic decision-making as well as cost avoidances during the production phase. Due to the significant project cost increases that is caused by untimely and inadequate funding of major projects, an indepth study should be made sampling a wide variety of defense and space systems. Insufficient and delayed funding for major contracts appears to be a universal problem and is costly to the taxpayers. This research project confirmed that funding problems caused significant cost increases to the Orbiter development project. Findings from the literature review support the significant impact of the funding problems on other projects. An indepth study of the types of contract changes and the rationale for approving the changes on the Orbiter project should be an excellent study. Included should be an analysis of the mandatory and non-mandatory changes, and an evaluation of the cost and technical tradeoff's in the decision-making process to approve the change. An assessment could also be made of the changes that were recommended to the project manager for approval by the contractor and also by government personnel that were disapproved. The results of such a study could be of significant value in the change management of future major acquisitions of defense and space systems by the government. While this study evaluated the award fee from the government's perspective, a worthwhile research project would be to conduct

PAGE 282

267 an indepth study of the perceived effectiveness of the award fee type of contract from the contractor': s perspective. Included should be an assessment of the strengths, weaknesses and motivation implications of the award fee type of contract. An expansion of the study to include the use of the award fee contract as a decision-tool for management would add an excellent dimension to the study. It is hoped that this study will contribute toward an increased understanding of the management of future NASA R&D procurements. It is also hoped that the results of this research will further a better understanding of the relationships between contract type and management techniques, and more cost effective government procurement as NASA moves into the procurement activity for the Space Station Program.

PAGE 283

NOTES -CHAPTER VI 1 : Raymond G. Hunt, "Use of the Award Fee in Air Force System and Subsystem Acquisition," Final Report to Air Force Business Management Research Center, Wright-Patterson AFB, Ohio, March 1980, p. 21. 2Ibid. 3Merton J. Peck and Frederic M. Scherer, The Weapons Acquisition Process: An Economic Analysis (Boston: Harvard University Press, 1962), pp. 593-94. 4Frederic M. Scherer, The Weapons Acquisition Process: Economic Incentives (Boston: Harvard University Process, 1962), pp. 22-9. 5 J. Ronald Fox, Arming America: How the U.S. Buys Weapons (Harvard University Press, 1974) pp. 81-2. 6Ibid.

PAGE 284

BIBLIOGRAPHY Publ .ished Works Armed Services Procurement Act. U.S. Code, vol. 5, sees. 219b, 412b, vol. 41, sees. 151-161 (1948). Babbie, Earl R. The Practice of Social Research. 2nd ed. Belmont, Calif.: Wadsworth Publishing Co., Inc. 1979. Demong, Richard F. and Strayer, Daniel E. "The Underlying Theory of Incentive Contracting." Defense Management Journal 17 (1st Quarter 1981): 42-51. Dews, Edmund and Smith, Giles K. Acquisition Policy Effectiveness: Department of Defense Experience in the 1970's (R-2516-DRE). Santa Monica, Calif., The Rand Corporation, 1979. Federal Property and Administrative Services Act. U.S. Code, vol. 5, sec. 630; vol. 40, sees. 471-475; vol. 391 (1949). Fisher, Irving N. A Reappraisal of Incentive Contracting Experience (RM-5700-RR). Santa Monica, Calif.: The Rand Corporation, 1968. Fisher, Irving N. Improving the Effectiveness of Incentive Contracting (P-3870). Santa Monica, Calif.: The Rand Corporation, 1968. Fox, J. Ronald. Arming America: How the U.S. Buys Weapons. Boston: Harvard University Press, 1974. Harman, Alvin J. A Methodology for Cost Factor Comparison and Prediction (RM-6269). Santa Monica, Calif.: The Rand Corporation, 1970. I Hunt, Raymond G. "Concepts of Federal Procurement: The Award Fee Approach." Defense Management Journal (2nd Quarter, 1982): 8-17. Hunt, Raymond G. "Contractor Responses to Award Fee Contracts." National Contract Management Journal 15, (Winter 1982): 84-90.

PAGE 285

Hunt, Raymond G. "Cross-Purpose in the Federal Contract Procurement System: Military R&D and Beyond." Public Administration Review 3, (May/June 1984): 247-256. Hunt, \ Raymond G. "R&D Management and Award Fee Contracting." The Journal of the Society of Research Administration 6, (Summer 1974): 33-40. Kerlinger, Fred N. Foundations of Behavioral Research. 2nd ed., New York: Holt, Rinehart, and Winston, Inc., 1973. Lorette, Richard J. "Major Acquisition Problems, Policy and Research." National Contract Management Journal 10, (Winter 1976-77): 1-15. Marshall, A. W. and Meckling, w .. H. Predictability of the Costs, Time, and Success of Development (P-1821). Santa Monica, Calif.: The Rand Corporation, 1969. Meneely, Frank T. "Determining the Appropriate Contract Type." Concepts, The Journal of Defense Systems Acquisition Management 5, (Summer 1982): 44-49. Nash, Ralph C., Jr. Government Contract Changes. Washington, D.C.: Federal Publications, Inc., 1975. Nash, Ralph C. and Cibinic, John, Jr. Federal Procurement Law. 3rd ed. Washington ,D.C. : The George University Press, 1977. National Aeronautics and Space Administration. Annual Report Spinoff 1981. Washington, D.C.: U.S. Government Printing Office, April 1981. National Aeronautics and Space Administration. Cost-Plus Award-Fee Contracting (NBH 5104.4). Washington, D.C.: U.S. Government Printing Office, 1967. National Aeronautics and Space. Administration. Major System Acquisition. (Management Instruction 7100.14A). April 19, 1978. 270

PAGE 286

i National Aeronautics and Space Administration. Procurement Regulation. (NHB 5100.2C). Washington,D.C.: U.S. Government Printing Office, 1981. National Aeronautics and Space Administration. Space Shuttle (SP-407). Washington, D.C.: NASA Headquarters, 1976. National Security Industrial Association. Defense Acquisition Study. Washington, D.C.: n. p., 1970. Nelson, Richard R. and Peck, Merton J. Technology, Economic Growth, and Public Policy. Washington, D.C.: The Brookings Institute, 1967. Novick, David. Are Cost Overruns a Military-IndustryComplex Speciality? (P-4311). Santa Monica, Calif.: The Rand Corporation, 1970. Novick, David. Costing Tomorrows Weapon System (RM-3170-PR). Santa Monica, Calif.: The Rand Corporation, 1962. Novick, David. The Cost of Advanced Weapons (P-2556). Santa Monica, Calif.: The Rand Corporation, 1962. Peck, Merton J., and Scherer, Frederick M. The Weapons Acquisition Process: An Economic Analysis. Boston: Harvard University Press, 1962. Perry, Robert L. System Acquisition Experience (RM-6072). Santa Monica, Calif.: The Rand Corporation, 1969. Perry, Robert L. Acquisition Strategies (R-733-PR). Santa Monica, Calif.: The Rand Corporation, 1971. Reece, James S. 11The Management Change: A Catchword or an. Opportunity.11 National Contract Management Journal 5. (Spring 1971): 127-128. Rule, Gordon W. and Cravens, James E. 11The Post and Future in Cost-Plus-Award-Fee Contracting." Defense Management Journal 5. (Winter 1968-69): 27-29. Scherer, Frederick M. The Weapons Acquisition Process: Incentives. Boston: Harvard University Press, 1962. Smith, G. A. "A-109: It's Purpose and Implications in Contracting Costs ot' Major System Acquisitions." National Contract Management Journal 14. (Winter 1980): 33-34. 271

PAGE 287

Stekler, Herman 0. The Structure and Performance of the Aerospace Industry. Los Angeles: University of California Press, 1965. "Systems Acquisition: How A-109 Can Help Process." Government Executive 9. 10-12. Shorten the (October 1977): Truth in Negotiations Act. U.S. :Code, "vol. 10, sec. 137 (1962). U.S. Commission on Government Procurement. Summary Report. Washington, D.C.: Government Printing Office, 1972. U.S. Congress. House. Report of the Special Panel on Defense Procurement Procedures of the Committee on Armed Services. "Weapons Acquisition Policy and Procedures: Curbing Cost Growth." 97th Cong., 1st sess., 1982. U.S. Congress. Senate. Report of the Committee on Governmental Affairs. S. Rept. 95-715, 95th Cong., 2d. sess., 1978. U.S. Council of Economic Advisers. Economic Indicators. Washington, J?..C.: U.S. Government Printing Office, June 1981. U.S. Department of Defense, General Services Administration, and National Aeronautics and Space Administration. Federal Acquisition Regulations. Washington, D.C.: U.S. Government Printing Office, 1984. U.S. Department of Defense. Major System Acquisitions. (DOD Directive 5000.1). 1971. U.S. Department of Defense. Office of the Deputy Secretary of Defense. Improving the Acquisition Process (Memorandum). Washington, D.C., April 30, 1981. U.S. General Accounting Office. Cost Growth in Major Weapon Systems (B-163058). Washington, D.C.: U.S. Government Printing Office, March 26, 1973. U.S. General Accounting Office. Defense Spending and Its Relationship to the Federal Budget (PLRD-83-80). Washington, D.C.: U.S. Government Printing Office, June 9, 1983. 272

PAGE 288

U.S. General Accounting Office. Financial Status of Major Acquisitions (MA.SAD-81-13). Washington, D.C.: u.s. Government Printing Office, March 20, 1981. u.s. General Accounting Office. Financial Status of Major Acquisitions (PSAD-77-62). Washington, D.C.: U.S. Government Printing Office, June 30, 1976. U.S. General Accounting Office . Government Contract Principles. 2nd ed., Washington, D.C.: u.s. Government Printing Office, 1978. U.S. General Accounting Office. Multiyear Authorizations for Research and Development (PAD-81-61). Washington, D.C.: U.S. Government Printing Office, June 3, 1981. U.S. General Accounting Office. Progress of Federal Procurement Reform Under Executive Order 12352 (PLRD-83-88). Washington, D.C.: U.S. Government Printing Office, June 17, 1983. U.S. General Accounting Office. Status of Major Acquisitions as of September 30, 1981: Better Reporting Essential to Controlling Cost Growth (MASAD-82-24). Washington, D.C.: u.s. Government Printing Office, April 33, 1982. U.S. General Accounting Office. Weapons Systems Overview: A Summary of Recent GAO Reports, Observations, and Recommendations (NSIAD-83-7). Washington, D.C.: U.S. Government Printing Office, September 30, 1983. U.S. Office of Management and Budget. Major System Acquisition (Circular A-109)., April 5, 1976. U.S. Office of Management and Budget. Proposal for a Uniform Federal Procurement System. Washington, D.C.: U.S. Government Printing Office, 1982. U.S. Presidential Executive Order 12352. Federal Procurement Reforms. March 17, 1982. "Why the Carlucci Initiatives Aren't Working." Government Executive 14 (August 1982): 30-32. 273

PAGE 289

Unpubl:ished Works Hunt, Raymond G. "Managing R&D in a Federal Setting." Paper presented to Academy of Management. Public Sector Division, New York, August 18, 1982. Hunt, Raymond G. "Use of the Award Fee in Air Force System and Subsystem Acquisition." Final Report. Wright-Patterson AFB, Ohio: Air Force Business Management Research Center, March 1980. Jenkins, Gwilyn Howard, Jr. "Decision Cricteria for Cost-Plus-Award-Fee Criteria." Masters thesis, Naval Post Graduate School, Monterey, Calif.: March 1979. McGlashan, Robert, Jr. "An Analysis of Cost Estimate Growth on a Complex Development Project." Ph.D. dissertation, University of Texas at Austin, 1969. 274

PAGE 290

APPENDIX A QUESTIONNAIRE FOR,GOVERNHENT PERSONNEL AWARD FEE CON'l'RACTING-

PAGE 291

Na11ona1 Aeronautcs and Space AdmlniSifahon N/\5/\ L,.,.,. B.Jotr.anii)ICiaC. Houston Texas 77058 TO: ftOH: Iurvey Participant IC4/I. I. J:uley .. rch 13. 1984 SUBJECT: laaearcb Project DD Orbiter T&E Award Fee .ad Coat Growth l aa cooductia& a raaaarcb project reaardina the perceived effectiYeneaa of the award fee t7Pe of cootract for the Orbiter DDT&! project. lD addition, I .. aaaeaainB the .. tn cauaea of coat arovth of the Orbiter DDT&I project. liDce you played a vital role in tbe avard fee activity and in the ..aaa .. ent of the contract, Jour of the attached queltionnaire vvuld a11Dificantly coatribute toward the value of thia reaearch &Ddeavor. lf you could apara approztaatalJ 20 a1Dute1 to complete the encloaure 1t vvuld be areatly appreciated. returD aelf addreaaed envelope ia eacloaed for your convenience. a etated on tbe aurvey 1D1tnaent, JOU do DOt bave to indicate your D&ae on your reaponae. lf you could return the completed quelt1onna1re by March 21 it vvuld be areatly appreciated. tb&Dkl.

PAGE 292

QU!STICIIRAIIE 1'01 CCJVEIIiM!NT PDSmiREL MlAID PEE CORTUCTIMG !he purpose of thia i1 to .. iD ,our perceptioaa TesardiDS_the use of the .ward fee type of coatract for the Space Shuttle Orbiter project. aDd to aolicit your baliefa of cbe u.in cauaea of coat arovth. hlect a aaber OD a acale of 1 to 5 that beat indicate ,our .-eDt. Uae the acale below for all atat .. enta 1 throush 8. lelpODU 1 StroDsly lfesative 2 suahtly aauve 3 8eit_her Poaitive aor 8eaaUve 4 lli&htly Poaitive 5 ltroDsly Poaitive 1. the daaree to which you believe the ... rd fee type of contract coatributed to the techDical aucceaa of the Orbiter Project. 2. !be dearee to which ,ou believe the .ward fee type of contract CODtributed to .tDiaiaiDa tbe' COlt arovth of the Orbiter Project. 3. the dearee to which your Telationahip with the coDtr8ctor were affected by the award fee type of coatract. 4. !he desree to which you foUDd the coetTactor reaponaive UDder the nard fee coatract. S. fbe dearee to which you believe that the .ward fee type of coatract affected tbe coatractor'a u.naa ... Dt atteDtion to .. jor iaauea and probl..a. 6. The dearee to which you believe that the ... rd fee criteria and areas of ..,halia provided the coDtractor TepraaeDted the kay areaa of coacern of IASA. 7. The dearee to which ,ou believe that the evaluation frequency va appropriate. 277 0 Don't In ow 8, !be dearee tO nich JVU believe tbat the .ward fee type Of CODtract affected deci1ion-..kiD8 between the Governaeat .ad the coatractor. (For 90 100 and 110 lelect I aaaber OD I acale of 0 to 10 (ahOVD belov) that beat 1Ddicatel ,our belief). 8oee 0 1 2 3 4 5 6 7 8 t 10 totally !be desree to nich you believe that tha evaluation criteria abould bave be.a objective. 10. !be de1ree to which you believe the nard f .. proce1a could have been t.proved for the Orbiter coatract. 11. !be de1ree to which ,ou believe the f .. deter.inatioD official (lDO) ... fully bafoned reaardtDa the periodic ard fH naluatioaa.

PAGE 293

12. Vbat do you coa.ider to be tbe .. jor atr.aatb of ..ard fee type of coatract for .. jor I6D acquiaitioaa? ; I 13. Vbat do you coaaider to be tbe .. jot ... tuee of award fee type of for .. jor acquia1t1ona7 14. From your perapective, tadicate What you believe to be the beat form of coutractius (cbooae froa the liat aivea) for the indicated type of aituatioua. Alao iDdicate dasree of prefereuee. l.iat of Type of coutract: lituaUou: A. Major I6D (DUE Pbaae) I. t.t.itad productiDD (l co 10 Diu) C:. Gperatioul npport of UD .,.,_. 1. CPFF (ca.plet1DII fonD) 2. CPFF (level-of-effort) 3. CPA7 (cpletiaa fonD) 4. CPAF (level-of-effort) 5. CPIF (caapletiou fonD) 6. Ptaed Price lDceutive 7. Pira Fixed Price !rpe of C:Outract (Select Gaber froa above lilt) lank De,ree of Prefereuce, uaiua a Scale of 278

PAGE 294

15. Pleaae bdicate uy otber c-.nu JOU wnld Ulr.e to aake naal'dia& t ue of Avud Pee t7Pe of cntract: for the Orbitar project. 16. Please aake any other appropriate reaarding tbe uae of avard fee type of contract for aajor R&D acquiaitioDa. 279

PAGE 295

coST GROVl'H 1. lhat do you eoa.ider to .. the .. jor cauaea of coat arovth for the Orbiter project! 2. Susaeat aay vaya you believe IASA could have reduced the cost OD the Orbiter project, 3. laperieDCe auaseata that fundift8, tecbDical aDcertainty/difficulty, aad teclmical often tbe .. iD cauaea of coat arovth for .. jor liD projecta. Pleaae rank theae cauaea on a acale'of 0 to 10, you aee thea affectiD& the coat arovtb of the Orbiter project. lODe 0 1 2 3 4 5 6 7 8 9 10 Totally a. !Dadequate or tDatable faDdiD& b. !echDical aDcartaiDtyldifficulty c. reclmical advUlcaeDt 280

PAGE 296

1. Do you have aay ausaeation. or co..enta cGDceraiDs IASA'a acqa!aition policies that, if cban1ed. would foater 1Dcr ... ed efficieDey aad "better vaya" of doina buainesa. a.apondent Data 1. Position (a) Project Kanas .. ent; (b) Procur.-ent Management/ ContractiD& Officer; (c) Proar.m/Budaet Analyat: (d) Other 2. lu.ber of years ezperience in R&D Syat .. Aequiaitions. 3. IIUIIber of yean Govenment experience. 4. Educational level: (a) Bachelors desree: (b) Kaaters degree: (c) Doctorate dearee; (e) ao dearee. Please return.queatioanaire to IC/l.E. laaley in the encloaed envelope by March 21 !ou do not have to place your aame on the questionnaire. In any.caae. all reaponaea vill be treated confidentially. 281

PAGE 297

DEMOGRAPHIC DATA OF SUIVEY PAlll'ICIPANTS

PAGE 298

283 EKP IH 1&.0 ,_CIUISITJ JOHS RELATIYE D.tliSTED CLI .. FREw FP:EQ CATECCI!:., I..AE:EL tDtE Ff.EO t PCT) < fCT) ( F:CT ) 2. 3., 3.:! ... 3.' 3.2 2 6.3 '' 12.9 ?. 3. I 3.2 U.l I 0. i 6.3 .;.s i&!.6 II. 3. I 3.2 2!:.8 12. 3.' 3 -. '"' 2S.O '' 3 '" 9.7 3.?' 20. 2 6.3 E.S .. 21. 3 '" 9.7 54.8 2 6.3 6.5 61 .3 23. 2 6.3 &LS 67.7 2S. 6 19.8 ''" (17., 27. :!1. 3.2 9(:. 3 28. 3., 3.2 n.s 32. 3., 3.2 9i-.6 33. :!1. I 3 -. ... I Ol. 0 o. 1 J, I "ISSJHG '0(. 0 -----------------TOTAL 3;: lOCI. 0 t(l o." "EAN 18.516 fiEDJH 21.000 "CIDE 2!i.(l(l(l STD CEY .... ?'7 VAR J At4CE ?I RAioiC.E 31. (1(1(1 YALJD 'ASEi 31 IUSSJNC CASES

PAGE 299

284 JO& COY[RNI'IEHT rtELilTIYE .IllS T E [I CUI'I FREO FHQ CATEG;ui<'r' LAeEL C0[1E FRECI > < F"C. T)
PAGE 300

285 LOS YEAs COYERH"ENT RELATIYE ACJ JliS TED CUI'! UTE f;EQ FF:EO C.ATCCIP.'r' LAHL COl Ffcii ( f'C T ) ( PCT > (f'Cl) s. 3. 1 3.2 3.2 10. 3.1 3.2 11 2 6.3 '' 12.9 12. 3.1 3.2 1t. I 1 e. 2 6.3 6.5 3 .. 1 3.2 20. 4 u.s 12.431 3e.'1 2 6.3 6.5 24. 2 6.3 '' 5 .f. 25. I 0 31.3 32.3 [iJ:.9 26. 3 3.2 fl';". 1 29. 3.1 9(.3 32. 3. 1 3.2 93.5 3!5. 3., 3.2 96.8 36. 3.1 3.2 I 00.0 3.1 IUSSJNC 1(10. 0 ----------------TOTAL 'oo. 0 I liO. ( "AN 22.065 .. ED JAN "ODE STD 6.976 YARIAHtE 4.6'2 RANCE 31. 0(1(1 VALID Cte&ES 11 llltSINCi CAS

PAGE 301

286 EDUC--TIOHAL LEYEL REL,.TIYE ADJUSTED cu .. AIIS01.UTE Fli:Ur n:EQ n" LA&EL CO[I[' FICEO CPtT) HO 1. 3., 3.2 3.2 B.IIL 2. 34.4 35.5 3E<.7 "" 3. 17 SJ .1 54.8 93.S PHO ... 2 6.3 6.5 no .CI o. 3.1 "JSSJHC 100.0 -----------------TOT.:tL 10(1.0 100.0 VALID 31 "JSSllfC C.::tSES

PAGE 302

APPENDIX I STA!ISTICAI. ANALYSIS OF StJllVEY QUESTIONS 1-11

PAGE 303

fth AppeDdia I cntailla CCIIIputer aalyda of tla u.pcruu co .urvey queattODa 1-11. two &Dalyaea are tDcludad for aach .-tiOil: ne. baaed 011 the nttre FDUP pariic1pat1Da u the aune1; ud, en, .... d 011 the joll cateaor,. of the parUcipuu. Quaatina 1 ..... &A5Vered baaed 011 the folloviDa acale: 1 2 l 4 5 0 ltrDDBlJ llilfltl,. hither Sliahtly ltroDalJ Doll1t leaauve laaative Poa1t1ve Poa1t1Ye Podtive IDov a or aUve Queatioaa t-11 re auvered 111 Udicat:lq a napoue of 0 co 10. 288

PAGE 304

Atl Jlt!-TE[I FwEC tATECOP.'I' LABEL tOliE FlicEO < F't T > HEITHEIO: f'(IS.-HEC. 3. s ,,,6 15.6 SLCl-fTL.'I' POSITIVE ... ,, ,,,4 'f POSITJ\'E ' :::.o ---------------TOT.:tL 3:: ICICI.ll 1 00. 0 t'ltAN ... 094 "EDIAH ftO!E ,,.., .41(1 RAI4C;E \fAL.JC 32 ftJSSJHC CASES 0 Ill CCIUHT J PtT SLCHTLY ROW POSJllYE J 3.J 4.1 JO& I. I 12 J 4 I ;: I i "CT I J 50.0 I 25.0 I :t. I :2 I 6 I 4 I 1;: PROtURE"EHT RCT I 16.7 I J 33.3 I 3e.7 -J---------------r--------J 3. I IUDCET J 0 J 5 I .o J 100.0 J 0 I 5 .o J 16.1 OT'fEF: -J--------J--------J--------1 . J J 0 I 4 J 2 J 6 .. D J 66.7 J 33.3 J -J--------1--------J--------J ; 4 1t I 3t tOtAL 11.9 61.3 2S.I IID.O 289 Cl'l'l F;; Elj (f'C":'i ?!': 0 l(o(o, 0 4. (IC( (qj (I

PAGE 305

z. !ba deBTee to tlblc1l 71"1 .U.ne awrd fee type of aatract I cntributed to fn1p1 riq tbe cot .lrCNtb af Cba Orbiter Pl'ojectl AD TECI ABSOLU'TE FliECJ FHCI CATCoiCP.'I' Ul&EL CDtE FreEl? < PCT) (PCT> CPtT > 2. 2 6.3 ,.;3 6.3 3. 6 u.e te.e .0 SLCMTLV tOSJTIYE 4. tS 46.9 46.9 71.9 POSJTJYE s. 28.1 28.1 1 0(1," -----------------TOT.:tL 32 too.o t 00.0 .. ECIIAH 4. 0:!) '"'rE 4.00(1 3.0(1C, 3.969 STD .I,EV ; &D 1 VARlMHCE .741 I YALJ() tASES 32 .. lSSJHC C'SES 0 02 COUNT J I PCT JSLCHTL'I' POS.-NE' POSITIVE TOlHl J 2.1 I 3.1 4.1 S.J JOi --------J ------ J --'------J -------I--------I t.J oti 11 41 31 e F-ROJECl "CiT J .0 I :U.S J SO. 0 J I -J --------I--:------J ----I--------J 2. I I J i 3 J 5 I 3 1 1:: RCT I 8.3 I 25.0 J 41.7 J J -1--------J--------J--------J--------J 3. I P.UtiCET I 01101 31 :!J .0 J : .0 I 60.0 J 40.0 J tf.t J --------J--!-----1------.--1-------J 4.1 Ill 21 IJ 6 I 16. 7 I 133 3 I 33. 3 I 16. 7 J 19. 4 -J--------J--------J--------1--------1 COLUMN 2 I 6 14 9 J 1 TOTAL 6.5 lt9.4 .,.2 290

PAGE 306

J. fta ear to wb!ch JDar .S.tll tile caDtractor 'wra affected ., tba .. rd fee c,. of caDtract. LABEL POS.-HEC. f'OSITIYE POSITJYE 3.531 1. 077 VALID CASES 32 83 I RELraTJYE 1111[ JliST[ FUD CO[E Fj;c ( f'C. T ) o. 3.1 3.1 2. 2 6.3 f-,3 3. 13 40.6 40.6 4. 10 31.3 31.3 s. 6 18.8 18.8 -----------TOTAL :s;;: 101).0 1 OCI. 0 ftE[II 3.5(1(1 "OtE RAI.C.E fUSSIHC tAS 0 COUNT ROW PCT IDONT KNO ILCHTLV NEITHER SLGHTLY IW HECwTJYE POS.-NE' f'OSlTJVE l O.J 2.1 3.1 4.1 5.1 JOB --------J-------1--------1--------1--------1--------1 CU"' FHO C f'C T ) 3.' 50' 0 81.3 tOO.O 1.1 II II 31 31 II 8 "'T I .0 I 12.5 I I J 12.5 I -J--------J--------1--------J--------J--------J :t. I "CT I I I .o I I I 6 l .2 I 4 l 12 3. tUCJCET ,O I I 16.7 I 33.3 l 3&.7 -J--------J--------J--------J--------J--------1 I 0 I I I 0 I l 0 I J .o I 20.0 I .o I 80.0 I 0 l J I 0 I 3 I I I I 16.7 I .o J so. 0 I 16.7 I 16.7 I -r--------J--------1--------J--------J--------J I 2 12 10 a.a 6.5 :18.7 l.i.3 19.4 s "' 6 31 1 0(1. (1

PAGE 307

292 4. !be to wblcb J'IN fOUDd tlae CODtractor .. IDler tbe awrd fee catract! I liEL.ATIYE IACIJUSTED cu" 'ABSOLUTE FF:H FH" FHw tODE FI
PAGE 308

5. !:a. .Jeii'M to wO:a.icll JOU MUfte t!aat cu awrd fee r,pe of CODtract affected the attuti.DD co S..auee ud 'l'obl ... RELATIYE ,_OJUSTECI "E ';Oi.. UlE nEer FHr.. (.HTCCrY L,_EIEL tO tiE FficE ll < f'C T > < f'CT) POS.-NEC. 3, 4 12.S SLCHTLY POilTJYE 4. ? 2L9 2L9 POSITIVE s. 21 65.6 65.6 ---------------TOT-.L 32 "'". 0 uo." 4.531 4.73S ftOttE STD .718 YAIUNHCE .5n5 vALl[) 32 RISSJNC tSES I COUNT J aow PtT JNEITNER SLCHTLV ITINCLV ROW JPOS.-NEC POSITIVE POSITJYE TOTAL I :J,J 4.1 '. J JOB J J 2 I 6 J 8 ,.RO.IECT .. I I as.o I '75.0 J as.e. a. J t J I J 10 J f'RCitUREftEHT ftCT I '3 1 1.3 I 83.3 J 38.7 3. I a J 2 I I 5 _.NALYST I 40.'0 I 40.0 I 20.0 I 16.' -J--------J--------1--------J 4. J I 2 I 4 I 0Ti4ER I I 33.3 I 66.7 J U.4 -J----------------J--------1 COL.UMtof 3 ,, 31 totAL 9.7 a2.6 61.7 ..... 293 eu"
PAGE 309

6. fte dell' wtalc:'b JOU t.ellft'l tbat the aard r .. criteria ad area of .-phaia prcrrtcaea t.De caaatr.Kur the by uu of c.ncen of IIASA 7 PL,:eTJYE C:tTEr.OF-',' LABEL Ff At: ,lliSTECJ HEG:
PAGE 310

7. 1"' liqr .. 'o wbicb JDD Mline Cite naluaeloa fnquesu:J appnpriatef REL.ATIYE AOJUSTECI CUI'! CATC.OP'I' LAP-EL. ABi-OL.I.IT Fli:E&l Hc(:1 CO [IE FRECI < f'CT) ( f't T J CPCl > 2. 3.1 3.1 3.' 3. 7 21.9 21.9 IS.O SLCHTLY FOSlTlYE 4. 12 62.5 $TRHCLY POSJTIYE s. 12 37.5 '37.5 10(1.0 ----------------TCITAL 32 I 00.0 roo. o ftEDJAH 4.167 ftOI:lE . 000 Y.:.li:lANtE .733 fcAI4C.E 3.U(l "EAN 4,094 STD {)"-' 85o YALJ[) 32 "JSSJNC c.-SEs 0 117 COUHT I ROW PCT JSLCHTL'I NEITHER SLCHTLV STRNC.L 'r' ROW IHEGt:ITIYE .-NEG. POSIT lYE POSITIVE TOT.:tL J 2.1 3. I 4. J 5. I JOB I J J : 3 I 4. J I J fROJECT "'' I .o I 3?.5 I so. 0 J 12.5 I 25.8 2. I 0 J 3 I 2 I 7 I 12 F-ROCURE"EHT "CT I .o I 2!1.0 I u.? I SIL3 I 38.7 -1--------J--------J--------J--------J 3. I I 0 I I 4 I 5 PUDC.ET ANALYST I .o J 0 I 20.0 I 10.0 I "' ---------J--------J--------J--------1 I I I I I 4 I 0 J OTHEP J 16.7 I 16.7 I 66.7 I 0 I 19.4 CDLU"N ., 11 12 3t TOTAL J:5.5 J8.7 100.0 295

PAGE 311

. Dall ciearee to ntcil JChl tilat tile IIW&'Ii '" ,,.,. of CODtract affected dec1a1cnt..,k1na betweD tbe an.n.at aDd tbe ODDtractor? RELATJYE IUlJLIUED 296 cu .. U16E1.. AtSOLliTE C0[1E i FREQ ( PCT > FP.EQ < F-tT 1 FP.EO
PAGE 312

ta'TEC.OF-'r' HOHE TOT,.LL'r 6Ttl 0',1 t. de&ree to wb!eh JOU lle11eve that the .alutioa criteria bava olljactive2 RELRTJYE ,_D JLtSTED AE!SCLLTE LAE:EL CO[IE FREO < fC. T) ( f'C T) o. 3 9.4 9.4 '. 4 12.5 12 2. 3 9.4 9.4 3. 5 1!5.6 1!5.6 4. 2 6.3 6.3 2 6.3 6.3 2 6.3 6.3 7. 6 1e.e 16.8 e. 3 9.4 9.4 9. 3.1 3. I I 0. 3.' 3 .I ----------------TOT.:.I. 32 '00. 0 1 DO, 0 4.37'5 ftEtiJIIIN 4.000 1101\E 2.94i 8,,,4 RA..,C.E V,_LJD CASES 32 IIJSSJNCi CASES (I 297 CUI"' FHO < f'C T J t.4 21 31.3 46.9 53, I ..... 93.8 teL 9 100.0 7. 0(1(1 tCI.OU

PAGE 313

JOEl t. !be dqrH to wblch JDU .au. .... that tile ... luaciaD ebould ba objacc1 I COUHT I ROW PCT JHOHE J 298 1.11 2.1 3.1 4.1 S.J I --------------------1---------------l1 'I 2 J 011 I J t J 1 J I I f' OJEC"!' "CT J :!S.O I .0 I U.S I t2.5 I U.S I U.S J -1--------1--------11 l -------------J-------J-------1-il. 1 0 J 2 I 2 J 2 I 0 F-ROCUREI"'EHT "CT I 0 1 16 .7' 1 16.7 I l6 7 J O 0 I -1-------J -------l..z. I . o J J --------------J--------J--------J8 3 I J t J 8 I t I 0 I 0 J UDCT -.NAL'I'$T J 20.0 I 20.0 I .D I ,O J. .O J .O I -J-------1--------1I 4. J D I I I 1 .0 I 16.7 I .0 I I .0 1 7 1 COLUI"'H 3 4 3 S 1 2 ,COHTJHU) TOTAL 9.7 I 9.7 16.1 3.2 6.5 89 COUriT I ROW F-C.T I I TOT.:.LLV ROW TOTNL t.l u.J Joe ---------------s--------J--------i--------1--------J I. I 0 I 1l1 I 1 0 J 0 I I I 6 .I 7. I JEC.1 "CT I (I I 12 S I 12. S I 0 I 0 I 2S. I 2. I 2 I 3 I 1 I 0 I 0 I 12 "CT I 16.7 I 25.0 I I 8.3 J .o I .o I 38.7 --------1--------J--------J--------1--------l :J, I II I I I t I 0 J I I 5 l'UDCET ANALYST J .o 1 2o.o, 1 20.0 1 .o 1 ao.o 1 16.1 OT"'EP. 0 I 1 I 0 1 t I .o J 16.? I 0 J 6 .o 1 I J .0 I 16,71 I COLUMN 2 6 3 I I t 3 t TOTAL 6.1 tt.4 t.7 3.2 3.2 100.0

PAGE 314

10. de1ne to wblc:h yeu lleU.ne the awrd fee proc:e11 could ban Ilea Sllpnw&d 1 for tJae Orbiter CGDtrac:t? .-p .JLIST IIBSOi..UTE Fli:EQ tATE,OF-V LABEL CODE HEO < F'CT > 2. 7 21.9 21.9 3. 4 12.5 4. 2 6.3 6.3 5. .. 12.5 12.5 6. 4 12.5 12.!5 7. 4 12.!5 12.5 e. 5 15.6 15.6 I 0. 2 6.3 6.3 -----------------TOT,:jL 3;: '00. 0 100.0 5.1.! ftEDIAH :1.250 ftODE STtl OE'"' 2.!5( YAIUAHCE 6.286 lcAHGiE YALI" 32 ftlSSlHC CASES 0 299 CUI'! FP. CPCT > 21.9 3 .... .0.6 5J.' 65.6 78.1 93.1 100.0 2.000 e.ooo

PAGE 315

10. fte ..-to which ,aa th awar '" proee cou" lane bftll t.pnn.t lor tha eontract' COUNT I '-OU PCT I TOT,.LLY. lttU I TOlAL I 2.1 3.1 4.1 5.1 6.1 7.1 8.1 10.1 JOB --------I------I--------I--------J--------J--------I-------I------I-------1 t. I 2 I 2 I 1 I 0 I I I 2 I 0 I 0 I 8 F'IOJCT "GT I I 25.1.1 I l'-.'5 I ,(1 I 12,., I I ,0 I .0 I 25.8 -1------------------------l----------------l--------l--------1--------l 2, I 2 I . 2 _I 0 I 2 I t I t I 4 I 0 I 12 F R('IC'IJR"EHT "CiT I 16. 7 I 16, 1 I tt I I(.. 7 I 8. l I I. 3 I 33. :t I 0 I 38. i ---------l--------l------------------------l--------1---------------3. I 3 I 0 I 0 I 0 I I I 0 I 0 J I I S PUDCET AHAU'ST I 60,0 I .0 I .o I .0 I 20.0 I ,O I .0 I 20.(1 I 16.1 -I----------------1----------------J--------I--------I---------------I . I 01 01 01 21 I I t I I I I I 6 OT!-f[R I 0 I (I I 0 I 3J. I 16. 7 I I 6, 7 I I 6 7 I I 6. 7 I I 9. 4 -l--------l--------l--------l--------l--------l--------l--------1--------l 7 I 4 4 4 S 2 Jl TOTAL 22.6 12.9 3.2 12.9 12,9 16.1 6.5 101.0 w 0 0

PAGE 316

u. '!'be dell' to .Uch JOU bel.iCYe tbe fee detarCD&tiDD official (I'DO) ... fullJ iDfonad reaerdiD& tba par:loclic ... rd fee .valuatioal? RELiliTJYE At> JUS TED Ffi:Er HECI tOtE FREO < PCT J < PCT J HOHt o. 1 3.' 3. t 3. 3 3 9.4 7. 7 21.9 21.9 e. s tS.6 15.6 9. s 15.6 15.6 TOTALLY 10. 8 25.0 25.(1 -----------------TOTAL 32 HCI.O 100.0 "E. AN '1, 4 '38 ft[1 I 7.900 MOt'IE STD DEY 2.539 VARIANCE 6.448 RANGE YALID CASES 32 MJSSJHC CASES 0 301 cu .. FF:EQ C PCT) 3.' 12.5 21 .9 43.8 ,9,4 .o 100.0 10.00(1 tO. 00(1

PAGE 317

11. the -.ree te yau '-lt... r .. ter.!natloa efflelel (PDG) ... fallJ lefor.ed reaerdlnl the periodic averd fee eYaluatlon? COUNT I ROU PCT TOTALLY ROW I TOTAL I 0,1 J.l 7,1 lt.l 9.1 10.1 JOI --------J--------J--------1-------I--------I--------J--------I-------I I. I II t I 01 II I I 21 21 8 J-ROJECT "CT I I 12.!!5 I .0 I I 12,5 I 25.0 I 2S.O I 25.8 -l--------------------------------l--------l----------------1 2, I -0 I 0 I 3 I -3 I I I l I 2 I 12 ,_ROClfltE"EHT "CT I 0 I 0 I 0 I 25. 0 I 8. 3 I 25. 0 I 16. 7 I 38. 7 -l----------------l-------l--------1----------------l--------l 3, I 0 I I I 0 I I I 0 I 0 I 3 I fltiDCET ANALYST I ,0 I 20.0 I .0 I 20.0 I .o I ,O I 60,0 I 16.1 -----------------------l--------r--------r--------r--------r ... I OJ I J 81 II 31 OJ II 6 J .0 I 16.7 I .0 I 16,? I S,,n I oO I 16.1 I 19,4 -1--------r--------i--------r--------1--------1--------1--------1 COLU"N I J 3 6 5 S 8 31 TOTAL 3 o 2 9 1 9 1 19 4 16 o I I 6 I ::IS 8 100 o 0 w 0 N

PAGE 318

.APPENDIX C ANALYSIS OF STRENGTHS OF A'WABD FIE CONTRACTING

PAGE 319

c-at 7. Porce .u.aaaeDt bwob--t frequetly; fee s.. aood aauaaut teol: .... r .. actin role of anerDeDt ...,..en. COUNT I ROW PtT I I I 0.1 7'. J Ruw TOTAL --------1--------------1 ,s. 1 6 1 2 1 e F-RO.IECl "'T l 17!!5.0 l 2!.0 l 2:i.8 ,6, J I 0 l 2 J 12 FROCLIRf,.EHT "'T J 83.3 I 16.7 I 38.7 -1--------1--------J 97. J 3 I 2 I 5 liUI>CET ANAL'I'ST l 60.0 l l ".' 9&. I 5 J I 6 OHER I 13.3 I 16.7 l 19.4 -I------I--------J CCLU"H 24 7 31 77.4 22.6 too. o ea-eDt. I. aard fee va proper for Orbiter DDTU CODtract. Jetter perfor.aace raeulted aDd better wia1b111ty. COUNT 1 ROW PCT l J T0Tf4L l O.J J JOBCOI>E 9:i. l 6 l 2 I e F-RO.IECT "CT I rs.o I 2!.0 I 25.8 96. J e J l 1:! "CT I I 3,.3 I 39.7 97. l I I J !I IUI>CET ANALYST l 80.0 I 20. (I I 16. 1 -l--------J--------1 98. I 3 I 3 l 6 I 50.0 l so.o I 19.4 -1--------J--------J tOLU"N 21 10 ,, TOTAL 67.7 32., 100 0 MUniER OF RISSJHC D8SERYATIOHS 304

PAGE 320

305 ea-Dt 1. lu.alatll fl'-aut mtcatt.n. .. COliHT I f.Ot.l F'C.T I ROlli J T0Tf4L J D.J t.J JDBCOI:IE ,s, I '4 I 4 I e "(;.T I so. (I I so.o I -r--------1--------1 ,6, I ,5 I 7 I F R)C:LIRE "E !'IT ft(;.T J 41 ,'7 J ss.:; J 39.7 97. I J 3 I s ltUDCET IIH.:tLYST J 40.0 I 60. (I I 16. I -J--------1--------J 98. I 2 I 4 I 6 OT ... EF' J J 6E. 7 I 19.4 -1--------i--------J tOLli"N 13 IS 31 TOTAL 41.9 se.' IOCI.O HU,EEP. or "IUJNC DfsSEf!VATJOHS 1 c-ea.t 2. Cl'ut cooperetiDa ud t ... l'e1at1Daah1p. COUHT I ROW PtT I ,,,.., J TCITML I o.r 2. J JOBCOtiE --------1--------1--------1 95. J I 1 I .e :::RCoJECT .. ,, J 87.! J 12.S I 2S.8 J------------1 .,. J 8 J 4 I r-ROCURE,.EHT ftCT J 66.7 J 33.3 J 3e7 --------J--------J 97. J 3 I 2 J 5 ltUD;ET AHIILYST J 60.0 J 4(1.0 I ". 1 -J--------1--------J 9t. I 5 I I J 6 I 13.3 I 16.7 I "" -J--------1--------J COLU"H 23 3t TO TilL 74.2 1(10.0 NU,.E'ER OF ftJSSJHC 08SERYATJONS

PAGE 321

c-zat l. !mel7 ud periMic f...SNck CO C.tr&e&Ol' U nprd co JlCforiuu. 'tOUNT I rcow PeT I ROW I TOT,:.I.. I o. J 3. I .IOiCOCIE --------l--------l--------1 9S. J I 7 I 1 I 8 F-ROJECT ftl:iT J 87.S I 12.5 I '' I t I 3 I t2 f. ROClRt"E I'IT ft(';T J I 25.(1 J 3&.7 -J--------1--------J 97. I 3 J 2 J s lll1C1CET ANALYST J 60.0 I 40.0 J t6. 1 -z--------1--------J 9&. I 6 J 0 J OT"iE'Fc J 100.0 I 0 J -1--------J--------J C'C'II..li"H 25 6 TOTAl. "'' 19.4 10(1.0 OF tUSSJHIO c-at IIDtivacor co cnuactor llaior ..-a..mt; attatiOD 1etter. COUHT J PtT J RCII.' I TCITfool.. J 0. J 4. I JOStOt'IE ts. I 4 J 4 J < F'RO.IECT ftCT I so.o I !I o .o J 25.6 -J--------1--------1 96. J 9 I 3 J ,_RCICU'-E"EHT ftCT J 75.0 J 25.0 J 3S.7 -J--------J--------1 97. I I 4 I 1 I 5 PUDCET ANALYST I eo. o J 20.0 J ". 1 -1--------1--------J 98. I 2 J 4 J J 33.3 J 66.7 J 19.4 COI..U"H 19 u ,, TOTAL 61.3 38.7 10(1.0 .. UnEER OF ftiSSJHC 08SERATIOHS 306

PAGE 322

c:..-at 5. labjectt'l"ltJ all.,_. for efnf ,....tty for ... r ten coat pcnrtb tf teclmie&llJ -.accuaful. JOB CO [I COUffT J RCIW PCT I I I O.J 5. I --------J--------------J 95. I I 8 J 0 J ftCT I J .0 J I 10 I J ftCT J 83.3 I 16.7 J -1--------I-------I 9':", I 4 J I J JUDCET I 80.0 I 20.0 J -1---..---J--------I 1'8. J ' J 0 J J uo.o I .o J DHU ; 28 3 90.3 9.7 HUNEER OF 1'1J$SIHC TOTMl 8 12 3&.7 5 U.l 31 100.0 6. F1aibU1ty to, eccept cbaqea fD v1tb&Nt jor cODtract' pertub&tioDi cDDtractual prvriaiDDA aiapl.1fie4. JDBCOI:IE COUNT I P.OW PC.T I l I o. J --------l--------1--------1 95. 1 s I 3 I ftCT I f2.S I I J 8 J ,. J ftCT I ,66.7 I I -J--------J--------1 ,.,., J 3 J 2 J 8UDC:ET ANALYST I 60.0 J 40.0 I 98. I B J J DT!oiEII: J !83.3 J 16.7 I -J--------J--------1 COLU1'1N I 21 1(1 IDUl 67.7 34'.3 MUftiErc OF ftiSSIHC DBSERYATlDHS 8 12 3&.7 5 16.' 19.4 31 lfiO. 0 307

PAGE 323

APPENDIX D AHAl.YSIS OF WEAENESSES OF AWARD PEE CONTRACTING

PAGE 324

e-Dt u. W1ridua1 ... luatora are ltiaaed; tbeJ .-cai" llaat tbey ua Fd1A& tta.ael .... .tOBCOtJE COUNT ROW PeT I I 0. I '1.1 --------1--------J--------J ROW TOTML 95. I 4 I 4 I I "CT J 50.0 I 50.0 I 25.8 -J--------J--------1 96 I : 5 I 7 I t 2 I' 41.7 I 58.3 I ,!,7 t?. I I 3 I 2 I s ANHLYST I 60.0 I 40.0 I 16.t OTHER te. I 2 I 4 I 6 COLllf'IH I 33.3 I J 19.4 -1-------J --------J 14 : 45.2 '7 54.e 31 IOCI.O HUftEEP. OF "ISSIN' DBSEP.VATJOHS I c-Dt 12. ud apeul'We adUDillttati'We proc COUNT I ROW PC.T I ROW I TCITML. I 0.1 12. J JOB CODE 95. I 7 I 1 I e "ROJECT "'T I 87.5 I 12.5 I 96. I 9 I 3 I '2 ... OC.UrtENENT "CT I 7t'.O J 25.0 I 38.7 97. I 3 I 2 I 5 uDGET AHAL.Y6T I :60.0 I 40.0 I 16. 1 -J--------1--------J .. I 4 I 2 J 6 OTiotEP. I 66.7 I 33.3 I -1--------J--------J COLUNH 23 I 31 TOTAL 74.2 25.8 100.0 I HUftiER OF "JSSIHG OBSERVATIONS 309

PAGE 325

.lOB CODE COUNT I ROW PCT J I J ".I 13 .I -------.-1-------J --------J J I 2 I "CT I I 25.0 I -J--------1--------1 '' I 10 I 2 I .. EtoiT ftGiT I 83.3 I 16.7 I -J--------1--------1 97. I 3 J 2 I 8UtltT ANALYST I 60.0 I 40.0 I 98. I s I 1 I OHEI< I 83.3 I 16.7 J -1--------1--------1 TOT.:.i.. 8 25.& 38.?' 5 16. 1 COLU"H 24 7 TOTML 77.4 100.0 NUftEEP. OF fti$SIH' OtSEPYATJOHS c-at 14. rd to keep aw'E'd fee -..bub iD M.lMce with project;. Gcwer..at doea DDt Wt -..bub aDd criteria .aauab. .IOEiCDtiE r-aOJECT COUNT I ROW PtT J I I 0.1 14.1 --------J--------J--------1 I 8 I (I I "CT I '00. (I I .o I -1--------1-------J J 8 I 4 I ROCURE,.EHT "CT I 66.7 J 33.3 I -J--------J--------1 97. J 2 I 3 J JUDCET ANALYST I 40.0 I 60.0 I -J--------1--------1 9(1. I s J 1 J OT'4ER I 83.3 J 16.7 I ---------1--------J tOLU"H 23 e TOTAL 74.2 25.8 NUft&ER OF ftiSSIHC OISERYMTJOHS TCITML 6 2:.6 12 36.7 16. I 19.4 3t 10CI.O 310

PAGE 326

e-Dt 15. .aeb f1lter!q froa walaaton to .-rd f .&luAtiaD lloud ad fee detendutiaD offidala. COUNT I ROW PtT I P.OIJ I TCITJ.41.. I 0.1 15.1 --------1--------1--------1 tS. I 7 I 1 I e "GT I 87.5 I 12.5 J -J-------J--------1 .,. I 10 I 2 I "'T I e3.3 I 16.7 I 3&.7 -I--------1--------1 9?. I 2 I 3 I 5 uDtET ANHLYST 1 I 60.0 I 16.1 -1--.------J --------1 !a8. I S I 1 I 6 I 83.3 I 16.? I -1-.. ----1--------J 24 77.4 NUftiER OF O;SERYATIONS .,. :!.1 106.0 c-at 16. &ard fee an 1CN to -.aluatora0 .-Mn ofta nplaced vith alterutea. COUNT I P.OW ftT I ROlli 1 1 O.I 16.1 JOBCO()E I 8 I 0 I 6 P.ROJECT "CT I 100.0 I .o I ::!-.6 -z--------1--------1 !aE.. I I 0 I 2 I 12 F-ROt:URE"EHT "" J 83.3 I 16.7' I 3SI.7 -1---------------1 9?. I 15 I 0 I 5 IUDC:ET AHAL'I'ST I 100.0 I 0 I liS I -J-------l--------1 98. I 4 I I 6 OTlotER I 66.7' I 33.3 I 19.4 -1-------1--------J COLU"N 27 31 TOUL 87.1 12.t 1(10.0 HU"EER OF "ISSJHC DBSEP.YATIOHS 311

PAGE 327

e-Dt 17. C..tractor .u .. t.s to t:tfJ Gum: t ......... ... baa :ad DDt MCUUI'ilJ tM ... t _,_ COUNT I ROW PCT I I TOTNL I 0.1 1?.1 JOBCODE --------1--------1--------1 95. I 7 I t I 6 ftCT I &7.5 I I I I I I ft.:O T I 91:,7 I & 3 I 36. 7 t7. I 1 4 I I I S ANALYST I 10.0 I 20.0 I 16.1 -1--------1--------1 '8. I : 5 I I I 6 I 13.3 I 16.7 I tt.4 -1--------1--------1 COLU"H 27 4 31 TOTNL 87.1 12.9 UICI.O HU"fER ftiSSJNC 18. Mlard fee DDt lar1e awgb to Dftl'ca.e other corporate 1oala, e. a. M1Dt.a1D WDrll:.force, ate. COUNT I PCT I ROiol I TOTHL l 0.1 IIIL I JOBCO[l --------1--------J--------1 ,5, I 7 I I I 8 FROJECT ftCT I 87.5 J 12.5 I -1--------1--------1 .,. l 12 I 0 I :: ,_ROCLREftE.NT ftCT I tOO. 0 I ,(I I 38.7 -J--------1--------J 97. I 4 I 1 I s uocn I 0 I 2D.O I 16. I 98. l I I 6 OTitEf( I 83.3 I 16.? I ,, .. -r--------1--------1 CCILU"H 28 3 31 TOTAL tD.3 9.7 too. o NUftEER OF ftiSSIHC DBiERYATJDNS 312

PAGE 328

APPENDIX E ANALYSIS 01 TYPE OF CON'l'!AC'l' PIEFERENCE

PAGE 329

.,,.a.u.a I cntaiu Qe e-.ater of ...-ctn 14 of Clle Qlauttft 14 aunc, 81lon .. lw: 14. h J0111' ,.r.,.cU udicate wlaat J'OU .. lien to .. tlae .. n fon of eoatractiq (claoo ,.._ tlae liat: linD) for tlaa Sadicated me of SD&Iicate earee of prefereace. IJ.n of type of Coacract: lituatioD: 1. CPrr (ca.plaUn fon) 2. CPP'F (lnel-of-effort) J. OAF (ca.plattoa fon) 4. CPAP (le.el-ofeffort) 5. CPU (ca.pleUoa fon) 6. rued Price laceathe 7. fin fiucl1 h'tce txpe of Contract cselect auaber fr .. lt.t) A. jor 1611 projecu (DIIt6! Pb.ue) 1. u.tced producU.oa (3 to 10 111lita) c. Operatioul npport of 1611 .,naa lak Desree of fnferaace, utaa a Scale of 314

PAGE 330

315 ... ,_ RAJOR ... PROJECTS ODT&E PH.SE RELATJYE ,_[)JUSTEO tu"' CATCOP.'r' LAE:EL FP.EP FF-EfJ CO [I ( f'CT > CPFF-Cti .. PLETJOH t. 2 6.3 6.'3 t..J CPFc-LEYEL EFFT 2. 2 6.3 6.3 12.S .. PLETJ CIN 3. 21 65.6 6S.6 '"' EFFT 4. 3 .... '"' 87.5 C'PJF-CO .. F-LETJOH s. 3 9.4 .... 96.9 FIP.M PRICE 7. 3.' J,' 1(10.11 ----------------TOTHL. 32 HI C. Cl ICIO.O VALli> CASES ftJSSJNC CASES 0 ...... COUNT J ROW PtT JCPFF-COI'I CPFF LEV CPAF-CCII'I tPAF-L.' C.P 1 F -C Cr:1 ROW JPLETJOH EL EFFT PLETIOH EL EFFT PLETICIN TOTAL J t.J 2.1 3. J ... J 5. I JOEi --------J--------J--------1--------J--------J--------J I. J 0 J 0 J 6 J 0 J (I I 6 F-ROJEtT "'' J .o J 0 J 100.0 I 6 J 0 J 25.6 -J---------------J--------J--------J--------J 2. J t I '2 I 6 J ;: J t J t:! foROL.ltRE"EHT "'' I 8.3 J t6.7 J 50.0 J 16.7 J e.J. J 39.7 -J-------J--------J--------J--------J--------1 3. J t J 0 J 3 J 0 J J IUtiCET IIHIILYS1 I 20.0 J .o I 60. (I J .(I J 20.0 I 16. I -1--------J--------J--------J--------J-------J J 0 I 0 J .. I 1 J 1 J J .o I .o I 66.7 J u .. 7 J 16.7 J -J--------J-------1--------J--------J--------l COLU"N 2 2 21 :s :s :II tOTAL '' '' 67.7' 9.7 9.7 1110. 0

PAGE 331

316 Q14.-R aANk OEG,EE OF PREFE-EHCE REL.ATIYE ADJUil[ C:UI'I llle-OLliTE FREt:' Ft.EO CATE"n LABEL CODE FREQ ( ,.,, ;, < PCT)
PAGE 332

eOUHT I ROU PCT ILOUEST P I REF I 0.1 :J. I 5 .I '' 7.1 8.1 HIGHEST tiOV PPEF .. tOTAL 9.1 .... JOI --------l--------l--------l------l--------r--------1--------l------l--------l 1.1 II 01 01 01 II 41 II II I f'tiOJECT HCT I 12,5 I ,0 I ,o I .o I 12.5 I SO. O I 12.5 I 12 5 I 25.8 -l--------------------------------l--------1----------------l--------l f, I 0 I 0 I D I I I I I I I I I :J I 12 P.OCQRE"'HT HCT I ,O I .0 I .0 I I 8.3 I SO,O I 8,3 I i 38.7 -l--------l--------1----------------l--------l--------l--------l--------l 3. I 0 I t I t I 0 I 3 I 0 I 0 I D I S PUDCET ANALYST I ,O I 20.0 I 20.0 I .o I 10,0 I ,O I ,0 I .o I t6.1 -1--------1--------r--------r--------r--------r--------a--------1--------r 4, I t I 0 I 0 I f I 0 I I I I I 3 I 6 OTMU I f6, 7 I 0 I 0 I I 6 7 I 0 I 16 7 I 0 I 50, 0 I ". 4 I------I----M--1---------------II--------I----I--I COLUHN 2 f I 2 S I I 2 7 31 TOTAL 6.5 3.2 J.: 6.5 16.1 35.5 22.6 100.0 w ,_. .......

PAGE 333

318 "'411 LUUTEO UHJTS RELATIYE DJUSTEIJ CUI'! UI&EL liTE Ff.H F'f(E(J F'H" CODE' FreE" < F-C. T > ( PCT) (,_( T ., !5. 10 3L3 3L3 31 ,J EFFT 4. 2 6.3 6.3 n.s CP JF-CCol'tF"LET J OH s. I 2S.CI 25.0 FliC[[I 6. I 25.0 25.0 81.5 FIR" FIXED PRICE 7. 4 12.5 12.5 I 00.0 ----------------TOTAL 32 I 6(1. 0 100.0 Y,_LIO CASES 32 "lSSIHC CASES 0 1148 COUNT I PCT CPAF-LE tPJF-tOH IPLE'TIOH EL FFT ICE TOTAL J 3.1 4.1 S.J 6.1 7.1 JOB t.J 21 01 21 41 oJ e "CT J 25.0 I .0 J 25.0 J SO.O J .0 J -J--------J-------J--------I--------1--------1 2. I 6 I t I 3 I 0 I 2 I 12 "'T I SO.O I 1.3 I 25.0 I l 16.7. I Je.7 -J--------J--------J--------1--------1--------J 3. J t I 0 I 2 I 2 I 0 I 5 JUD,ET ltHALVST I 20.0 I .0 J 40.0 I J I 16.1 -J--------J--------1--------1--------I--------J 4. J I I I J I I i I 1 J 6 OT=-tEP. I 16.? I 16.7 J 16.7 I I 16.7 I 19.4 -1--------J--------J--------J--------J--------1 COLU"H to 2 8 8 3 31 TOTitL. 22.3 6.5 25.8 25.8 9.7 10\.0

PAGE 334

319 0148R .ANK DEGREE OF REL.aTJYE AI>JLISTEO cu" AEIS.OLUlE Flic(:1 FH" tATEGCR'r LAiEL CODE FREQ ( F'CT >
PAGE 335

tl COUNT I IOV PCT ILOV!tT P I REF I t.l 3.1 !.1 1.1 HICH!!T .... PP.EF TOTAL 7.1 9.1 11.1 ,.. I. I I I 0 I I I t J I I 3 J t I 2 I I PnJ!CT "GT I 12.S I .o I t2.S I .o I 12.S J 37.S I .o I IS.O I 2S.8 ----------------------------------------------------------------I, I t I 0 I I I I I 3 I 2 I I I I 12 PnCURERENT RGT I .t I .o I 8.3 I 8.3 I 29.0 I 11.7 I 1.3 I 33.3 I 38.7 3. I IUDGET ANALYST I t I I I .o I 20.0 I I I 2 I I I ,O I 40,0 I 21.0 I I I I I .o I 21.0 I t I S 0 I tt. I OT;tER --------------------------------------------------------------4. I I I t I o I t I 0 I 0 I 0 I 3 I t I 2 I COLUMf TOTAL I 11.7 I 0 I ,o I !0.1 I .0 I 33.3 I 19.4 2 .. I 3.2 I 1.5 3 9.7 .. 29.8 I .. 29.1 31 ..... w N 0

PAGE 336

014(; euPPORT .. D LABEL CPFF-CO"PLETIDH CPFF' LEYEL EFFT EFFT t.ftiF-COPIPLETIOH FliCED PRICE FIR" FIXED PRICE VALID Cli'SIES 814C COUNT I 32 aBSOLUTE CODE FRED I I I I 2. 4 3. 3 4. 18 s. 4 .,, I TOTftL 32 IUSUNG CASES 321 RELATI'IE ADJUSTED CU" FRE" Flli:EQ FRE" <,CT) CPCT) 3.' 3.1 3.1 12.5 12.5 t.4 t.4 25.G 56.3 56.3 .,, 12.5 u.s t3.8 3.' ::1. I t6.9 3., 3.1 uo.o -----_,_ ____ 100.0 100.0 0 ROW PCT ICPFF-CO" CPFF LEY CPAF-CO" CPAF-LEY tPIF-tO" FIR" FIK JPLETIOH EL EFFT PLETIDH EL EFFT PLETIOH PRICE I t.l 2.1 3.1 4.1 5.1 ?.1 t. I tDJECT "CT I GJ 21 II 41 II .0 I 25.0 I 12.5 I 50.0 J 12.5 I G I I .1 I 2S.t ,_ -1---------J -------1-------1 a. 1 tDCURERENT RCT I Gl II 21 71 21 0 I 12 .o I a.3 I 16.7 I 58.3 I 16.7 I .0 I 38.? l ... l------.-.-.J---:----1------------I------J 3. I I I I I I I 2 I 0 I I I S ID,ET I 20.0 I 20.0 I 0 I 40,0 I .o 1 zo.e 1 16.1 4. I I I I .o I 0 I 0 I I J 5 I I J Cl I 6 .o I 83.3 I 16.7 I .0 I 19.4 COLUfiH TOTAL I ,,., 1 4 12.t 3.2 :!II 100.0

PAGE 337

322 ,. Clt4C:R aAMY. DE,REE OF PRfFEREMCE RELATIVE .-D.IUSTECI CLI" ABSOLUTE FREP FP.E" F!::EC: UTEC.OF:'r' LABEL. CO[I FREQ C f'C T > LCM.IEST PREF o. 2 6.3 6.3 6.3 3. I 3.1 3.' 5. 2 6.3 6.3 t5.6 6. ? :!t.9 2L9 37.5 ?. 6 18.8 IS.8 56.3 e. 3 t.4 9.4 3 9.4 9.4 ?:.o HJ::HEST PREF 10. 8 25.0 25.0 10(1 0 -----------------TOTHL 32 tOO. (I IOC.O ftUM ?. 125 ftECIJAH 1. tt1 "0[ to. OOCI STD DE'.' 2.n,. YARJNHtE 6.t5.2 i"Hr.E VALID tASES 32 ftJSSJM' CASES 0

PAGE 338

814Cit COUNT I ROU PCT ILOUEST P I REF 1 o.t 3.1 6.1 HIGHEST ROV PREF TOTAL 7 .I 1.1 ,,1 10.1 JOI --------l--------1--------r--------r--------I----------------I----------------J I, I I I 0 I 0 J I I I I I J 2 I 2 I 8 "GT I 12.! I ,0 I .D I 12.5 J 12,5 I I 2!.0 I 2!.0 I 2!.8 2. I 0 I 0 I 2 I t I 3 I I I I I 4 I 12 "GT I .0 I ,O I 16.7 I 1,, I 2!.0 I 8,3 I 8.3 I 33.3 I 38.7 -l--------l----------------l--------l--------l--------1--------l--------l 3. I 0 I I I 0 I 3 I I I 0 I D I 0 I PUOCEf ANALVIT I ,O I 20,0 I .o I 60,0 I 20.0 I .o I .o I .o I 16.t ---------1--------l--------l--------l--------------------------------l 4, I I I 0 I 0 I 2 I I I I I 0 I I I 6 I 16.7 I ,O I .0 I 33,3 I 16.7 I 16.7 I ,O I 16.7 I 1'.4 ---------1--------l-----------------------------------------------tOLU"N 2 I 2 7 6 3 1 '1 :n TOTAL '.! 2 6 22. 6 19. 4 9. 7 _9. 7 22, 6 110. 0 ""

PAGE 339

APPENDIX P ANALYSIS COST GROWTH QUESTIONS

PAGE 340

Appeadis F CODtaiDa tbe co.puteT u.al7d1 of n"eJ .... UOD J aader tbe Coat aactioD of 1Datru.eat. !be cateaoriaa of coat arcnrch are laMlecl: eoat 1 lDadequau rwuttaa; Coat 2 'feclullc&l Vacertaiaty; Coat 3 Teclmtcal AdvaDCDIDt. rot uch of tba cateaoriaa COlt arovtbo tbe reapoaae for tbe &Dtira 1roup aa a .. ole la liVeD firlto followed bJ tbe tllpDRael bJ tbe participaDtl' job er&l. 325

PAGE 341

326 COST I INADEQUATE FUHDINC RELATIYE --D JLIST[1 cu .. ABS.Ol.UTE FJtU FJcQ FF:E" L--BEL CO !:IE FREO CPCT > CPCT) HOlle o. 1 3. I 3. t 3. I 2. 3.1 3. I 3. 2 6.3 6.3 12.S 4. 3.1 3. I 1!.6 6. 4 12.5 12.5 2t .1 7. 3 9.4 :S:'.S e. 9 28.' 28.1 65.6 5 15.6 1$.6 ,, 3 TOTAL.L'r' I 0, f. 18.8 16.8 to('. 0 ----------TOTML 32 "'"" "'". 0 tc[AN 7.313 "EIIIMH ?.t "ODE e.ono $TD (''.' 2.533 YARIAHCE 6.415 flANCiE 1(1.0(1(1 VALID CASES 3:! "JSSJH!; CASES

PAGE 342

COITI CDUMT I ow II'CT IMOHI I I 1.1 2.1 3.1 4 .I TOT'ILLV TOTAL '' 7. I e .1 10.1 .tal ------------------------------------------------------1------------.. -1 I. I t I I I I I I I I I I I 0 I I I 2 I a 1'.0 JC T "GT I I I I 2 :S I I 2 I I 2 5 I I 2 :S I I 2 :S I 0 I t2 :S I 2' 0 I IS a --------1--------r--------r--------l--------r--------r----------------r--------l 1. I I I 0 I 0 I 0 I 2 I 0 I 6 I 2 I 2 I 12 ,.ltt.ttu.EfftNT "C:T I I I o I 0 I 0 I 16 7 I 0 I SO 0 I 16 1 I 16 7 I 38 1' ---------------1-------------J; 1 -I 1. 0 I I I . .. 0 f -t I I I 2 I t I 0 I 5 PUDCET AN,.LVIT I .0 I .o I 20.0 I 0 I 20,1) I 20.0 I 40.0 I o I .o J 16.1 OTNtlt -l------------------------l--------l--------l--------l------l--------1--------l 4. I I I 0 I 0 I 0 I t I I I t I 2 I I I 6 I 16.7 I .o I .o I 0 I I 16.7 I 16.7 I JJ,J I 16.7 I -l-------r--------r--------l------------------------1-------r--------r--------r COLU"N I I 2 t 4 3 9 S 5 3 t TOUL J,2 3.2 6.!o 3.2 12,9 29.0 16.1 16.1 100.0 UJ N .......

PAGE 343

328 CDST2 TECHNICAL UNCERTAINTY fcEL.:tTJYE ADJliSTED CUPI ltfiSOLLoTE FF:EQ HEw tATE,OF;'i' LA&EI. COE>E FR C F'CT) C F'C. T > ( ,_, T > HONE o. 3.1 '3. t 3. t 3. 4 12.S li.S IS.6 4. 3. t 3. t 18.8 5. 3 9.4 9.4 28.1 6. 5 tS.6 15.6 43.8 7. 8 25.0 2S.O 6e.e e. 6 u.e 18.8 87.5 9. 4 12.5 12.5 00." -----------------TOTML too. o too.o ftEAH 6.281 REDJAH 6.7SO RO[I 7.000 STD 4.6E-O RAHi;E 9.00[1 YALID tASES 32 C.:.SES 0

PAGE 344

COSTI COUNT I ow PCT INONE I 1 o.1 3.1 4 ,I 5.1 6.1 7.1 9 .I 9.1 ow TOTAL 1.1 II II l'l tl tl 31 tl II 8 PRII.IECY "CT I ,O I 12.5 I .o I 12.5 1 12.5 I 37.5 I U.S I U.S I 25.t' 2. I D I 2 I I I I I I I 3 I 3 I I I 12 "IIGCUr.MNT IIGT I .0 I 16.7 I 8.3 I 8.3 I 8.3 I 23.0 I 23.0 I 8.3 I 38.7 -l--------l--------1--------l--------l--------l--------l--------l-------J, I 0 I 0 I 0 I I I 2 I 0 I 0 I 2 I S PUDCET f'HAL 'IS T I 0 I 0 I 0 I 2 0 0 I 4 D 0 I 0 I 0 I 4 o 0 I 16 t ---------------------------------l----------------l--------1-------4.1 II tl 01 01 II 21 II 01 6 OTHER I 16 1 I 16 7 I 0 I 0 I t 6 7 I 33 3 I t6 7 I 0 I 19 4 -l--------l----------------l----------------1--------l--------l--------l COLU"" I 4 I l S 8 4 31 TOUL 3.2 12.9 3.2 9. 7 tt-.1 25.8 16.1 12.9 100,0 w N ID

PAGE 345

330 COST3 TECHNICAL DYAHCE"ENT ltELATJYE' RO JLIS TEO CU" R8SCILUTE n:ECI HE" FREQ CODE FrcEo < f'CT> < F-C. T ) < f'CT) HOHE o. 2 6.3 6.3 6.3 I 2 6.3 6.3 12., 2. 2 6.3 6.3 18.1 3. 6 18.8 te.e :J?.S -12.5 t2.S s. .. 12.S 6. 3 9.4 9.4 71.9 7. 6.3 6.3 76.' e. 2 6.3 6.3 ...... 9. 3 9.4 '" 93.e TOTj:jL.LY '0. 2 6.3 6.3 '0(1. 0 -----------------TOTAl.. I "fl. (I 100.0 ftEAN 4.844 ftEOIAH -.,0(1 "0[1E 3.0(10 STD I)E\1 2.864 YARIAHCE 1.201 aAHCE 1(1.00( YALID CASES 32 "JSSJHC CASES 0

PAGE 346

COST3 COUNT J ROW PCT JHOHE I I O. I 1.1 :! I 331 3.1 4. I s. J JOB t.J OJ OJ 21 J fo"ID"'ECT I .0 I .0 I 2:1.0 J I 2:.0 J 12.:1 J 2, I 0 I 2 I 0 I I I I I 2 I PROCURE"EHT "'' I .0 I 16.7 I .0 I I.J I 8.3 J 16,7 I -J--------------I-------J--------J--------J--------J 3. J Cl I 0 I 0 I 3 I 0 I I I IUOCET ANALYST J 0 I 0 J 0 I 60.0 I 0 I 20.0 I C CONT J NUEtJ > -J-------1--------I-------J--------J--------J--------J 4. I 2 I 0 I 0 I 0 I I I 0 J I 33.3 I 0 J 0 I .0 I 16.7 I 0 I 2 '' C:OSTJ COUNT I i' 6.S 2 '' 5 16, I 4 12.9 4 12.9 ROW f'CT I TOTALLY RuW I .TOTAL. I 6.1 7.1 8.1 !1.1 ICI.J --------I------I--------J--------J--------1--------J 1.1 II OJ Cll II OJ 8 JECT "CT I 12.:; I 0 I Cr J 5 I 0 I 25 E< -1-------J--------J--------l--------1--------J 2. J I I I J 2 J (I I i I Iii: "'T J 1.3 I I 16.7 1 .0 I 16.7 1 3. J e I 6 J Cl J 1 I 0 J 5 eUDCET ANALYST I I .0 1 .( J 20.0 J .I. J 1..1 -------J--------1--------1---------------J I I 1 OJ I I II GTHER I 16.7 I 16. 7 J 0 J 16 ? I I J 1 9 .. C:OLUitH I 3 2 2 3 2 3, TOTL 9.7 6.5 9.1 ICIO.O

PAGE 347

APPENDIX G CAUSES OF COST GIOWTB

PAGE 348

c-t 21. laadequate fadiq/fadill& ls.itatlna, caudq replun1m& ud nacblllluliq. J08CODE F>ROJECT CCIUUT 1 P.CT 1 I I' o. J 21.1 --------J--------1--------J J. .. J .. I ftGT J: so. 0 I so. (I J -1--------1--------J 96. J 2 I 10 I PtCiT J 16.7 J 83.3 I -1--------t--------I '' I I I .. I PUDGET J 20.0 I o. o l -1--------1--------I 98. J I s I OT-IEF' J 16.7 J 83.3 I -1-------1--------1 e ::I 25.8 74 .:! COUHT J R:uw PC.T J J J 0.1 i.?.J JOSCOtE --------1-------I--------I ,!5. I 4 I 4 I .. ROJEtT ft.:OT I 50.0 I Sl. 0 I -I--------J--------1 9t. J 7 I 5 I F-ROC'IIREftENT M.:OT I J 41 .7 l -I--------1--------1 97. l 3 J 2 J 8UtCET IU4AL .. 'ST J 60.0 I 40.(1 I -1--------J--------J '' I 2 I 4 J J 33.3 I 66.7 I -I--------1--------I COLU!'IrJ 16 1!5 10TAI.. 51.6 48.4 "Uft&ER OF "ISSJNC OBSERYATJONS e 25.EI ,;: 39.7 5 16. I 6 14i.4 Jl I (1(1, 0 fCIII TOT"L o 25.8 38.7 !5 16. I 6 U.4 31 106.0 333

PAGE 349

c-at 23. !banal PI'Otecttn IJat (t11ea) ..-ctf:lcallJ idatlfted u Uclmical COUNT I ---R"W fC. T I ROY I TOT.:tL J t.l --------l--------J--------1 ts. J s J 3 I 6 !o-ROJEtT "'T I u.s I 37.5 I -1--------l--------J 9. I 11) I 2 I 12 ft(;;T I &3.3 J If. 7 I 36.7 -J--------J--------1 9?. I 5 J 0 I s SULC!E'T IIHHL..YST J 00.0 J 0 I IE. I -J--------J--------1 9&. I 5 J J I 83.3 I 16.7 I .... -1--------1--------J COLUffN 25 '31 TOlAL 60.6 Ht.4 10(1.0 Ct:" ftiSSJHC. (lf:SEF.-Y.:ITJOHS c-at 24. OpUaiattc proar ,J.au ad acbUW.ea. COUHT I RCW f'c.T I IHt. I TOTNL I O. I ..... I JOBCOtt: I 7 I I e. f'ROJECT ftC.T I e7.s I I 2S.c -1--------1--------J J 12 I Cl I t2 fiC.T J 100.0 J .o I 3::0.7 -z--------1--------1 97. I 4 I I .. ., AHAL.VST II 10.0 I 2ll.(l I ,, -J--------1--------J 9e. I s J I I OTHER I 83.3 I 16.7 I ... --------1--------1 tOLU"H 28 3 TOTAL to.3 9.7 00. 0 NUrtEER OF fiiSSJHC. OiSERYATJOHS 3-34

PAGE 350

JO!COLIE !I'ROJECT COUHT J ROW Ptl 1 J I o. J -------1--------1--------1 I 7 J I "'T I 87.5 I l -1--------J--------J !16. J 8 I J F-lOC IJRE .. Er4T ftCoT l 66.7 J 33.3 l 97. 1 4 I I PUCICT IINHL'I'ST I eo. o J 2[1.(1 J -J--------J--------1 9S. J s I J OTiiEii J 83.3 J 16.7 I -J--------:r--------"1 ROW (I t:! s 16. t 6 COLU"H 24 7 3t TOTAL 77.4 OF OiSERYIITJOHS c:-t 26. .nuc ... t/product aad other coatract cb&Daea. .JOB COllE C.ROJECT COUNT 1 lcOIJ PCT I J I 0.1 --------I-------J--------1 '' J 6 I 2 I ft(.T J 7$.0 J 25.0 J -:r--------:r--------1 t6. I 8 I 4 I HT I U.7 I 33.3 I -J--------1--------J 97. I 3 I 2 J up en IIHNLYST J 60.0 I 4(1, (I I -J--------1--------1 91. I 4 J 2 I OTHEF' I t-6.7 J 3:!1.3 I COLU"H 21 I 0 TOTAL 67.7 32.3 NUttEU OF nss1 HCi OBSERYATIDHS P.C' e 12 38.7 s 16.' 6 ,, .. 31 woo 0 335

PAGE 351

.lOB CODE eDUHT' 1 ROw PC.T J J J O.J 27.1 --------1--------J--------J !15. J 7 J t J "CT J I?.S J u.s J -1--------J--------J !IE.. J t2 I 0 J F "'T I '"". 0 I .o I -J--------J--------1 97. J I J IUCT ANAL. 'r'ST J ao. o J eo. o I '-1--------J--------J 9&. I 6 I 0 I DT;.t-I 100.0 I 0 I -1--------J--------J COLUI'IN 26 TCITAL. 13.9 16.' 'TOT.OL e H: 38.7 16., 6 :!!1 too. o 336

PAGE 352

APPENDIX I ME'l'BODS NASA COULD lAVE USED TO lEDUC! COST GlOWl'B OF ORBITER PI.OJECT

PAGE 353

c-at 31. lc.l!WtJ of faDil/budat..b& for dnel.o,...at .. /1 budaettDa. J08t0[)[ "ltOJECT CDliHT I f'CT J I I o. J 31.J --------J--------J--------1 !tS. I s I 3 I MCT I f.2.S I 37.5 I -J--------1--------I 96. I s I 7 I "CT I 41.7' I !.3 I -I--------I--------1 97. 1 3 I 2 I JU()CET ANAL\'ST I 60.0 I 40.0 I -J--------J--------1 I 2 J 4 I CIT"4EP I 33.3 I 66.7 I -J--------1--------1 C'OLU"H IS 16 lOUL 48.4 51.6 OF "ISSJHC 6 25.8 12 3S.7 ''.' 19.4 31 '00. (j c-at n. qu&U.fJ dnalopM!Dt .,ac .. llefor 1'ftN!uct1011/ frea&e liD .. r17. COUHT I P.OW fot.l I R0\4 I TOTAL I 0. J 32. J JOe CODE --------J--------J--------1 J 5 I 3 I 8 FROJC'T "C.T I 62.5 I 37.!1 I -J--------1--------J 9. I '0 I 2 I ,.ROCURE .. EHT "'T J 83.3 I ". 7' J 36.7 -1--------J--------J '' I I 0 I 5 IUDCET ANALYST I 100.0 I 0 I '6.' -1--------J--------J 98. I J J f OT14E5\ I 13.3 J 16.7 J t9.4 -1--------J--------J tOLU"t4 2! 6 31 TOTAL 10.6 19.4 toli. D NUftEEP. DF "ISSIHC DBSERYATIOHS 338

PAGE 354

c.-nt 33. flace re Te8J10UiblUt7 cna CODttacton. S..e -.itorill&. JOEcCCitl F-ROJECl COUNT I IOU fCT l I J 0. I --------1--------J--------1 95. I 8 I 0 J "GiT I 1(10.0 I .o I -J--------1--------1 96. I 1(1 I 2 I lRE"Er4T "CT I 83.3 J H. 7 1 -1--------1--------1 ,?, I s I 0 1 f;U[G;ET At4HI..'r'ST I 1 DO. 0 J ,(I J -J--------1--------J 98. I s l 1 1 DHEI=: J 83.3 J tE- ., J -1--------1--------1 CCtllii'IN 2(1 3 TOTAL 90.3 9.7 HUI'IEl=: 0!" fU$SIHC OBSEP.YATIOH$ c-at J4. 'ft.abter CDDti'Dl of CODtract cbaD1e. CCUNT I RuU PCT 1 I J 0, I J4 I JOBtCtE" --------1--------1--------1 I I 3 I "CT I 62.! I 37.!5 J -I--------I--------1 J J 0 J "CT I too.o I (I I -1--------1--------1 9':9. I 2 I 3 I I'UI>CET ANALYST I fi.O I 60. (I I -1--------J--------1 te. I 4 I 2 I OT:iER J 66.7 I 33.3 I -J--------1--------J C'OLU"N 23 8 TOTAL 74.2 25.8 ROW TCTAL e 2,.8 12 3S.7 s 1 6 19.4 31 10(1.(1 f. (IIJ TCl.:CL. 8 12 5 I i. t (. u.c )1 100.0 339

PAGE 355

e-Dt 35. l!dld all produet1DD Orbiter. at .CS.. ud co &be MM duiiJl. etlliHT J RI)W PCT I RCi
PAGE 356

c-t J7. lu1ld a at.plu .,.ceuaft wltb lua ad ftlue. .JOiCO[lE ,.ROJECT COUNT J RC.Iol PCT I I J 0. I 37. I --------1---------------J 95. I 7 J J fiCT J 87.5 I u.s I -J--------J--------1 96. I 12 I 0 I P.RCICiJ&S'I'fE'NT fl(iT J .100.0 J 0 I J--------J--------1 97. I 5 I 0 I JliDCET ltNALVST I too. o I .o I -1--------l--------J 9e. I 6 I 0 I OTiofEIO: I ICIO,O I .o I COLLiftH 30 I TOT.:IL 96.8 3.2 NUnEER OF ftiSSIHC RCIJ e 25.8 12 3S.7 s ".1 6 19.4 31 100.0 c-ct Jl, bcatl CODtnct wald ban baa re eeen1naful to coat. COUHT I ROiol PCT I I TOTAL J 0.1 38.1 .JOBCO[t --------1--------1--------J 95. I 7 I 1 I e ftCT I 87.!S I t2.S I 25.8 -J--------1--------J 96. I 11 I 1 I t2 FROCUP.EfiEHT fiCT I 91.7 I 8.3 l 3&.7 -J--------1--------J I 5 I 0 I 8UDCET ltNAL'I'ST I 1 oo. 0 I .o I ".' -1-------J--------J 98. I 6 I 0 I OT'fEJic I uo.o I 0 I -J--------1--------J 'OLUftH 29 2 31 TOTAL t3.5 6.5 10(1. 0 MUfi'ER OF ftJSSINC DISERVATIDNS 341

PAGE 357

COUNT I 101.1 PCT J J I O.J 39. I JOB CODE ------------------------1 ,s, J 8 I 0 J foROJECT ftGT I tOO. 0 I .o I ,6. J II J I J II.RCIC:URE"'EHT Rr.T I 91.7 J 8.3 I ---------1--------J 9?. J 5 I 0 J IIUliCET .HAL'I'ST I tOO.O J .o J -1--------J--------1 98. J 6 J 0 I OT:.tER I 100.0 J .. o I ---------1--------J COLUftH 30 I TOTAL 96.8 3.2 NUftEER OF ftiSSIHG OSSERVATIOHS c-t 40. lettu utmate of project coat. JOB CODE !i'RCIJECT COLIHT I ROW Ptl I J I 0. I 40 .J !t5. I 7 J I ftCT I 8?.5 I I -J--------J--------1 96. J t' I J ,_aOCURf"fHT "CT I 91.7 I 1.3 I -I---.-----1--------J t7. J .. I I I IUPCET .HAL'I'ST I u.o I .20. (I I -1----------------J t8. I 6 I 0 I OT .. I 100.0 J 0 I COLUftH 28 3 TOTIIIL 90.3 9.7 MUftll. 0' RilliNG 08SERYATJOHS TOT14L 8 .25.8 12 38.7 5 ".' 19.4 31 tOO. 0 ROU TO TilL e .zs.e u 38.7 5 hi.' 6 19.4 31 100.0 342

PAGE 358

COUNT I ROU PCT I I 1: o.l 39.1 JOB CODE --------1--------1--------J ,s, I I 0 J ,.ROJECT ftCT I 100.0 J .o I -J---..---1--------J 96. I 11 I I I c.aotURE .. EHT RCT I ".? I 1.3 I PUDCET DT'4ER NU"EER OF JOBCO[IE -J--------1--------J 97. J s I 0 J ANALYST J, ICICI.O I .o J -1--------l--------J 98. J 6 I I J' too.o I I -J--------J--------1 30 TOTAL 96.8 3.2 IUSSJHC OBSERVATIONS COLNT 1 ROW PtT J I J o. J 40. J --------1--------J--------J 95. J 7 l I ltCT I 87.5 J ...... I -J--------1--------J 96. I 11 I 1 I PROCUREftEHT RCT I 91.7' I 8.3 I -1--------1--------J 97. I .. I 1 I uocET ANALYST I 10.0 I 20.('1 I ,e. I 6 I 0 J OT .. Efr J uo.o I .o I -J----------------1 COLU"N 28 3 TOTL o.3 9.7 HUftiU OF ftJSSIHC 08SUATIOHS ROW 8 25.8 12 38.? s 16.' 6 19.4 ,, 100.0 ROW TOTIIIL e 25.8 12 38.?' 5 .. 6 "" 31 100.0 343