Citation
The impact of state policy changes on superintendent evaluation in Colorado

Material Information

Title:
The impact of state policy changes on superintendent evaluation in Colorado
Creator:
Straface, George J
Publication Date:
Language:
English
Physical Description:
xvi, 194 leaves : illustrations, forms ; 29 cm

Subjects

Subjects / Keywords:
School superintendents -- Evaluation -- Colorado ( lcsh )
School superintendents -- Evaluation ( fast )
Colorado ( fast )
Genre:
bibliography ( marcgt )
theses ( marcgt )
non-fiction ( marcgt )

Notes

Bibliography:
Includes bibliographical references.
General Note:
Submitted in partial fulfillment of the requirements for the degree, Doctor of Philosophy, School of Education and Human Development.
Statement of Responsibility:
by George J. Straface.

Record Information

Source Institution:
|University of Colorado Denver
Holding Location:
|Auraria Library
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
25725759 ( OCLC )
ocm25725759
Classification:
LD1190.E3 1991d .S77 ( lcc )

Full Text
THE IMPACT OF STATE POLICY CHANGES
ON SUPERINTENDENT EVALUATION IN COLORADO
by
George J. Straface
B.A., Western State College, 1967
M.A., University of Colorado, 1973
A thesis submitted to the
Faculty of the Graduate School of the
University of Colorado in partial fulfillment
of the requirements for the degree of
Doctor of Education
School of Education
1991

J


This thesis for the Doctor of Education
degree by
George J. Straface
has been approved for the
School of
Education
by
tei- v
Date


Straface, George J. (Ed.D., Education)
The Impact of State Policy Changes on Superintendent Evaluation in
Colorado
Thesis directed by Associate Professor W. Michael Martin
The purpose of this study was to determine if the
performance of superintendents in selected Colorado school
districts changed as a result of state mandated policy changes
enacted in 1986.
Specifically, this study was designed to answer the
following questions: What has been the impact of policy changes
on superintendents' evaluation? To what extent has the
performance of the superintendent changed as a result of
implementation of policy changes? As a result of policy changes,
was there a difference in the accomplishments of the district's
written objectives? To what extent was the model, approach, or
type of superintendent evaluation system adopted by school
districts consistent with policy changes and accepted practices?
As a result of policy changes, what formal evaluator training
has been provided to board members? What was the attitude of
board members and superintendents regarding the credibility,
fairness, believability, and effectiveness of the model,
approach or type of system used to evaluate superintendent
performance? And finally, was there consistency in the model,


approach, or type of superintendent evaluation systems used by
selected Colorado school districts?
The study identified 15 school districts that had
superintendents whose tenure spanned the years prior to and
after 1986. Data were gathered by a structured face-to-face
interview with a board member and the superintendent from each
district. The survey instrument was designed and interviews
conducted by the researcher.
Findings from the study include: State mandated policy
changes had limited impact on superintendent performance.
Accomplishment of district goals and management of the
district's planning processes and fiscal resources improved as
the result of superintendent leadership rather than policy
changes. And superintendent evaluation systems were consistent
with policy changes and accepted practice.
Conclusions drawn from the findings include:
Superintendent performance has improved, but not as a result of
policy changes. There is greater emphasis on formative
evaluation processes than summative. Formal processes are in
place in all districts included in the study. And
superintendents and board members view the process as fair,
credible, believable, and effective.
The researcher further delineated recommendations for
additional research in the area of superintendent evaluation.
iv


The form and content of this abstract are approved. I recommend its
publication.
Signed
W/Michael''" Martin
v


DEDICATION
Without the special love and support of five very important
people, this project would never have been completed. It is
dedicated to them.
My two daughters, Lesley and Jenna, sacrificed time with their
father. I thank them for being there when I needed them. Their
mother, Kathryn, is my friend. She deserves significant recognition
for the completion of this project.
Ellen spent countless hours at the word processor putting my
scribbles into readable form. This document is the fruit of her
effort. I thank her for every hour and minute.
Lastly, I want to say thank you to my father, Pete. Without
his willingness to give of his time and sleep in order to drive me
from Grand Junction to Boulder every week for two years, the
completion of the required coursework would not have been possible.
Thanks, Dad.


CONTENTS
Tables......................................xii
Acknowledgements............................xvi
CHAPTER
1. INTRODUCTION ................................... 1
Purpose of the Study...........................5
Statement of the Problem ..................... 6
Significance of the Study......................9
Delimitations of the Study....................10
Limitations of the Study......................10
Definition of Terms...........................11
Organization of the Study.....................14
Specific Procedures...........................15
2. REVIEW OF THE LITERATURE........................18
Historical Perspective of the
Superintendency ............................ 18
Historical Perspective of Superintendent
Evaluation...................................20
Procedures of Superintendent Evaluation. . 26
Frequency of Superintendent Evaluation ... 37
Methods of Superintendent Evaluation .... 38
Summary.......................................44
3. DESIGN AND PROCEDURES...........................46
Problem Questions.............................48
Procedures....................................49
The Population................................52


I
Variables and Source Collection..............57
Statistical Treatment........................60
Summary......................................62
4. ANALYSIS OF DATA................................64
Introduction ............................... 64
Research Questions One and Two...............66
Survey Item 8............................67
Survey Item 9............................69
Survey Item lib..........................70
Survey Item 12b........................ 71
Survey Item 13b..........................73
Survey Item 14b. ........................74
Survey Item 21a..........................76
Survey Item 41...........................77
Research Question Three......................80
Survey Item 10...........................82
Survey Item 12...........................83
Survey Item 13...........................84
Survey Item 14...........................85
Survey Item 10a..........................86
Research Question Four .............. 89
Survey Item 11...........................89
Survey Item 11a..........................90
Survey Item lib..........................91
Research Question Five .............. 93
viii


Survey Item 22...........................94
Survey Item 23...........................95
Survey Item 24...........................95
Survey Item 24a..........................96
Survey Item 25...........................98
Survey Item 25a..........................99
Survey Item 26..........................101
Survey Item 26a.........................102
Survey Item 27..........................104
Survey Item 27a.........................105
Survey Item 21..........................107
Research Question Six........................110
Survey Item 17..........................Ill
Survey Item 17a.........................113
Survey Item 18..........................114
Survey Item 19..........................115
Survey Item 20..........................116
Survey Item 28..........................117
Survey Item 28a.........................118
Survey Item 29..........................120
Survey Item 29a.........................121
Survey Item 30..........................123
Survey Item 30a.........................124
Survey Item 31..........................126
Survey Item 31a.........................127
ix


Survey Item 32...........................128
Research Question Seven....................129
Survey Item 16..........................130
Survey Item 16a.........................131
Research Question Eight....................133
Survey Item 34..........................134
Survey Item 34a.........................135
Survey Item 35..........................137
Survey Item 35a.........................138
Survey Item 36..........................140
Survey Item 36a.........................141
Survey Item 37..........................143
Survey Item 37a.........................144
Research Question Nine.....................147
Survey Item 15..........................147
Survey Item 18..........................149
Survey Item 19..........................150
Survey Item 20..........................151
Survey Item 31d.........................152
Survey Item 33..........................154
Survey Item 33a.........................156
Survey Item 38..........................158
Survey Item 39..........................158
Survey Item 40..........................159
5. SUMMARY, CONCLUSIONS AND RECOMMENDATIONS . .162
x


Summary......................................162
Questions Posed in The Study.................162
Design of the Study..........................164
Summary of the Findings......................166
Synthesis of Findings with
with Related Research......................171
Conclusions..................................175
Recommendations..............................178
Recommendations for Future Study.............179
APPENDIX...................................................181
BIBLIOGRAPHY............................................. 191
xi


TABLES
Table
3.1. School districts in categories II and III with
those eligible for study specifically identified. . .55
3.2. Evaluation variable by study or source....................58
4.1. Impact of § 22-9-109, C.R.S...............................67
4.2. Impact of H.B. 1338 ......................................69
4.3. Impetus for accomplishing objectives......................71
4.4. Impetus for improving management of the district's
fiscal resources........................................72
4.5. Impetus for improving management of the district's
planning process........................................74
4.6. Impetus for improving management of the district's
personnel supervision and evaluation systems............75
4.7. Impetus for improvement...................................77
4.8. Benefit of public access to selected sections
of superintendent evaluation............................78
4.9. The overall performance of the superintendent
has improved............................................82
4.10. The management of the district's fiscal resources
has improved............................................83
4.11 The management of the district's planning process
has improved............................................84
4.12. The management of the district's personnel supervision
and evaluation systems has improved.....................85
4.13. Extent to which the overall performance of the
superintendent has improved.............................87
4.14. Formal district objectives have been accomplished ... .90


4.15. Subjective reasons why formal district objectives
were accomplished.......................................91
4.16. Objective reasons why formal district objectives
were accomplished.......................................92
4.17. Regular cycle for evaluation..............................94
4.18. Frequency of evaluation...................................95
4.19. Mutually identified specific criteria.....................96
4.20. Extent to which criteria have been mutually
identified..............................................97
4.21. Mutually agreed upon standards of expected
performance.............................................99
4.22. Method by which agreement is reached on standards
of expected performance .............................. 100
4.23. The sources used to gather data are predetermined . 102
4.24 Description of formal sources used to gather data . 103
4.25. There are formal methods established to gather
objective data concerning the superintendent's
performance............................................105
4.26. Descriptions of the formal methods utilized to
gather data............................................106
4.27. The evaluation included a plan for improvement...........108
4.28. Purpose..................................................112
4.29. Where purpose for the superintendent's evaluation
is stated..............................................114
4.30. The evaluation process is formal.........................115
4.31. The evaluation is written................................116
4.32. Key components of the evaluation system..................117
4.33. Self-appraisal...........................................118
4.34. Descriptive comments regarding self-appraisal .......... 119
xiii


4.35. Specific objectives established ........................ 121
4.36. Extent to which specific objectives have been
established...........................................122
4.37. Defined time period......................................124
4.38. Descriptive comments regarding defined time period. . 125
4.39. A specific board meeting is held.........................126
4.40. The type of meeting held.................................127
4.41. Where evaluation process is included.....................128
4.42. Formal evaluator training .............................. 131
4.43. Extent to which board members have received formal
evaluator training.....................................132
4.44. The process is fair......................................135
4.45. Reasons why the process is fair..........................136
4.46. The process is credible..................................138
4.47. Reasons why the process is credible......................139
4.48. The process is believable................................141
4.49. Reasons why the process is believable....................142
4.50. The process is effective.................................143
4.51. Reasons why the process is effective.....................145
4.52. Models, approaches, or types of evaluation systems
used to evaluate superintendent performance .......... 149
4.53. The evaluation process is formal.........................150
4.54. The evaluation is written................................151
4.55. Key components included in the evaluation system. . 152
4.56. Method by which evaluation is disclosed ................ 153
4.57. Process board uses to reach consensus .................. 155
xiv


4.58. Provisions for minority opinion are included.............157
4.59. Relation of evaluation to contract extension............158
4.60. Relation of evaluation to salary increases..............159
4.61. Relation of evaluation to benefit increase..............160
xv


ACKNOWLEDGEMENTS
I sincerely wish to acknowledge and express my deep
appreciation for the cooperation, guidance, and encouragement that I
have received from my family, friends, teachers, and associates.
Without them and their care, this study would not exist.
Mike Martin, Myrle Hemenway, and Bob Taylor have demonstrated
outstanding patience and support throughout this entire project. I
know there were times they may have felt it was not to be, but they
continually encouraged me to finish.
I must recognize David McElroy for his assistance. I thank him
for giving of his time and statistical expertise to help me. Ken
Vedra and Mike Massarotti were instrumental in my completing this
project. Without them to help me to understand what I was doing and
to focus my efforts, it would not have been completed. I am deeply
indebted to them.
I also want to acknowledge the financial support of The Cherry
Creek School District. The dollars I received from scholarships and
my professional growth fund substantially eased the financial burden
of this project.
Lastly, this study is a result of the support and loyalty of
many friends. They have encouraged me and most of all understood
when I could not be with them. I thank them for being there.
xv i


CHAPTER 1
INTRODUCTION
When the National Commission on Excellence in Education
released A Nation at Risk: The Imperative for Educational
Reform in April of 1983, public education in Colorado was not
unlike that found in most states. Local school districts were
charged by the state constitution with the responsibility for
the education of the children of the state. The local districts
were functioning with increasing state control to offer a
comprehensive educational program for all children in grades
K-12.
The sweeping reforms called for in A Nation at Risk were
intended to improve the quality of education in America's
public schools. The focus of this report was accountability,
with emphasis placed on the decline of the teaching process
itself:
We conclude the declines in educational performance are in
a large part the result of disturbing inadequacies in the
way the educational process is often conducted, (p. 12)
The Colorado Legislature responded to this report, and
others, with specific legislation designed to improve the "way
the educational process is conducted" by focusing on the
evaluation of the performance of the certified personnel in its
schools. The Certified Personnel Performance Evaluation Act was


signed into law in May, 1984. This act is commonly known as
House Bill 1338 (H.B. 1338, 1986.)
This law might best be seen as a series of minimal fair
process and due process guarantees. It established a
framework of procedures for the evaluation process, while
leaving specific decisions as to process and methodology to
the discretion of local districts. (Conley, 1986)
While certainly the primary intent of the legislature, when
it passed H.B. 1338, was to focus on the evaluation of the
performance of teachers, it also encompassed all administrators,
including superintendents, under its provisions.
As districts began to implement H.B. 1338, the question
arose as to whether the specific contents of any evaluation
required by H.B. 1338 was open to the public or was it
confidential information? Teacher associations, administrator
associations, and school districts lobbied to ensure the
confidentiality of these evaluation reports. The result was
that the legislature, in 1986, enacted Section 1, Article 9 of
Title 22, Colorado Revised Statutes (§ 22-9-109, C.R.S.). This
law made all evaluations exempt from public inspection except
for the evaluation of the CEO (Superintendent) of any school
district in four specific areas:
...as it relates to the performance of the CEO in
fulfilling the adopted school district objectives,
fiscal management of the district, district planning
responsibilities, and supervision and evaluation of
district personnel, shall be open for inspection by any
person at reasonable times. (§ 22-9-109, C.R.S.)
2


In addition, in 1988 the Colorado Legislature completely
revised the School Finance Act of 1973. This was accomplished
through proposed legislation commonly referred to as House Bill
1341 (H.B. 1341, 1988), and finally incorporated into State
Statute as Colorado Revised Statute (§ 22-53-101 et seo.) The
major focus of the act was to provide funding for K-12 public
education in the state in a more equitable manner. As an
integral part of the Act's mechanism for funding, each of the
state's 176 public school districts was categorized into one of
eight different categories.
These categories were identified as follows:
Category I Core City Denver
Category II Denver Metropolitan Area
Composed of districts located within the Denver-Boulder
standard metropolitan statistical area which are
primarily suburban in nature.
Category III Urban Suburban
Composed of districts which comprise the state's major
population centers outside of the Denver metropolitan
area and their immediate surrounding suburbs.
Category IV Outlying City
Composed of districts in which most of the pupils live in
population centers of seven thousand persons or more but
less than thirty thousand persons.
3


Category V Outlying Town
Composed of districts in which most of the pupils live in
population centers in excess of one thousand persons but
less than seven thousand persons.
Category VI Rural
Composed of districts with no population centers in
excess of one thousand persons and is characterized by
sparse widespread populations. These districts do not
meet enrollment criteria for Category VIII small
attendance.
Category VII Recreational
Composed of districts which contain major recreational
developments that impact the cost of property values,
community income and other cost-of-living components.
Category VIII Small Attendance
Comprised of districts which are rural in nature and have
enrollments of less than one hundred fifty persons.
(§ 22-53-102 et sea.I
Over 90% of the students in ^the state are represented by
those districts in Categories I, II, and III. Because Category
I is descriptive of only one district, Denver, the researcher
focused the investigation of this research on those districts
included in Categories II and III only. The districts, in these
two categories, represented over 70% of all students enrolled in
K-12 public education in Colorado.
4


Conley (1986) conducted a study to assess the impact of
H.B. 1338 (1986) on policies and practices of local school
districts related to teacher evaluation. His study did not
investigate the impact of either H.B. 1338 or § 22-9-109, C.R.S.
on the evaluation of superintendents.
Assuming that Estes (1986) was correct in his assertion
that:
The superintendent of schools has to be described as the
person who, more than anyone else in the community, influences
the quality of public education, (p. 1)
The question arises as to what has been the impact of H.B.
1338 and § 22-9-109, C.R.S. on the performance of superintendts
in those districts included in Categories II and III of
§ 22-53-102 et sea.
Purpose of the Study
The purpose of this study was threefold. The first was to
determine if superintendent performance had changed as a result
of the implementation of § 22-9-109, C.R.S. and H.B. 1338 in
selected school districts included in Categories II and III in
§ 22-53-102 et sea. (H.B. 1341). The second was to determine if
the models, approaches or types of evaluation systems used in
the selected districts included in Categories II and III in
§ 22-53-102 et sea, were consistent with selected requirements
of H.B. 1338 and accepted practice. And, lastly, to assess the
attitudes of the superintendents and board members of these
5


selected school districts toward § 22-9-109, C.R.S. and H.B.
1338 as well as the evaluation model, approach, or type used to
evaluate the performance of the superintendent in their
respective district.
Statement of the Problem
The problem identified in this study addressed the impact
of state mandated policy changes enacted in 1986 on
superintendent evaluation in selected school districts in
Colorado as of winter 1990.
Questions arose as to the impact of § 22-9-109, C.R.S. and
H.B. 1338 on the performance, training, and evaluation of
personnel and school board members. The implications of these
relationships were:
1. If Colorado school districts in Categories II and III
of H.B. 1341 (1988) evaluate the performance of the
superintendent, then the possibility existed that districts used
a combination of informal and formal processes to evaluate the
superintendents' performance.
2. If Colorado school districts in Categories II and III
of H.B. 1341 (1988) evaluated the performance of the
superintendent, then the possibility existed that districts used
both formative evaluation (evaluation undertaken to improve or
redirect individual performance toward the desired outcomes) and
summative evaluation (evaluation undertaken to make final
6


judgement regarding performance over a prescribed period of
time) processes to evaluate the superintendents' performance.
3. If Colorado school districts in Categories II and III
of H.B. 1341 (1988) evaluated the performance of the
superintendent, then the possibility existed that districts had
a common purpose for the evaluation of the superintendents'
performance.
4. If Colorado school districts in Categories II and III of
H.B. 1341 (1988) evaluated the performance of the
superintendent, then there existed the possibility that there
were no established district procedures to measure the
superintendents' performance.
The questions arising from the problem statement which were
to guide the research design and methodology were summarized as
follows:
1. What has been the impact of § 22-9-109, C.R.S. (1986)
on superintendent evaluation in selected Colorado school
districts?
2. What has been the impact of H.B. 1338 (1986) on
superintendent evaluation in selected Colorado school districts?
3. To what extent has the performance of the
superintendent changed, comparing the performance of the
superintendent prior to July 1, 1986, to the current performance
of the superintendent, as a result of the implementation of
§ 22-9-109, C.R.S. and H.B. 1338?
7


4. As a result of the implementation of § 22-9-109, C.R.S.
and H.B. 1338, was there a difference in the accomplishments of
the selected Colorado school districts' written objectives prior
to July 1, 1986 and the present?
5. To what extent was the model, approach, or type of
superintendent evaluation system adopted by the school district
consistent with H.B. 1338?
6. To what extent was the model, approach, or type of
superintendent evaluation system adopted by the school district
consistent with accepted practices of superintendent evaluation
as identified in the literature?
7. As a result of the implementation of § 22-9-109,
C.R.S.and H.B. 1338, what formal evaluator training to evaluate
superintendents has been provided to board members of selected
school districts in Colorado?
8. What was the attitude of a board member and the
superintendent of selected Colorado school, districts toward the
credibility, fairness, believability, and effectiveness of the
model, approach, or type of evaluation system the district used
to evaluate the current performance of the superintendent?
9. Was there consistency in the model, approach, or type
of superintendent evaluation system used by selected Colorado
school districts?
8


Significance of the Study
The impact of policy changes enacted by the Colorado
Legislature on the evaluation of the performance of
superintendents in the State of Colorado has not been
investigated. This study:
1. Measured the impact of § 22-9-109, C.R.S. and H.B. 1338
on the superintendent's performance as perceived by Board of
Education members and superintendents of selected school
districts in Colorado. Specifically, the study focused on the
performance areas identified in § 22-9-109, C.R.S. These were:
- the accomplishment of district objectives,
- the management of fiscal resources,
- the management of the planning processes, and
- the management of the personnel supervision and
evaluation systems.
2. Provided data on the changes made to the models,
approaches, or types of evaluation systems used in the
evaluation of the performance of superintendents in selected
districts in Colorado as a result of the implementation of § 22-
9-109, C.R.S. and H.B. 1338.
3. Assessed attitudes of members of Boards of Education
and superintendents toward the model, approach, or type of
evaluation system used to assess the performance of the
superintendent within selected Colorado school districts.
9


Delimitations of the Study
The districts selected for inclusion in this study were
those districts identified in § 22-53-102 et sea. (H.B. 1341) as
Categories II and III whose superintendent's tenure began prior
to July 1, 1986. The only state mandated policy changes
investigated in this study were CRS, Article 9, Title 22, and
H.B. 1338 (1986). Respondents to the interview guide were
superintendents and current members of the Boards of Education,
having primary responsibility for the evaluation of the
superintendent. The board members also must have served in
their respective positions prior to July 1, 1986, to the
present. The study was influenced by the reliability of the
interviewer conducting the interview.
Limitations of the Study
The study utilized ex post facto research. The findings
from the study may not apply in school districts in states
outside of Colorado nor to Colorado school districts in
Categories I, IV, V, VI, VII, and VIII of § 22-53-102 et sea.
(H.B. 1341, 1988). The responses of the interviewees to the
interview questions may be inconsistent. The responses of the
interviewees may be influenced by the sensitive and confidential
nature of evaluation. The results of the study may represent
the impact of § 22-9-109, C.R.S.and H.B. 1338 in Colorado as of
10


winter 1990. The results in the study may be impacted by the
memory of the parties interviewed. The findings of the study
may be limited to the lack of access to previous evaluations.
The interview guide was self-reporting, and its validity was
limited by the honesty, accuracy, recall, and clarity of the
respondents. The findings of the study were limited by the
problems inherent in any interview guide. The results of this
study presented the status of the superintendent evaluation in
selected school districts in the State of Colorado as of winter
1990. These practices continued to evolve over time.
Definition of Terms
The following definitions, as developed by the researcher
and referenced in H.B. 1338, provided clarification of the terms
used within this study:
Board of Education The duly elected body charged by law
with the responsibility of establishing school
district policy to be administered by the
superintendent.
Believabilitv Perception of selected individuals as to
the "professional quality" of the evaluation system.
Change The process by which an attitude, behavior or
situation is altered, modified or made different,
either in the positive or negative sense.
11


Credibility (Referenced in H.B. 1338)
The evaluation system measures those areas of the
superintendent's performance that the Board of
Education has defined as important and reflects
attitudes of those the superintendent works with as to
their feelings of credibility.
Effectiveness (Referenced in H.B. 1338)
The evaluation system will cause positive change in
the performance of the superintendent. This positive
change is determined by the Board of Education in
relation to the goals and objectives they have
established for the superintendent's performance.
Evaluative Criteria -The standards by which performance is
evaluated.
Evaluation A systematic procedure for collecting
information, based on predetermined objectives and/or
criteria set at the local level from identifiable data
sources which include provisions for the analysis and
sharing of that information with the evaluatee.
Fairness (Referenced in H.B. 1338)
The precepts of due process are included in the
superintendent's evaluation process, e.g., clear
criteria provided to the superintendent, specific
observations with feedback, suggestions for
12


improvement, and appropriate time to implement
suggestions are given.
Formal Evaluation A planned and structured evaluation
procedure which usually includes: a written job
description, criteria, scheduled dates for
feedback, and a written final evaluation report which
is shared with the evaluatee.
Formal Evaluator Training Planned workshops, classes or
instructional modules which are focused toward the
precepts of effective evaluation techniques, methods
and models in which school board members may
participate.
Formative Evaluation Evaluation undertaken to improve or
redirect individual performance toward the desired
outcomes.
Impact A positive or negative change in the performance
of the superintendent as perceived by the
superintendent and board members, and/or the attitude
of the superintendent and board members toward the
evaluation process.
Informal Evaluation An unplanned and unstructured
evaluation procedure which is usually spontaneous,
with little detail, does not include predetermined
criteria or consistent procedures.
13


Job Description The written document which outlines the
duties of the superintendent of schools.
Performance The act, process or behavior related to the
degree of accomplishment of an assigned task,
responsibility or expectation.
Professional Quality (Referenced in H.B. 1338)
References an arbitrary measure of the quality of the
evaluation system of certificated staff
(superintendent) as measured by the individual
district's Certificated Personnel Performance
Evaluation Council.
Summative Evaluation Evaluation undertaken to make final
judgement regarding performance over a prescribed
period of time.
Organization of the Study
The investigation was conducted as an ex post facto
descriptive case study employing an interview guide developed by
the researcher and based upon relevant research. The interview
guide, developed by the researcher, focused on current
administrative and superintendent evaluation practices and
processes as defined in literature and requirements as outlined
in H.B. 1338 (1986).
Chapter 2 provides a review of previously completed
studies in the area of superintendent evaluation.
14


Research design and methodology are reported in Chapter 3.
Included in this chapter are the quasi-independent variables
with specific areas of interest described under each.
Findings of the study are presented in Chapter 4. Data
that were derived from the research are categorized to simplify
the reading and interpretation of results.
Chapter 5 identifies and summarizes the findings of the
study, as well as provides conclusions and recommendations.
Specific Procedures
The school districts studied were those districts included
in Categories II and III of § 22-53-102 et sea, whose
superintendents were employed prior to July 1. 1986. The
respondents were selected board members who began their service
prior to July 1, 1986, and their respective superintendents.
They were interviewed using a research based developed guide.
The interview guide was based upon research findings and
educational literature on the topic of superintendent
evaluation. A panel of experts was established to review the
guide. The panel was composed of Dr. Gerald Difford, Executive
Director of The Colorado Association of School Executives; Dr.
Randall Quinn, Executive Director of The Colorado Association of
School Boards; Dr. David Conley, Executive Director of
Curriculum for The Poudre R-l School District in Ft. Collins and
15


a recognized expert on H.B. 1338; and Dr. Richard Koeppe,
Superintendent of The Denver Public Schools. The members of
this panel were asked to review the interview guide,
independently, with respect to its: (1) completeness, (2)
length, (3) clarity of questions, (4) the intent of questions,
(5) appropriateness of questions, (6) forming of questions, and
(7) ability to discriminate in answers. Following the panel's
initial review, corrections were made to the interview guide.
Following the revisions of the interview guide, it was field
tested with Dr. Michael Massarotti, Superintendent of Adams
County School District #50. Dr. Massarotti was not eligible to
participate in the study, but was representative of
superintendents included in Category II. Dr. Massarotti's
response to the interview guide was evaluated in terns of his
ability to respond to the interview guide's specific items. In
addition, Dr. Massarotti was asked for suggestions for
improvement regarding items one through seven on page 15. The
final revision of the interview guide was reviewed for clarity
and accuracy prior to beginning the individual interviews. The
content of the structured interview focused on the assessment of
board member's and superintendent's attitudes toward the degree
of change (impact) in the superintendent's performance as a
result of the process implemented due to the mandated policy
changes required by § 22-9-109, C.R.S.and H.B. 1338. The
researcher conducted the interviews.
16


The researcher has held the positions of Director of
Personnel Services and Executive Director of Human Resources in
Mesa Valley County School District #51 in Grand Junction,
Colorado, and in Cherry Creek School District #5 in Englewood,
Colorado, for a total of 11 years. The researcher was trained
extensively in the interview process, has conducted literally
thousands of interviews, and has been a consultant on
interviewing for organizations such as The Colorado Association
of School Executives, Western State College, Mesa State College,
Metropolitan State College, Regis College, University of Denver
and The University of Colorado. The researcher can be
considered a recognized expert in interviewing techniques.
17


CHAPTER 2
REVIEW OF LITERATURE
This review of literature related to the evaluation of the
superintendent focused on the following areas:
- Historical Perspective of the Superintendency
- Historical Perspective of Superintendent Evaluation
- Procedures of Superintendent Evaluation
- Frequency of Superintendent Evaluation
- Methods of Superintendent Evaluation
Historical Perspective of the Suoerintendencv
The position of local superintendent was originated by
school boards as an extra legal position to help meet demands
which the boards were unable to satisfy by themselves (Grieder,
Pierce, & Rosenstengel, 1961). Boards first employed
superintendents without statutory authority, but relied on
implied authority. Court cases strengthened the common law
status of superintendent. One such case Stuart vs. The School
District #1 of the Village of Kalamazoo in 1874, established the
common law principle that, in the absence of enabling
legislation, the local school board had the implied power to
employ a superintendent of schools and pay his salary out of
public funds. Since this landmark case, states began to


recognize the legitimacy of the superintendency, and
legislatures have specifically authorized the position of
superintendent.
Public school districts were organized by state statute and
were governed by lay boards. The school boards hired
professional educators as their superintendents and together
they assumed the responsibility for the operation of their
districts. A board traditionally developed policy, and the
superintendent executed that policy. While state statute often
set forth the minimal responsibilities as a superintendent,
they usually did not convey authority. Authority was conveyed
(delegated) by the school board, thus, the school board could
not avoid responsibility for how its powers were used or misused
by saying the superintendent was responsible.
Although the board holds all final authority regarding
school operations, the board does not exercise it fully; and,
therefore, boards increasingly have granted more authority to
the superintendent as school administration has become more
complex and involved. The school board and superintendent are
together accountable to the public for the educational program.
This concept of school board responsibility is important
in the way it affects the board/superintendent relationship.
When the superintendent is delegated insufficient authority,
the board will have a weak superintendent. On the other hand,
if the board delegates authority to the superintendent with no
19


guidance on how to use it, and never checks to see that he or
she is using it responsibly, then the superintendent probably
has too much authority. A good relationship for the board and
the superintendent to strive for is one in which both parties
understand their respective rights and duties (Booth & Glaub,
1978).
Today, performance appraisal of the superintendent of
schools is receiving greater coverage in the professional
literature than it has in previous years. There has been
considerable interest shown in the literature since the mid
1960's, which is partially due to the accountability movement in
education. Most school boards today realize that they cannot
account to the public unless they can successfully measure the
performance of teachers, administrators, the superintendent, and
the educational program. The Board of Education, in order to
ensure accountability, should start their evaluation process
from the top down, i.e., superintendent, administrators, and
then the teaching staff. Superintendent performance, which was
once judged purely by subjective means, is in a state of flux as
objective ways to evaluate are being investigated.
Historical Perspective of Superintendent Evaluation
Past practices in evaluating superintendents have been
something of an eclectic patchwork of techniques and procedures.
The practice of informal, unwritten evaluations of the
20


superintendent's performance prevailed for a long time (Redfern,
1980). Carol (1972) reported that 3% of the 207 districts
participating in a study in New Jersey and New York, indicated
they used formal procedures to evaluate their chief school
officer; 63% used informal procedures; 5% used a combination of
formal and informal procedures; and 29% did not have any
procedures to evaluate their chief school officer.
Formal evaluation of the superintendent's performance is a
relatively new phenomenon. A major research effort in the
evaluation of the superintendent was conducted by Griffith
(1952). In his study, he attempted to determine the attitude of
school board members evaluating the superintendent. Griffith
concluded that 82% of the boards had no method available to
evaluate the superintendent, and that there was an expressed
need for an instrument of evaluation.
Since the late 1960's, the related literature has
increasingly focused on the evaluation of school administrators.
This emphasis was caused, at least in part, by the
accountability movement. The first reproduced superintendent
evaluation instrument found by this researcher was described by
Kibbey (1965) in the August, 1965, School Management Magazine.
The evaluation instrument was used in some California school
districts. This form was developed by the Association of School
Administrators and later was adopted by the Washington
Association of School Administrators. Carol (1972) was cited as
21


the benchmark in all of the studies on superintendent evaluation
conducted since 1980. This study: (1) identified the status of
the evaluation procedures for chief school administrators in two
populous states, New York and New Jersey, (2) investigated
existing practices, (3) determined the extent to which
formalized systems of appraisal had evolved, and (4) assessed
the desires of school boards to develop- such procedures.
Data collected by Carol (1972) confirmed her original
premise that the state of the art of superintendent evaluation
was very primitive. Her conclusions based on 207 responses
follow: six (3%) used formal procedures only, 129 (63%) employed
informal procedures, 11 (5%) used a combination of formal and
informal procedures, and 61 (29%) did not have any procedure at
all. There was not any significant relationship between a
district student population and the application of formal
evaluation in districts ranging in size from 620 to 21,500
pupils.
Prior to the 1970's, there was little mention of
superintendent evaluation in the literature. Campbell (1971) in
a session of the American Association of School Administrators
Annual Convention, noted that administrative evaluation was a
tool the profession could use to police itself and upgrade
itself in order to serve the larger society. The Educational
Research Service (1972) published their Circular No. 6 in
November, entitled Evaluating the Superintendent. Their thorough
22


research of the related literature revealed that only four other
articles had been published on the subject at that time. The
early authors included Boyd, 1966; Moffatt, 1967; McCarty, 1971;
and Turner, 1971. Prior to this, The American Association of
School Administrators provided some very general criteria for
school boards to use in evaluating their superintendents in
their 24th yearbook.
Writing in the foreword to the Evaluating Superintendents
and School Boards (1976), Educational Research Service, Inc.,
Robinson addressed the fact that while much attention had been
given to the development of effective procedures for assessing
student, teacher, and administrative performance, comparatively
little effort had been given to the development of effective
procedures for objectively evaluating the performance of the
school superintendent.
Sullivan, Wogaman, & Borshay (1971) stated that evaluation
had been one of the most neglected aspects of education. They
stated that it was urgent that educators develop systems of
evaluation that would lead to improvements in performance and
ensure the public that the schools were doing the best job
possible.
The 1970's witnessed a substantial decrease in student
achievement scores. As a result, the education program came
under close scrutiny by the public. The superintendent was in
the public eye defending the educational programs. The
23


superintendent was being held accountable to the board and, in
turn, the board to the public for student achievement. Boards
began as Heller (1978) stated,
...to realize that they cannot account to the public
unless they have some method to assess the performance
of teachers and school administrators, along with an
evaluation of the educational program. From the board's
perspective, accountability, i.e., evaluation, must begin
with concentration on the school superintendent, (p. 3)
As public pressure on the educational program increased,
boards held superintendents more accountable for program output.
As a result, superintendents became more actively involved in
the evaluation process. Since the evaluation had much to do
with the success or failure of a superintendent, the American
Association of School Administration (AASA) and The National
School Boards Association (NSBA) proposed that the evaluation be
part of the contract of the superintendent. In a joint
publication, Evaluating the Superintendent (1980), The AASA
and NSBA stated,
At the time a superintendent is employed, it is important
to discuss the method that will be used to assess
performance. In fact, a provision should be included in
the contract clarifying how evaluations will be
conducted. Today, more and more superintendents and
boards are insisting on clarification of evaluation
procedures at employment time. (p. 24)
McCarty (1971) indicated that the superintendency was
difficult to fill and keep filled. He argued that although a
systematic evaluation procedure alone may not bring effective
administration to the schools, without it "...the educational
24


leadership we all so desperately want will hardly be assured"
(p. 68).
Rosenberg (1971) judged the administrative appraisal
systems, then in existence, as being "woefully inadequate and
unfair," and felt that they were based on "unacceptable evidence
collected with undesirable methods from undesirable sources"
(p. 213). It has been 20 years since Rosenberg's opinions were
published, and judging from the large number of school boards
that continue to utilize check lists, rating scales, and other
subjective methods of evaluating administrators, his assessment
may essentially still be true.
Evaluation was a necessary and desirable feature of any
organization. Drucker (1980) stated, "There is a need to
appraise management." Management groups such as the American
Management Association indicated to their members that
performance appraisal was absolutely necessary (Median, 1981).
Education was no less important than any other endeavor in
American life. Managers in education, including superintendents
of schools, should be evaluated. In 1984, public attention in
America focused on education. The National Commission on
Educational Excellence and its A Nation at Risk (1983) report
ushered in a host of reports and publications dealing with
effective schooling. Goodlad (1979) proposed in What Schools
Are For that "superintendents must be held accountable for a
good many responsibilities best classified as management"
25


(p. 96). Superintendent evaluation, then, was a part of this
national attention directed toward quality education in America.
After all, if the single most important task of the school
board was choosing the superintendent, common sense demanded
that the second most important task was to direct and shape
his/her performance. Evaluation helped achieve this goal.
Although the percentages of districts conducting formal
evaluations seemed to be on the rise, recent studies reflected
only up to 50% of the districts reporting some degree of formal
evaluations (Cunningham & Hentges, 1982).
Procedures of Superintendent Evaluation
There were two broad categories of evaluation practices
that were currently being used to evaluate superintendents.
They were usually referred to in the literature as formal or
informal. There, of course, also was a combination of both. In
Evaluating the Superintendent (1972), Educational Research
Service, Inc., 55% of 1,954 responding school systems reported
using a formal procedure for evaluating the superintendent. The
author of this circular stated "the larger the school system,
the more likely it is to have an evaluation of administrators
and supervisory employees" (p. 1).
The informal evaluations which were unplanned and
unstructured, have been utilized by boards for decades. While
this type may still serve some districts adequately, this
26


informal type of evaluation has its limitations. The main
difficulty, according to Evans (1979), was that the informal
evaluation was most often done as an outgrowth of a crisis
situation. Under this condition, the purpose of the evaluation
was usually linked to the current circumstances, rather than the
total responsibilities of the superintendent and to the school
district's goals. When this type of emotional climate existed,
it significantly reduced the chance for a thorough and valid
evaluation.
While it was obvious that an informal evaluation procedure
was based on subjective observations with no written feedback,
and limited, if any discussion, a formal evaluation involved a
written assessment of the superintendent's job performance
discussed in a meeting between the superintendent and the Board
of Education. Some school boards and superintendents may have
used a combination of formal and informal procedures.
Formal evaluations, those that were planned and structured,
were becoming more prevalent, and while there was a good deal of
variation in formal plans, two approaches stood out. As
reported by Evans (1979), they were evaluation through the use
of rating scales, and evaluation based on the development of
performance objectives. The performance objectives approach to
personnel evaluation was a procedure in which the superordinate
(supervisor) and the subordinate (supervisee) agreed on the
appropriate educational objectives for the subordinate, proposed
27


strategies for achieving them, set up activities to accomplish
the objectives and periodically evaluated the progress made
toward achieving the objectives. This method of formal
evaluation was sometimes referred to as "job improvement
targeting" or "management by objectives."
Both the National School Board Association and the
American Association of School Administrators recommended a
formal approach to the evaluation of superintendent. Redfern
(1980) stated:
For too long, superintendent evaluation was extremely
informal. "So long as everything is okay, you won't
hear from us." Never sure how the school board viewed
his (her) performance, the superintendent could not
help but feel a twinge of insecurity. Gradually it
became obvious that there were definite advantages to
systematic written evaluations, (pp. 1, 8)
Fitzwater (1973) also echoed the need for formal
evaluation procedures if the evaluation was to be a positive
activity of a forward looking nature. Dickinson (1980) agreed
with that position:
Casual, unspecified evaluations of a superintendent
doesn't [sic] work. They don't head off
misunderstandings that develop between a board and its
chief executive officer and they don't facilitate
efficient conversion of board policy in the school
system practice. What you need is an evaluation
process that is formal, specific, and structured--and
one that follows a set time table, (p. 34)
School boards implemented both formal and informal
superintendent evaluation programs. In an informal evaluation
of the superintendent, the board usually provided the
28


superintendent with a verbal appraisal of his/her performance at
a regularly scheduled board meeting.
The Educational Research Service, in its publication
Evaluating Superintendents and School Boards (1976), identified
the following as informal evaluation procedures:
1. General discussions about the superintendent's
performance held at private meetings of board members.
2. Special meetings of Boards of Education that were
called because of the dissatisfaction with some or all
aspects of the superintendent's performance.
3. Evaluations that take place continuously through
constant association with the superintendent and
through informal feedback from the community.
4. "Open-ended" discussions among board members that
include a wide range of school related topics, (p. 8)
In writing about informal evaluation procedures, the AASA
and NSBA (1980), concurred that:
This approach is likely a common practice in many school
systems, this method probably works when things are going
well and there is continued continuity in the
superintendency. It is also reasonably satisfactory in
those instances where board-superintendent relations are
cordial and reasonably stable. On the other hand, to
rely exclusively upon oral understandings involves many
risks. Different persons hear things differently.
Memory of what was said is less than dependable, (p. 18)
However, the informality of this evaluation approach was
fraught with potential problems such as lack of documentation of
performance for future references, disagreement of the
procedures to be followed, and the possibility of infrequent or
unscheduled evaluation sessions. The consensus of research at
29


that time was that a formal, structured, mutually agreeable
evaluation program be put in writing.
Halpin (1966) discussed his perception of the formal
evaluation process by stating, "accurate and judicious
evaluation of an individual's performance admittedly involves a
more complex process than a straight forward description of what
he does or how he behaves" (p. 112).
In her study of New Jersey's superintendents and boards,
Carol (1972) found that 62% of the districts responding used
informal rather than formal evaluation procedures. Sixty-five
percent of the superintendents and board presidents in those
districts expressed a desire to formalize their procedures.
Jones (1979) found similar results seven years later when he
also surveyed superintendent evaluation practices in New Jersey.
Sixty-four percent of the superintendents and 75% of the board
members responding were still using informal, verbal appraisals.
When asked for suggestions to improve existing evaluation
processes, both superintendents and board presidents listed
formalizing the process as the top priority for improving the
process.
In another study conducted in California involving school
districts that reported they formally evaluated their
superintendents, only 43% of those districts could actually
produce a written document (McGrath, 1971).
30


The Educational Research Service, in Evaluating the
Superintendent (1972), suggested these essentials be included
in the formal evaluation of the local district superintendent:
1. The superintendent should know the standards against
which he will be evaluated. Better yet, he should be
involved in the development of those standards.
2. Evaluation should be at a scheduled time and place,
with no other items on the agenda, at an executive
session with all board members present.
3. The evaluation, if written, should be a composite of
the individual board member's opinions, but the board
as a whole should meet with the superintendent to
discuss it with him.
4. The evaluation should include a discussion of strengths
as well as weaknesses.
5. Evaluation should be fairly frequent--at least once a
year, but more often for contracts which run only a
year or two. Thus, in case the decision is reached to
not renew a superintendent's contract, the board can
point to previous "warnings" of deficiencies.
6. Both sides should prepare for the evaluation--the
superintendent by conducting a rigorous self-
evaluation, the board by examining various sources of
information relative to the superintendent's
performance. Areas reviewed by a board might include
the superintendent's job description; district goals,
plans, and projects; situational factors which may
influence the superintendent's performance; previous
performance evaluations; and instances of outstandingly
excellent or deficient performance.
7. The board should not limit itself to those items which
appear on the evaluation form or on the list of
performance objectives. It would be difficult to
develop a form or set of guidelines which will
guarantee that every area is covered.
8. Each judgement should be supported by as much rationale
and objective evidence as possible. One board member's
opinion should not be the sole basis for judgement on
an appraisal item.
31


9. The superintendent should have the opportunity to
evaluate the board, individually as well as
collectively. Ideally the evaluation includes an
examination of the working relationships between the
board and superintendent, (p. 3)
Lipham (1975) offered four central considerations in
planning for the formal evaluation of administrative
performance.
1. The purpose of evaluation must be raised from an
implicit to the explicit level.
2. The procedures used must be viewed as relevant, valid,
and reliable by all parties involved in the process.
3. The evaluation process should include at least the
immediate superiors and immediate subordinates, as well
as the individual being evaluated.
4. Evaluation must be a continuous process, day-to-day
rather than once a year, and it should give attention
to both formative and summative evaluation, (p. 14)
Page (1975) was of the opinion that formal evaluation was
the key to strengthening the performance of both school boards
and superintendents. She stated:
Only through careful, honest, open appraisal can a board
hope to improve its performance and that of its
superintendent. The board must recognize that the
school system can be no better than the board. That the
superintendent can go no further than the limitations
the board will allow, (p. 3)
In the 1970's, several manuals appeared which suggested
methods and procedures for formal evaluation. Of these, the
American Association School Administrators (1977) suggested the
most formal approach which included these steps:
32


1. The board in consultation with the superintendent,
establishes the evaluation plan and provides a
timetable for the annual evaluation cycle.
2. The superintendent might develop procedures for
obtaining information from staff and community on
performance of the superintendent and needs for
improvement.
3. Based on the needs assessment, the superintendent
prepares a tentative set of objectives for the next
school year, including program and activities to
achieve the objectives and evidence of year-end
accomplishments.
4. The board reviews the objectives, modifying as
necessary, until they are mutually satisfactory.
5. The superintendent develops a plan of action for the
following year.
6. The board reviews the plan.
7. Interim progress review sessions are held as needed
during the year with the board and the superintendent.
8. The superintendent prepares a self-evaluation at the
end of the year summarizing progress made toward
accomplishing the objectives.
9. The board reviews the report. Each objective is
sometimes placed on a rating chart with rating for
performance ranging from unsatisfactory, (1, 2, 3) to
satisfactory (4, 5, 6) to commendable (7, 8, 9). Each
board member rates the performance on each objective
after which the ratings are complied and averaged. The
board and superintendent review the ratings. An
overall rating from 7 to 9 may mean that compensation
above what is prescribed by other consideration is in
order. A rating from 4 to 6 may indicate lower
compensation. A rating below 4 may indicate the need
to terminate the superintendent, (p. 7)
Redfern (1980) used a continuum to reflect the evolving
nature of the formal evaluation of superintendents of schools in
the AASA publication Evaluating the Superintendent:
33


The following continuum depicts past practices and
the emergence of improved techniques. Actual dates
for 'then' and 'now' would vary from one school
system to another:
THEN NOW
A B C D E F G
A No planned procedures; reliance upon word-of-mouth
assessments
B Informal assessments; minimal feedback to
super intendent
C = 'Report Card' type evaluations; heavy reliance upon
trait rating
D = Refinement of checklist rating techniques; more
feedback to superintendent
E Better definitions of executive
duties/responsibilities; emergence of performance
standards; pre- and post-assessment conferences
F = Use of performance objectives; more emphasis upon
results achieved
G = Reciprocal evaluation techniques (two way assessments);
improvement in performance made a high priority in the
evaluation process, (pp. 7-8)
"School systems are at various stages along the continuum.
Some evaluation practices are unrefined, but considerable
improvement has taken place during the last ten years. However,
in many cases, much remains to be done" (p. 8).
Yates (1982) found in Illinois that only 32% of the responding
school districts used a written evaluation procedure to assess
the performance of their chief executive officers.
However, even into the 1980's, informal procedures for
evaluation seemed to remain more typical than formal procedures
34


even though board members and superintendents alike recommended
more formalized approaches to improve the process. Cunningham
and Hentges (1982) found 30% of the superintendents responding
to the 1982 AASA National Survey indicated they were formally
evaluated by their Boards of Education, while just over 37%
stated they were evaluated informally. Approximately 25% stated
they were evaluated using both procedures.
Miller (1982) found the procedures utilized in the
evaluation of superintendents of schools varied from quite
formal processes established through board policy and
administrative guidelines to rather casual and informal
exchanges.
Gould (1982) indicated that formal evaluation of the
superintendent was usually done by the board or a sub*committee
of the board, and the board may have used input from a
consultant from the staff and community when it drafted the
evaluation plan. The larger the district, the more likely the
district was to have a formal evaluation plan for the
superintendent. Formal evaluation plans produced more
credibility in terms of public accountability for the board.
Sarbaugh (1982) stated that if a Board of Education decided
to embark on a formal program of superintendent evaluation,
several considerations were essential. Evaluation purposes must
be clearly defined; the superintendent must know and be involved
in developing the standards against which he or she will be
35


evaluated; evidences for both strengths and weaknesses should be
included; built in restrictions of which the superintendent has
no control should be considered; both formative and summative
evaluation must be included; the process should follow a formal,
annual cycle; and an evaluation should occur at a scheduled time
and place in executive session with no other items on the agenda
and with all board members and the superintendent present.
Evans (1986) stated unquestionably, the trend among Boards
of Education across the country was away from informal
evaluation arrangements, which left much to chance, toward those
practices with more structure. However, as long as the formal
procedure for administrative evaluation was not implemented, the
informal prevailed.
The message in reviewing the above studies was clear. Many
superintendents and Boards of Education continued to rely on
informal evaluation procedures while espousing the virtues of a
more formal process. They believed evaluation should be formal,
in writing, and be conducted on a regular basis, but evidently
had some difficulty in practicing what they preached.
Frequency of Superintendent Evaluation
The frequency of formal evaluation was referenced often.
Formal evaluation of the superintendent should have occurred
minimally once a year; however, most researchers recommended
36


that the evaluation occur more often than once a year. Cuban
(1977) stated that:
Once a year is not enough because formal, year-end
evaluations (and their following conferences to discuss
results) place too much emotional weight on the employee.
They too easily become a garbage can for dumping an
entire year's unresolved issues, unanswered questions,
and untouched peeves. At least two formal conferences
each year should be held between the board and
superintendent. The rationale is that a school board can
influence the executive's behavior before the end of the
school year. (p. 24)
Redfern (1980) stated that both the National School Boards
Association and the American Association of School Administra-
tors recommended at least an annual evaluation with some
interim steps to enhance communications.
Resource publications, developed by superintendent and/or
board professional organizations at the state level, sometimes
included recommendations regarding how often the superintendent
should be evaluated. The Washington State School Directors
Association (1974) stated:
The evaluation should be conducted at regular intervals
(once a year, every six months, etc.). One or two
progress interviews in the interim would give the board
the opportunity to inform the superintendent whether or
not his or her efforts should be directed differently,
and these could help the superintendent to make the
necessary changes, (p. 8)
In Colorado there was legal mandate found in H.B. 1338
(1986) which required that every certified staff member be
appraised at least once every three years.
37


Whatever the frequency, it was important not to conduct an
evaluation of the superintendent of schools once a year and then
forget about it. Dittloff (1982) stated evaluation, to be
effective, must be continuous.
Methods of Superintendent Evaluation
While it was generally accepted that the need for
continuous evaluation of the superintendent existed, the method
of such an evaluation varied greatly. That a need existed was
given credence by Gross, Mason, & McEachern (1958) when they
stated:
What is required is some device whereby superintendents
can gain greater insights, understandings, and needed
help in their efforts to exert educational leadership in
their schools and in their communities, (p. 141)
Redfern (1980) stated there was emerging evidence that
boards and superintendents were working together to design well
structured and useful techniques for the evaluation of the
chief executive officer. Olds (1977) cautioned that there was a
trap in that it was easy to fall into believing that all
administrative efforts and responsibilities could be measured by
some handy dandy test, yardstick or checklist.
Redfern (1980) identified seven techniques or methods for
the evaluation of the superintendent of schools. They included:
- Essay Evaluations
- Graphic Rating Scale
38


- Forced Choice Technique
- Work Standards
- Performance Standards
- Evaluation by Objectives
- Management by Objectives
Redfern (1980) indicated some form of work or performance
standards along with specific objectives were being used with
increasing frequency, but stated:
School boards, however, probably will be inclined to use
a variety of techniques, as determined by their own views
and preferences. A good evaluation can be achieved in
various ways provided the process is thoughtfully
planned, cooperatively implemented and completed in a
professional manner, (p. 13)
Nygaard (1974) referred to five general techniques or
methods for the evaluation of administrative behavior:
Graphic Rating Scales the administrator is evaluated
according to how frequently a behavior is observed. An
example would be a typical checklist of behaviors.
Essay Appraisals the evaluator writes a narrative
description of the administrator discussing strengths,
weaknesses, and potential.
Field Review essay and graphic ratings by several
evaluators are combined into systematic review process.
Forced Choice Rating the evaluator must choose from two
or more statements that best describe the administrator's
behavior.
Critical Incident Appraisal administrator behavior is
recorded at critical periods or when significant
incidents occurs, (p. 41)
39


Booth and Glaub (1978), and Evans (1986) limited the
methods for evaluating school administrators, including the
superintendent, to rating scales and performance objectives.
Booth and Glaub stated:
Basically, there are two popular systems for evaluating
the superintendent and a multitude of variations
thereof. Checklists and rating scales are by far the
most common devices used in an evaluation. Boards that
limit superintendent appraisal to a checklist should
expect it to serve only as an indicator of basic
abilities or as a way to educate board members about the
superintendency.
Written objectives represent the other approach to
superintendent appraisal which is gaining in popularity
because of its orientation toward results in future
growth. Most such systems are derived from the widely
heralded system called "Management By Objectives" or MBO.
Properly written objectives contain both deadlines for
completion and measures for determining the
superintendent's success or failure, (p. 11)
However, Barraclough (1974) pointed out that, from an
accuracy standpoint, rating scales may be subject to a high
level of invalidity:
Since the evaluator(s) is stating his opinion of how an
administrator measures up to a set of standards, the
evaluation is highly subjective. Many instruments are
poorly designed. The administrator is rarely, ever,
consulted in establishing the standards against which he
will be measured. In addition, (the trade of)
performance standards are inflexible and do not allow for
changes in circumstances or specific tasks, (p. 17)
As reported by Bolton (1980), information could be
collected in one of three ways:
Observing behavior, asking questions and examining
written documents. Each of these ways may be used in the
evaluation of administrators. He then cites the
following types of rating scales that might be used as
40


methods to collect desired Information. Rank ordering,
forced distribution, absolute categories, verbal
descriptors, degree for existence and the extent of
agreement, (p. 70)
The performance objectives approach to personnel evaluation
was a procedure in which the superordinate and the subordinate
agreed on the appropriate educational objectives. The
subordinate proposed strategies for achieving the objectives,
set up activities to accomplish the objectives, and periodically
evaluated the progress made toward achieving the objectives.
This method of formal evaluation was sometimes referred to as
"job improvement targeting" or "management by objectives."
Evans (1979), stated there was evidence in the literature which
confirmed that this technique of performance improvement
targeting or management by objectives was the most widely used
type of formal evaluation plan being utilized by Boards of
Education.
When existing organization goals and objectives were linked
to clearly understood and specifically developed individual
efforts, along with the ability to document and to track the
record of growth and success, the work environment was
especially rewarding (AASA, 1977).
When using performance objectives as the basis of the
superintendent's evaluation, the superintendent and the board
agreed to base future judgments on the accomplishment of agreed
upon goals. Periodic checkups were conducted by the board
41


members prior to the final evaluation of the superintendent.
More and more school boards were currently practicing this
method of evaluation.
Evans (1986) found that school systems employing a
performance objectives' plan for evaluating their
superintendents usually organized the procedures along the
following lines:
1. Statement of objectives (a set of objectives, mutually
agreed on by the board and superintendent, and the
expected outcomes).
2. Calendar (target completion dates for fulfilling these
obj ectives).
3. Standards of performance (a description of activities
to be performed and/or the standards established as
acceptable achievement of objectives).
4. Measurements to be used (description of the
measurements which we used to access progress or
acceptable completion).
5. Results of evaluation (comparison of outcomes of the
original intent of the objectives).
6. Performance rating (an evaluation of the
superintendent's performance to identify areas needing
improvement and deserving commendation. Renewal or
nonrenewal of a superintendent's contract may also be
an outcome), (p. 83)
However, Coombs (1972) cautioned against total dedication
to and thus misuse of the performance objectives approach. He
pointed out that such behavioral objectives were written for
specific reasons and could not account for other aspects of job
performance. Redfern (1972) was a strong advocate of self-
evaluation, and stated that it "is the starting point of a
42


comprehensive assessment of performance effectiveness" (p. 7).
He noted that the administrator's assessment of his or her own
accomplishments and failures necessitated measuring behavior by
personal goals as opposed to comparing oneself with others.
Olds (1973) agreed and believed "self-evaluation may eventually
become the single most important factor in the entire
evaluation process in education" (p 6).
It was clear that if the superintendent was to be properly
evaluated he/she must know what the Board of Education expected
of him or her. The expectations should have been written in the
form of a position description or job description which served
as guidelines to the superintendent in performing the duties of
the office; and also a means for the board to evaluate his or
her performance. The position or job description also acted as
a public document in that it also informed others of the board's
expectations of the superintendent. Evans (1986) advocated the
creation of a written position description as the first step in
the development of an evaluation plan. The job or position
description was already in existence and a careful review and
possible revision became the first step. The job or position
description should be reviewed as times and circumstances
changed so that the current expectations of the superintendent
were reflected in his current job or position description.
The most significant potential value of this evaluation
plan according to Evans (1986), was that it literally forced the
43


board and superintendent to examine a multitude of needs of the
educational system and to set priorities for the ensuing year or
years based upon their collective analysis of those needs.
Snavely (1984) stated that school district goals and
objectives should be jointly agreed upon by the board and the
superintendent. The evaluation should have measured the success
or failure of the superintendent and school district in
achieving these goals and objectives.
Summary
In summary, the position of public school superintendent,
which has evolved over a period of more than 135 years, was not
yet nor ever will be, completely standardized as long as
schools remain decentralized in a free and ever changing
society. No matter which procedure or method was used to
evaluate the superintendent, e.g., checklists, performance
objectives, self-appraisal, the related literature suggested a
number of essentials that should be an inherent part of the
process.
AASA (1979) maintained the form the superintendency will
take in the future will depend in great part on the
professional vision, enterprise, statesmanship, and courage of
individuals in the generation of superintendents and board
members still to come. Evaluation of the public school
superintendent is as old as the position itself, and will remain
44


a critical process as Boards of Education continue to make the
final decisions to retain or dismiss their superintendents.
45


CHAPTER 3
DESIGN AND PROCEDURES
The purpose of this study was to determine if the performance
of superintendents in selected Colorado school districts had
changed as a result of the implementation of state mandated policy
changes enacted in 1986. Specifically, the policy changes referred
to above were § 22-9-109, C.R.S., a statute requiring specific parts
of the superintendent's evaluation be considered open records, and
therefore, public information; and H.B. 1338, a statute making the
evaluation of all certified personnel, including superintendents,
mandatory.
The research methodology used in this study was descriptive.
Lehman and Mehrens (1979) described descriptive research as being
"concerned with determining the nature and degree of existing
conditions." They also clearly stated that typically descriptive
research does not make predictions or causal inferences, (p. 81)
The specific approach used by the researcher in this study was
to interview two different groups, asking them to respond to the
same set of predetermined questions in a face-to-face interview.
The responses were then compared among the members of the group.


Denzin (1970), identified three forms of the sociological
interview. They were:
1. The "scheduled standardized interview" where the wording and
order of all questions are presented to each respondent in exactly
the same way each time.
2. The "non-scheduled standardized interview" where the
interviewee works from a list of the information required from each
respondent, but the particular phrasing of questions and their order
is redefined to fit the characteristics of each interviewee. This
procedure permits probing and clarification within the context of a
particular interview.
3. The "non-standardized interview" where no pre-specified set of
questions is employed, nor are questions asked in a specific order.
The interviewer has complete freedom to probe at will. (pp. 123-126)
For purposes of this study, a modified "non-scheduled
standardized interview" format was utilized. The same exact
questions were asked of each respondent, but the order of the
questions was varied depending on the flow of the answers by the
respondent. This format allowed the interview to be adapted to
each respondent's understanding of H.B. 1338, § 22-9-109, C.R.S. and
superintendent evaluation processes in an individual manner.
Data sources included the statutes § 22-9-109, C.R.S., H.B.
1338, and personal structured interviews with superintendents and
board members of selected Colorado School District.
47


Problem Questions
Several research questions were stated to guide the research:
1. What has been the impact of § 22-9-109, C.R.S. (1986) on
superintendent evaluation in selected Colorado school districts?
2. What has been the impact of H.B. 1338 (1986) on
superintendent evaluation in selected Colorado school districts?
3. To what extent has the performance of the superintendent
changed, comparing the performance of the superintendent prior to
July 1, 1986, to the current performance of the superintendent, as a
result of the implementation of § 22-9-109, C.R.S. and H.B. 1338?
4. As a result of the implementation of § 22-9-109, C.R.S.
and H.B. 1338, was there a difference in the accomplishments of the
selected Colorado school districts' written objectives prior to
July 1, 1986, and the present?
5. To what extent was the model, approach, or type of
superintendent evaluation system adopted by the school district
consistent with H.B. 1338?
6. To what extent was the model, approach, or type of
district superintendent evaluation system adopted by the school
district consistent with accepted practices of superintendent
evaluation?
7. As a result of the implementation of § 22-9-109, C.R.S.
and H.B. 1338, what formal evaluator training to evaluate
48


superintendents has been provided to Board members of selected
school districts in Colorado?
8. What was the attitude of a board member and the
superintendent of selected Colorado school districts toward the
credibility, fairness, believability, and effectiveness of the
model, approach, or type of evaluation system the district uses to
evaluate the current performance of the superintendent?
9. Was there consistency in the model, approach or type of
superintendent evaluation system used by selected Colorado school
systems?
Procedures
The following procedures were followed to gather the necessary
data to measure the improvement, if any, of superintendent
performance.
1. The researcher performed an analysis of § 22-9-109, C.R.S.
and H.B. 1338 to determine appropriate areas that were applicable
and pertinent to the study.
2. Utilizing the information gathered from number one above
and the research questions posed, the researcher developed an
interview guide to be used in selected Colorado school districts.
(See appendix) The guide was reviewed by a panel of experts to
establish content validity. The panel was composed of Dr. Gerald
Difford, Executive Director of The Colorado Association of School
Executives; Dr. Randall Quinn, Executive Director of The Colorado
49


Association of School Boards; Dr. David Conley, Executive Director
of Curriculum for The Poudre R-l School District in Ft. Collins and
a recognized expert on H.B. 1338; and Dr. Richard Koeppe,
Superintendent of The Denver Public Schools. The members of this
panel were asked to review the interview guide, independently, with
respect to its (1) completeness, (2) length, (3) clarity of
questions, (4) the intent of questions, (5) appropriateness of
questions, (6) forming of questions, and (7) ability to discriminate
in answers. Following the panel's initial review, corrections were
made to the interview guide. Following the revisions of the
interview guide, it was field tested with Dr. Michael Massarotti,
Superintendent of Adams County School District #50. Dr. Massarotti
was not eligible to participate in the study, but was representative
of superintendents included in Category II. Dr. Massarotti's
response to the interview guide was evaluated in terms of his
ability to respond to the interview guide's specific items. In
addition, he was asked for suggestions for improvement regarding the
guide's (1) completeness, (2) length, (3) clarity of questions, (4),
intent of questions, (5) appropriateness of questions, (6) framing
of questions, and (7) ability to discriminate in answers. The final
revision of the interview guide was reviewed for clarity and
accuracy prior to the actual interviews being conducted.
3. To control for inter-rater reliability, only the
researcher conducted the interviews. He had previous experience in
the related areas of Personnel Services and Human Resources in Mesa
50


Valley County School District #51 in Grand Junction, Colorado, and
Cherry Creek School District #5 in Denver, Colorado, for the past
11 years. The researcher has been trained extensively in the
interview process, conducted literally thousands of interviews, and
has been a consultant on interviewing to organizations such as The
Colorado Association of School Executives, Western State College,
Mesa State College, Metropolitan State College, Regis College,
University of Denver and The University of Colorado. The researcher
can be considered a recognized expert in interviewing techniques.
4. Colorado School Districts identified as setting Categories
II and III in § 22-53-101 et sea.. were used as an initial base of
school districts included in the study. The tenure of the
superintendents in these districts was reviewed, and those districts
whose superintendent's tenure began prior to July 1, 1986, were
included as the sample population.
5. In those identified districts, board members (preferably
board presidents) whose tenure began prior to July 1, 1986, were
selected to be interviewed.
6. In these identified districts, the superintendents and
board members were interviewed using the structured interview guide
confirmed by the panel of experts and field tested with Dr.
Massarotti.
7. The interviews were conducted face-to-face and individually
with each superintendent and board member. Each interview lasted
approximately two and one-half to three hours. Audio tape
51


recordings were not used because of the sensitive nature of
evaluation. The interviewer recorded the data provided in detail.
At the conclusion of the interview, the interviewer reviewed the
content of the notes for accuracy with the interviewee.
The Population
In the spring of 1988, the Colorado Legislature passed the
Public School Finance Act of 1988 (§ 22-53-101 et sea.). This act,
originally known as H.B. 1341, redesigned the financing of public
education in Colorado. One purpose for the act was to ensure
greater equitability among the school districts in Colorado. One of
the methods employed to assist in accomplishing this was to
categorize the 176 K-12 school districts in Colorado into eight
different "Setting Categories." These eight setting categories are:
- Category I Core City Denver
- Category II Denver Metropolitan Area
Composed of districts located within the Denver-Boulder
standard metropolitan statistical area which are primarily
suburban in nature.
- Category III Urban Suburban
Composed of districts which comprise the state's major
population centers outside of the Denver metropolitan area
and their immediate surrounding suburbs.
- Category IV Outlying City
52


Composed of districts in which most of the pupils live in
population centers of seven thousand persons or more but
less than thirty thousand persons.
- Category V Outlying Town
Composed of districts in which most of the pupils live in
population centers in excess of one thousand persons but
less than seven thousand persons.
- Category VI Rural
Composed of districts with no population centers in excess of
one thousand persons and is characterized by sparse
widespread populations. These districts do not meet
enrollment criteria for Category VIII Small Attendance.
Category VII Recreational
Composed of districts which contain major recreational
developments that impact the cost of property values,
community income and other cost-of-living components.
Category VIII Small Attendance
Composed of districts which are rural in nature and have
enrollments of less than one hundred fifty persons.
(§ 22-53-101 et sea.1
Category I consists of one district, Denver. School districts
in Categories II and III represent over 70% of the students enrolled
in public schools in Colorado. For this reason, the researcher
chose to survey those districts in Categories II and III whose
superintendent's tenure began prior to July 1, 1986.
53


Of the twenty-nine total districts included in Categories II
and III, sixteen had superintendents whose tenure began prior to
July 1, 1986. (See table 3.1).
The researcher interviewed the superintendent and a board
member, if his or her tenure began prior to July 1, 1986, of each
respective district identified in Categories II and III of
§ 22-53-101 et sea. (H.B. 1341) with the exception of the
superintendent and board member from Jefferson County RE-1, and the
board member from El Paso County, Harrison. These three were not
available for interviewing. These individuals constituted the
entire population for this study.
54


Table 3.1. School districts in categories II and III with those
eligible for study specifically identified.
Category II
Category III
Adams County 1 (Mapleton) Boulder County RE-1J (St. Vrain Valley) *
Adams County 12 (Northglenn Thornton) El Paso County 2 (Harrison) *
Adams County 14 (Commerce City) El Paso County 3 (Widefield)
Adams County 27J (Brighton) El Paso County 8 (Fountain) *
Adams County 50 (Westminster) El Paso County 11 (Colorado Springs)
Arapahoe County 1 (Englewood) El Paso County 12 (Cheyenne Mountain) *
Arapahoe County 2 (Sheridan) El Paso County 14 (Manitou Springs) *
Arapahoe County 5 (Cherry Creek) El Paso County 20 (Academy) *
Arapahoe County 6 (Littleton) El Paso County 38 (Lewis-Palmer)
Arapahoe County 28J (Aurora) El Paso County 49 (Falcon)
Boulder County RE-2 (Boulder Valley) Larimer County R-l (Poudre) *
Douglas County RE-1 (Douglas County) Larimer County R-2J (Thompson)
Jefferson County R-l (Jefferson) Mesa County 51 (Mesa Valley)
55


Table 3.1. (contd.)
Category II
Category III
Pueblo County 60 *
(Pueblo City)
Pueblo County 70
(Pueblo Rural)
Weld County 6
(Greeley)
TOTAL CATEGORY II 8 TOTAL CATEGORY III 8
* Superintendent evaluated prior to July 1, 1986
56


Variables and Source Collection
The dependent variable y, is the impact of § 22-9-109, C.R.S.,
Title 22 and H.B. 1338 on superintendent evaluation in selected
school districts in Colorado. The quasi-independent variables used
in the study were:
X^ Accomplishment of district's objectives.
X^ Management of district's fiscal resources.
O
XJ Management of systematic planning process for the district.
X^ Management of district's personnel supervision and
evaluation system.
X^ Evaluation in form of rating checklist or narrative or
both.
X Board members received formal training.
X^ Purpose of evaluation clearly stated in policy.
X Evaluation included in superintendent's contract.
X^ Evaluation includes written plan of improvement.
X^O Evaluation conducted on regular cycle.
xH Board and Superintendent identify specific criteria to be
evaluated.
X^2 Board and superintendent identify standards of expected
performance.
X^- Sources of data predetermined.
X^ Provision for gathering objective data.
Xl5 Provision for self-appraisal by superintendent
57


Xl6 Identification of specific objectives for next cycle.
X1? Provision for reasonable timeline to improve performance.
X^ Specific Board meeting held for purpose of evaluation.
Table 3.2. Evaluation variable by study or source
Evaluation Variable Study or Source
(Quas i-Independent)
X^- Accomplishment of district's
objectives
X^ Management of district's
fiscal resources
X^ Management of systematic
planning process for the
district
X^ Management of district's
personnel supervision and
evaluation system
X-* Evaluation in form of rating
checklist or narrative or both
X Board members received formal
training
X^ Purpose of evaluation clearly
stated in policy
X Evaluation included in
superintendent's contract
X^ Evaluation includes
written plan of improvement
§ 22-9-109, C.R.S.
§ 22-9-109, C.R.S.
§ 22-9-109, C.R.S.
§ 22-9-109, C.R.S.
Redfern, 1980
Nygaard, 1974
Booth and Glaub, 1978
Evans, 1981
H.B. 1338
AASA, NASBA, 1980
AASA, NASBA, 1980
H.B. 1338
58


Table 3.2. (contd.)
Evaluation Variable
(Quas i-Independent)
X^-0 Evaluation conducted on
regular cycle
Board and superintendent
identify specific criteria
to be evaluated
X^ Board and superintendent
identify standards of expected
performance
X13 Sources of data predetermined
X^ Provision for gathering
objective data
X^-3 Provision for self-appraisal
by superintendent
X^ Identification of specific
objectives for next cycle
X17 Provision for reasonable
timeline to improve
performance
X^- Specific Board meeting held
for purpose of evaluation
Study or Source
H.B. 1338
H.B. 1338
ERS, 1972
H.B. 1338
H.B. 1338
Redfern, 1972
AASA, 1977
H.B. 1338
ERS, 1972
59


Statistical Treatment
Each of the 18 quasi-independent variables was tested to
determine the degree of proportional agreement between
superintendents and board members in respective categories. The
data were gathered through a structured interview (see appendix).
Three types of responses were included in the interview:
1) Five point Likert Scale ranging from "strongly agree" to
"strongly disagree." These recorded the respondents'
relative perception to questions measuring perceptions of
degrees of implementation of law or practice.
2) A two-point scale of a "yes" or "no" response was included.
These questions measured the specific compliance to law or
practice, and provided an opportunity for the respondent to
choose one of two answers which represented the individual
respondent's perception. These were developed to measure
the relationship between an quasi-independent variable and
a specific law as perceived by the interviewee.
3) Unstructured responses to open-ended questions occurred as
the interviewer probed for further understanding of
selected quasi-independent variables.
All data were in their original form. Quasi-independent
variables that were measured by Likert Scale were; management of
district fiscal resources, management of systematic planning
process, management of district's personnel supervision and
60


evaluation system, accomplishment of district objectives,
identification of specific criteria to be evaluated, identification
of standards of performance, predetermined sources of data,
provision for gathering specific data, provision for self-appraisal,
and identification of specific objectives for next cycle.
The following quasi-independent variables were yes/no responses
in nature: evaluation included in superintendents contract,
specific board meeting held for purpose of evaluations, purpose of
evaluation clearly stated in policy, evaluation in form of rating
checklist or narrative or both, the evaluation included a written
plan for improvement, the evaluation was conducted on a regular
cycle, provision for reasonable timeline to improve performance, and
board members received formal training.
After all data were collected on the impact of § 22-9-109,
C.R.S. and H.B. 1338 on superintendent evaluation, the data that
were in the form of a Likert Scale or "yes" or "no" in nature were
input into the version 9.1 of the Statistical Package for the Social
Sciences to measure the standard error of proportion.
The standard error of proportion was determined to be the
appropriate statistical treatment because the size of the population
proved too small to perform other statistical treatments.
For purposes of performing the tests of proportion, "strongly
agree" and "agree" options were collapsed in "agree" values.
"Neutral" and all "disagree" options were collapsed as "disagree"
values. The proportion of "agree" responses were then tested
61


between superintendent and board subjects using the following test
formula:
Subgroup 1 represents superintendents, and subgroup 2
represents board members. The symbol "p" represents the proportion
of agreement of each subgroup, the symbol "f" represents the
frequency within each subgroup that agreed, and the symbol "n"
represents the total number in the population.
The critical values were determined to be + or 1.96.
Some quasi-independent variables required open ended questions
to further understand the implementation of or compliance to law.
Responses to those questions were listed by question, and by
superintendent or board member, word for word. These were analyzed
to determine common themes among the responses. The number of
responses that were indicative of the common themes, were then
identified, calculated, recorded, and reported in table form.
Eighteen quasi-independent variables were identified that
related to the impact of § 22-9-109, C.R.S. and H.B. 1338 on
superintendent evaluation in selected school districts in Colorado.
During the stammer and fall of 1989, 29 non-standardized structured
Pi P2
Summary
62


interviews were conducted in seven school districts in Category II
and eight school districts in Category III of H.B. 1341.
The findings of the study are reported in Chapter 4, with the
conclusions, implications, and recommendations discussed in Chapter
5.
63


CHAPTER 4
ANALYSIS OF DATA
Introduction
Chapter 4 presents the findings of this study of the impact of
state mandated policy changes on superintendent evaluation in
selected Colorado school districts.
The primary method of data collection used in this study was
the personal non-standardized structured interview. Interviews were
conducted individually with the superintendents and board members
from school districts that fell within the parameters of the study.
A total of 29 interviews were conducted.
Because the data utilized in this study were from a limited
number of school districts (15), it was appropriate to utilize
descriptive statistics in the analysis of the findings.
Two types of descriptive statistics were used. First, when
data were obtained in a format that allowed comparison of the
agreement between superintendnets and board members in statistical
format, the version 9.1 of the Statistical Package for the Social
Sciences was used to perform the statistical calculation of the
standard error of proportion on the data. For purposes of
performing the tests of proportion, "strongly agree" and "agree"
options were collapsed as "agree" values. "Neutral" and all


I
"disagree" options were collapsed as "disagree" values. The
proportions of "agree" responses were then tested between
superintendent and board subjects using the following test formula:
Subgroup 1 represents superintendents, and subgroup 2
represents board members. The symbol "p" represents the proportion
of agreement of each subgroup, the symbol "f" represents the
frquency within each subgroup that agreed, and the symbol "n"
represents the total number in the population.
The critical values were determined to be + or 1.96.
Secondly, when the respondent was called upon for an open-
ended response, the researcher recorded the response verbatim. The
responses were then categorized by the respective role of the
superintendent or board member. The researcher then identified and
grouped each response with other responses similar in concept.
Percentage of responses by concept were then calculated, recorded,
and reported in table form.
Data gathered in the study were used to answer each of the
research questions upon which the study was based. Data obtained
from survey questions 8, 9, lib, 12b, 13b, 14b, 21a, and 41
addressed research questions one and two. Data obtained from survey
Pi P2
65


questions 10, 10a, 12, 13, and 14 addressed research question three.
Data obtained from survey questions 11, 11a, and lib addressed
research question four. Data obtained from survey question 21, 22,
23, 24, 24a, 25, 25a, 26, 26a, 27, and 27a addressed research
question five. Data obtained from research questions 17, 17a, 18,
19, 20, 28, 28a, 29, 29a, 30, 30a, 31, 31a, and 32 addressed
research question six. Data obtained from survey questions 16 and
16a addressed research question seven. Data obtained from survey
questions 34, 34a, 35, 35a, 36, 36a, 37, and 37a addressed research
question eight. Data obtained from survey questions 15, 18, 19, 20,
31d, 33, 33a, 38, 39, and 40 addressed research question nine.
Research Question One
What has been the impact of § 22-9-109, C.R.S. (1986) on
superintendents and Boards of Education in selected Colorado school
districts?
Research Question Two
What has been the impact of H.B. 1338 (1986) on superintendents
and Boards of Education in selected Colorado school districts?
Nine specific items on the structured interview guide were
designed to provide data to answer research questions one and two.
Raw data were grouped by superintendent or board members and then by
concept. The frequency of the response options was then calculated
and percentage values assigned as shown below.
66


Survey Item 8
"What has been the impact of § 22-9-109, C.R.S. on your
(superintendent) evaluation?"
Results displayed in Table 4.1 show the frequency on the
response options for superintendents and board members regarding the
perception of the impact of § 22-9-109, C.R.S. on superintendent
evaluation. The descriptive analysis displayed in the table show
80% of the superintendents felt that § 22-9-109, C.R.S. had "minimal
impact", while 73% of board members indicated "minimal impact" as a
response. Twenty percent of the superintendents felt that § 22-9-
109, C.R.S. had a positive impact on superintendent evaluation,
while 27% of the board members felt the same.
Table 4.1. Impact of § 22-9-109, C.R.S.
Supt. Response Clusters Board Response Clusters
Impact of law has been
minimal. (12)
The law has had a positive
impact. (3)
Impact of law has been
minimal. (11)
Very positive forced
board to be more formal. (4)
Note. Parenthetical expressions display the number responding in
the response clusters.
Representative statements made by superintendents and board
members who perceived § 22-9-109, C.R.S. had very little if any
impact on superintendent evaluation were:
67


Superintendents stated:
"Did not alter what we do!"
"Impact as forced by state is not existent."
"Relatively little impact...if anything, it may have
influenced board members' evaluation to be more positive."
Board members stated:
"No impact... nothing has changed."
"None whatsoever!!"
Representative statements made by superintendents and board
members who perceived § 22-9-109, C.R.S. had a positive effect on
superintendent evaluations were:
Superintendents stated:
"It has made process more formal."
"It is an advantage to superintendent because board
is sensitive to negative comments."
"Made board more deliberate."
"More detail included in evaluation."
Board members stated:
"Caused board to more thoughtful thinking and more
complete."
Had positive impact forced board to be more formal."
Caused board to share more of the good things with
community."
68


Survey Item 9
"What has been the impact of H.B. 1338 on your (superintendent)
evaluation?"
Results displayed in Table 4.2 show the frequency on the
response options for superintendents and board members regarding the
perception of the impact of H.B. 1338 on superintendent evaluation.
The descriptive analysis displayed in the table show 80X of the
superintendents felt that H.B. 1338 had no significant impact, while
93% of board members indicated no significant impact as a response.
Twenty percent of the superintendents felt that H.B. 1338 had a
positive impact on superintendent evaluation, while 7% of the board
members felt the same.
Table 4.2. Impact of H.B. 1338.
Supt. Response Clusters Board Response Clusters
Impact of law has been
minimal. (12)
The law had a positive
impact. (3)
Impact of law has been
(minimal. (13)
The law had a positive
impact. (1)
Note. Parenthetical expressions display the number responding in
the response clusters.
Representative statements made by superintendents and board
members who perceived H.B. 1338 had little if any impact on
superintendent evaluation were:
69


Superintendents stated:
"Relatively little impact in process."
"Not any doing it the same way as before."
"The law had a positive impact."
Board members stated:
"The law has not made an impact."
"Very little we were in compliance prior to 1986."
"It has not had any."
"The law had a positive impact."
Survey Item lib
"Objectives were accomplished as a result of...."
Results displayed in Table 4.3 show the frequency of response
options for superintendents and board members indicating their
perceptions of why formal district objectives were accomplished on a
forced choice scale. Thirteen or 87% of superintendents indicated
that the district's formal objectives were accomplished as a result
of the superintendent's leadership regardless of law; while one or
7% indicated they were accomplished as a result of H.B. 1338 alone,
and one or 7% indicated they were accomplished as a result of the
interaction between § 22-9-109, C.R.S. and H.B. 1338.
Thirteen or 93% of the board members indicated the district's
formal objectives were accomplished as a result of the
superintendent's leadership regardless of law; while only one or 7%
70


indicated the district's formal objectives were accomplished as a
result of § 22-9-109, C.R.S.
Table 4.3. Impetus for accomplishing objectives.
Class, n Supt. Leadership Regardless of law HB 1338 § 22-9-109, C.R.S. § 22-9-109, C.R.S. and HB 1338
Supt. 15 13(87) 1(7) - 1(7)
Board Members 14 13(93) - 1(7) -
Note. Parenthetical expressions display percentages; percentages
may not always equal 100 due to rounding off.
Survey Item 12b
"Management of district fiscal resources were improved as
result of."
Results displayed in Table 4.4 show the frequency of response
options for superintendents and board members indicating their
perceptions of why the management of district fiscal resources has
improved on a forced choice scale. Fourteen or 93% of
superintendents indicated that the management of district fiscal
resources improved as a result of superintendent leadership
regardless of law; and one or 7% indicated the improvement of the
71


management of district fiscal resources was a direct result of the
interaction of § 22-9-109, C.R.S. and H.B. 1338.
Twelve or 86% of the board members indicated that the
management of district fiscal resources improved as a result of the
superintendent's leadership regardless of law; and two or 14%
indicated the improvement of the management of district fiscal
resources occurred as a result of the interaction between
§ 22-9-109, C.R.S. and H.B. 1338.
Table 4.4. Impetus for improving management of the district's
fiscal resources.
Class. n Supt. Leadership Regardless of law HB 1338 § 22-9-109, C.R.S. § 22-9-109, C.R.S. and HB 1338
Supt. 15 14(93) - - 1(7)
Board
Members 14 12(86) 2(14)
Note. Parenthetical expressions display percentages; percentages
may not always equal 100 due to rounding off.
72


Survey Item 13b
"Management of district's planning processes have improved as a
result of ...."
Results displayed in Table 4.5 show the frequency of response
options for superintendents and board members indicating their
perceptions of why the management of the district's planning process
has improved on a forced choice scale. Fourteen or 93% of
superintendents indicated the management of the district's planning
process improved as a result of the superintendent's leadership
regardless of law, and one or 7% indicated the improvement of the
management of the district's planning process was accomplished as a
result of the interaction of § 22-9-109, C.R.S. and H.B. 1338.
Thirteen or 93% of the board members indicated that the
management of the district's planning process improved as a result
of the superintendent's leadership regardless of law, and one or 7%
indicated the improvement of the management of the district's
planning process was accomplished as a result of the interaction of
§ 22-9-109, C.R.S. and H.B. 1338.
73


Table 4.5. Impetus for improving management of the district's
planning process.
Class. n Supt. Leadership Regardless of law HB 1338 § 22-9-109, C.R.S. § 22-9-109, C.R.S. and HB 1338
Supt. 15 14(93) - - 1(7)
Board
Members 14 13(93) 1(7)
Note. Parenthetical expressions display percentages; percentages
may not always equal 100 due to rounding off.
Survey Item 14b
"Management of the district's personnel supervision and
evaluation systems was improved as a result of...."
Results displayed in Table 4.6 show the frequency of response
options for superintendents and board members indicating their
perceptions of why the management of the district's personnel
supervision and evaluation system has improved on a forced choice
scale. Nine or 60% of the superintendents indicated the management
of the district's personnel supervision and evaluation systems
improved as a result of the superintendent's leadership regardless
of law; five or 33% indicated the improvement occurred as a result
of H.B. 1338; and one or 6.6% indicated the improvement of the
74


district's personnel supervision and evaluation system occurred as a
result of the interaction of § 22-9-109, C.R.S. and H.B. 1338.
Ten or 712 of the board members indicated the management of the
district's personnel supervision and evaluation system improved as a
result of the superintendent's leadership regardless of law; three
or 21% indicated improvement was a result of H.B. 1338; and one or
7% indicated the improvement of the management of the district's
personnel supervision and evaluation system was accomplished as a
result of the interaction of § 22-9-109, C.R.S. and H.B. 1338.
Table 4.6. Impetus for improving management of the district's
personnel supervision and evaluation systems.
Supt. Leadership Regardless Class, n of law HB 1338 § 22-9-109, C.R.S. § 22-9-109, C.R.S. and HB 1338
Supt. 15 9(60) 5(33) - 1(7)
Board
Members 14 10(71) 3(21) 1(7)
Note. Parenthetical expressions display percentages; percentages
may not always equal 100 due to rounding off.
75


Survey Item 21a
"If yes, (the evaluation includes a plan for improvement) as a
result of the recommendation for improvement noted in the
evaluation, improvement occurred as a result of...."
Results displayed in Table 4.7 show the frequency of response
options for superintendents and board members indicating their
perceptions of why improvements occurred in the performance of the
superintendents. Ten or 67% of superintendents stated performance
would have happened "regardless" of the laws; one or 7% stated
performance improved as a direct results of H.B. 1338; one or 7%
indicated performance improved as a result of the interaction of
§ 22-9-109, C.R.S. and H.B. 1338; and three or 20% had no response.
Twelve or 86% of the board members indicated performance would
have occurred "regardless" of law and two or 14% indicated
performance improved as a direct result of H.B. 1338.
76


Table 4.7. Impetus for improvement.
Class, n Would Have Happened Regardless of Law § 22-9-109, HB 1338 C.R.S. § 22-9-109, C.R.S. and HB 1338 No Resp
Supt. 15 10(67) 1(7) 1(7) 3(20)
Board 14 Members 12(86) 2(14) - -
Note. Parenthetical expressions display percentages; percentages
may not always equal 100 due to rounding off.
Survey Item 41
"Has the issue of public access to selected sections of the
superintendent's evaluation as defined in § 22-9-109, C.R.S. been
beneficial or has it been a problem?"
Results displayed in Table 4.8 show the frequency of response
options for superintendents and board members regarding the benefit
or harmfulness of public access to the superintendent's evaluation.
The descriptive analysis displayed in the table show that four
superintendents or 26.6% stated it was beneficial; nine or 60%
indicated they felt it was a neutral law (had no effect either way);
and two or 13.3% indicated the law had not made a difference on
performance.
77


One or 7.1% of board members stated public access was
beneficial; three or 21.4% indicated it was not beneficial because
it puts pressure on superintendents and the evaluation is watered
down because of consensus; nine or 64.2% stated they felt it was a
neutral law (had no effect either way); and one or 7.1% indicated
the law had not made a difference on performance.
Table 4.8. Benefit of public access to selected sections of
superintendent evaluation.
Supt. Response Clusters Board Response Clusters
Beneficial provides
good public relations
and builds trust. (4)
Neutral no effect either
way. (9)
Not made a difference. (2)
Beneficial builds trust.
(1)
Not beneficial puts pressure
on superintendent and
evaluation is watered down
because of consensus. (3)
Neutral. (9)
Not made a difference. (1)
Note. Parenthetical expressions display the number responding in
the response clusters.
Based on the data yielded from open-ended questions, it may be
generally stated that the impact of § 22-9-109, C.R.S. and H.B. 1338
on superintendent evaluation in selected Colorado school districts
has been minimal. Eighty percent of the superintendents and 73% of
the board members surveyed indicated § 22-9-109, C.R.S. had minimal
78


impact. While 80% of the superintendents and 86% of the board
members surveyed indicated H.B. 1338 had minimal impact.
The superintendents and board members surveyed were consistent
in their responses regarding the impact of three of the specific
provisions of § 22-9-109, C.R.S. that required four areas of the
superintendent's evaluation be made public:
- Eighty seven percent of superintendents and 93% of board
members indicated district formal objectives were accomplished
as a result of superintendent leadership regardless of law.
- Ninety three percent of superintendents and 86% of board members
indicated the management of district fiscal resources has
improved as a result of the superintendent's leadership
regardless of law.
- Ninety three percent of superintendents and 93% of board
members indicated the management of the district's planning
process has improved as a result of the superintendent's
leadership regardless of law.
- The fourth area in the superintendent's evaluation required to
be made public by § 22-9-109, C.R.S. is the management of the
district's personnel supervision and management system. The
data revealed that 60% of superintendents and 71% of board
members indicated the management of the district's personnel
supervision and management system improved as a result of
superintendent's leadership regardless of law. While 33% of
superintendents and 21% of board members indicated the
79


improvement was a result of H.B. 1338. While significant, the
results on this area were not close to the degree of
significance as noted in the first three areas.
H.B. 1338 required that all evaluations include a plan for
improvement. In responding to survey item 21a, 67% of
superintendents and 86% of board members indicated improvement
would have occurred regardless of the presence of an improvement
plan.
Survey question 41 asked superintendents and board members to
indicate if the public access requirement of § 22-9-109, C.R.S. was
beneficial. Only 26.6% of superintendents and 7.1% of board
members felt public access was beneficial. Sixty percent of
superintendents and 64.2% of board members felt it was needed.
Three or 21.4% of board members said it was not beneficial.
Lastly, the findings yielded in the survey that addressed
research questions one and two, indicated consistency between
superintendents and board members responding to each survey item.
In summary, both superintendents and board members felt generally as
well as specifically, that the impact of § 22-9-109, C.R.S. and H.B.
1338 on superintendent evaluation has been minimal.
Research Question Three
To what extent has the performance of the superintendent
changed, comparing the performance of the superintendent prior to
80


July 1, 1986, to the current performance of the superintendent, as a
result of the implementation of § 22-9-109, C.R.S. and H.B. 1338?
Five specific items on the structured interviews were designed
to provide data to answer research question three. Raw data from
four of the five structured questions were treated for the
statistical test of proportionality described in the introduction to
Chapter 4 because there were agreeing as well as disagreeing
responses. The critical values were determined to be a + or 1.96.
There were no disagreeing responses from survey question 10a,
therefore, the raw data were grouped by superintendent or board
member and then by concept. The frequency of the response option
was then calculated and percentage values were assigned.
Question 10 of the structured interview addressed the
perceptions of superintendents and board members of the improvement
of the overall performance of the superintendents. Questions 12,
13, and 14 addressed the perceptions of superintendents and board
members of the performance improvement of the superintendent in
three specific areas; the management of district's fiscal resources,
the districts planning process, and the district's personnel
supervision and the district's evaluation system respectively.
Question 10a addresses the perception of superintendents and board
members as to the extent to which the overall performance of the
superintendent has improved.
81


Survey Item 10
"The overall performance of the superintendent has improved."
Results of the test of proportion for question 10 are presented
in Table 4.9. As is displayed in the table, 12 out of 15
superintendents agreed with the statement, while 12 out of 14 board
members agreed; however, the test of proportion yielded a test
statistic whose value did not exceed the critical value of 1.96.
Results of the test, therefore, indicated no significance between
the proportion of superintendents and board members agreeing with
the statement.
Table 4.9. The overall performance of the superintendent has
improved.
Class. n No. Agree. Prop. Test Stat. Critical Value Status
Supt. 15 12 .800 -.41 1.96 NS
Board
Members 14 12 .857
Note. NS indicates no significance found; the test statistic did
not exceed the critical value.
82


Survey Item 12
"The management of district fiscal resources has improved."
Results of the test of proportion for question 12 (the
management of district fiscal resources has improved) is presented
in Table 4.10. As is displayed in the table, 12 out of 15
superintendents agreed with the statement while 14 out of 14 board
members agreed; however, the test of proportion yielded a test
statistic whose value did not exceed the critical value of 1.96.
Results of the test, therefore, indicate no significance between the
proportion of superintendents and board members agreeing with the
statement.
Table 4.10. The management of the district's fiscal resources has
improved.
Class. n No. Agree. Prop. Test Stat. Critical Value Status
Supt. 15 12 .800 -1.77 1.96 NS
Board
Members 14 14 1.000
Note. NS indicates no significance found; the test statistic did
not exceed the critical value.
83


Survey Item 13
"The management of the district's planning process has
improved."
Results of the test of proportion for question 13 are presented
in Table 4.11. As is displayed in the table, 14 out of 15
superintendents agreed with the statement while 14 out of 14 board
members agreed; however the test of proportion yielded a test
statistic whose value did not exceed the critical value of 1.96.
Results of the test, therefore, indicated no significance between
the proportion of superintendents and board members agreeing with
the statement.
Table 4.11. The management of the district's planning
process has improved.
Class. n No. Agree. Prop. Test Stat. Critical Value Status
Supt. 15 14 .933 1 VO 00 1.96 NS
Board
Members 14 14 1.000
Note. NS indicates no significance found; the test statistic did
not exceed the critical value.
84