Citation
Educator effectiveness in Colorado

Material Information

Title:
Educator effectiveness in Colorado the connection between evaluation and professional learning
Creator:
O'Brien, Colleen L. ( author )
Place of Publication:
Denver, CO
Publisher:
University of Colorado Denver
Publication Date:
Language:
English
Physical Description:
1 electronic file (110 pages). : ;

Subjects

Subjects / Keywords:
Teachers -- Rating of ( lcsh )
Teaching -- Evaluation ( lcsh )
Teachers -- Training of ( lcsh )
Genre:
bibliography ( marcgt )
theses ( marcgt )
non-fiction ( marcgt )

Notes

Review:
Purpose: The purpose of this study was to understand how schools and districts plan professional learning that provides the ongoing feedback and support needed to improve teacher performance under the new Colorado Educator Effectiveness Law. The study focused on the professional learning that follows a teacher's rating on the new educator effectiveness evaluation tool and on the roles teacher leaders have in the educator effectiveness system. The ultimate goal of this study was to understand and share professional learning practices implemented in the educator effectiveness system so that all students achieve at higher levels and are more engaged in classrooms. Research Design: The study includes district leader interviews and a teacher leader questionnaire, and the combination of these methods identified and explained how schools and districts use the new educator effectiveness law as an opportunity to support a teacher's development post evaluation. This mixed methods study includes 111 teacher leaders from teacher leadership academies in two school districts and the district leaders that support their work including the Directors of The Teacher Leadership Academy, the Directors of Professional Learning, and the Directors of Educator Effectiveness, the Manager of the Teacher Leadership Academy, The Director of Peer Observers, and the Director of Teacher Development and Leadership. The primary finding of this study is that in both districts post evaluation learning did not exist in the implementation of the ne evaluation system. In other words, teachers were not necessarily learning as a result of evaluation. The second major finding was that leaders in District A relied on teacher leaders (TLs) as a major strategy to increase the effectiveness of all teachers and District B aspired to do the same. However, district leaders concluded the teacher leader program could be more intentionally implemented.
Thesis:
Thesis (Ed.D.)--University of Colorado Denver. Education
Bibliography:
Includes bibliographic references.
System Details:
System requirements: Adobe Reader.
General Note:
School of Education and Human Development
Statement of Responsibility:
by Colleen L. O'Brien.

Record Information

Source Institution:
University of Colorado Denver
Holding Location:
|Auraria Library
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
900731742 ( OCLC )
ocn900731742

Downloads

This item has the following downloads:


Full Text
EDUCATOR EFFECTIVENESS IN COLORADO: THE CONNECTION BETWEEN
EVALUATION AND PROFESSIONAL LEARNING
by
COLLEEN L. OBRIEN
B.A., Colorado State University, 1991
M.S., University of Colorado-Denver, 1994
A thesis submitted to the
Faculty of the Graduate School of the
University of Colorado Denver in partial fulfillment
of the requirements for the degree of
Doctor of Education
Leadership for Educational Equity
2014


2014
COLLEEN L. OBRIEN
ALL RIGHTS RESERVED


This thesis for the Doctor of Education degree by
Colleen L. OBrien
has been approved for the
Leadership for Educational Equity
by
Kara Viesca, Chair
Carolyn Haug
Rodney Blunck
May 1, 2014


OBrien, Colleen L. ( EdD, Leadership in Equity)
Educator Effectiveness in Colorado: Following Evaluation with high Quality Learning
Facilitated by Teacher Leaders
Dissertation Directed by Assistant Professor Kara Viesca
ABSTRACT
Purpose: The purpose of this study was to understand how schools and districts plan
professional learning that provides the ongoing feedback and support needed to improve
teacher performance under the new Colorado Educator Effectiveness Law. The study
focused on the professional learning that follows a teachers rating on the new educator
effectiveness evaluation tool and on the roles teacher leaders have in the educator
effectiveness system. The ultimate goal of this study was to understand and share
professional learning practices implemented in the educator effectiveness system so that
all students achieve at higher levels and are more engaged in classrooms. Research
Design: The study includes district leader interviews and a teacher leader questionnaire,
and the combination of these methods identified and explained how schools and districts
use the new educator effectiveness law as an opportunity to support a teachers
development post evaluation. This mixed methods study includes 111 teacher leaders
from teacher leadership academies in two school districts and the district leaders that
support their work including the Directors of The Teacher Leadership Academy, the
Directors of Professional Learning, and the Directors of Educator Effectiveness, the
Manager of the Teacher Leadership Academy, The Director of Peer Observers, and the
Director of Teacher Development and Leadership. The primary finding of this study is
that in both districts, post evaluation learning did not exist in the implementation of the
new evaluation system. In other words, teachers were not necessarily learning as a result
IV


of evaluation. The second major finding was that leaders in District A relied on teacher
leaders (TLs) as a major strategy to increase the effectiveness of all teachers and District
B aspired to do the same. However, district leaders concluded the teacher leader program
could be more intentionally implemented.
The form and content of this abstract are approved. I recommend its publication.
Approved: Kara Viesca


Dedicated to Robert and William


TABLE OF CONTENTS
CHAPTER
I. INTRODUCTION......................................1
Problem of Practice...............................1
Research Questions...............................10
Significance of Study............................11
Local Context....................................12
II. THEORETICAL FRAMEWORK............................13
Critical Theory..................................15
Sociocultural Theory.............................20
III. LITERATURE REVIEW................................24
IV. METHODOLOGY......................................35
Research Design..................................35
Study Participants...............................35
Data Collection
39


Data Analysis
42
Conclusion.....................................50
FINDINGS AND IMPLICATIONS......................52
Findings.......................................52
Implications...................................68
REFERENCES.....................................73
APPENDIX.......................................76
A. Teacher Leader Survey.......................76
B. Interview Protocols
88


CHAPTER I
INTRODUCTION
Problem of Practice
Education is a civil right. It is essential for one to enjoy intellectual, cultural, and
financial freedom, and to attain the benefits of health, wellness, and political power.
Regrettably, too many students in the United States lack the educational experience
required to enjoy such rights and furthermore find it difficult to be eligible to study in
college or to gain access to political and social power (Goldin-Dubois, 1999; National
Center for Education Statistics, 2013).
Educating a human is an incredibly complex endeavor and multiple factors
influence human learning. Parents, educators, policy makers, and community members
cannot always agree on the purpose of education or define specific problems that need to
be addressed in education. Those who think critically about education realize that
focusing on one aspect of education as the root cause of the problem, i.e., poverty, health,
schools, standards, racism, teachers, parents, is an oversimplification of a complex
process and, in reality, all of these factors influence a childs success in school
(Brookfield, 2005). This study recognizes that teacher quality is one critical element of
effective education (Sanders & Rivers, 1996).
In the 2013-2014 school year, Colorado is embarking on another education solution
intended to improve student learning and is instituting a new educator evaluation system
intended to create more effective teachers. The State Council on Educator Effectiveness defines
1


an effective educator as one who knows content, establishes a learning environment, facilitates
learning, demonstrates leadership, reflects on practice, and demonstrates that students have
grown academically as a result of their class (SCEE, 2011). To measure effective teachers in
these six domains of teacher performance, the Colorado Department of Education (CDE) is
crafting a quality standards rubricone aspect of educator effectiveness. Once a teacher is given
a rating on the rubric, each school and district is responsible to provide the feedback and support
that improves the teachers performancethe second aspect of educator effectiveness (SCEE,
2011). Although the state is very clear on the criteria for an effective teacher, no criteria exists
for the feedback and professional learning required to improve teacher performance, and there is
no indication about who will lead teachers towards a higher level of effectiveness. A rating on a
rubric alone will not improve the quality of instruction and is an oversimplification of the
process of teacher growth. Further, it will not promote the collaboration required to improve
teacher learning (Darling-Hammond, 2013). To realize the goal of an effective educator in every
classroom, Colorado policy makers and school leaders must place more emphasis on quality
professional learning structures (Darling-Hammond, 2013; MET, 2012).
Is There a Crisis in Education?
Is there a crisis in education in the United States? Many Americans speak of a crisis
in education, and yet other Americans are satisfied with the education system. In 1983, the
National Commission on Excellence in Education explained their perspective on a crisis in
our country.
Our Nation is at risk. Our once unchallenged preeminence in commerce,
industry, science, and technological innovation is being overtaken by
competitors throughout the world. This report is concerned with only one of
2


the many causes and dimensions of the problem, but it is the one that
undergirds American prosperity, security, and civility. We report to the
American people that while we can take justifiable pride in what our schools
and colleges have historically accomplished and contributed to the United
States and the well-being of its people, the educational foundations of our
society are presently being eroded by a rising tide of mediocrity that
threatens our very future as a Nation and a people. What was unimaginable a
generation ago has begun to occur others are matching and surpassing our
educational attainments. (National Commission on Excellence in Education,
1983, p.3)
The conclusions drawn in A Nation at Risk are further supported by a number of
educational assessments that measure student achievement of academic standards. The
National Assessment of Educational Progress (NAEP), an exam that measures the
educational progress of U.S. students in math, science, and reading, reveals that there has
been minimal gain in student progress in reading since the 1960s (National Center for
Education Statistics, 2013). NAEP also reports a significant variance in the quality of
education between states. For example, longitudinal data over the last forty years reveals
36% of students in Mississippi scored basic or above in math compared to 77% of students
in Maine and North Dakota and 78% of students in Iowa (National Center for Education
Statistics, 2013). The Third International Mathematics and Science Study (TIMSS) reports
that in both math and science, U.S. students consistently decline in their performance on
the TIMMS as their time in school increases, and perform below their international
counterparts in both math and science in 12th grade (National Center for Education
3


Statistics, 2013). Although the racial achievement gap is beginning to narrow, when
student assessment data is disaggregated by race, black and brown children perform at a
significantly lower level than do their white classmates (Loyd, Tienda, Zajacova, 2001;
National Center for Education Statistics, 2013). Adding another layer of complexity to
student achievement, more than eleven million U.S. students have a primary language other
than English spoken at home, yet instruction in the United States remains mostly
monolingual (Auerbach, 1993; Meyer, Madden, McGrath, 2004, National Center for
Education Statistics, 2013; Slaven & Cheung, 2005). For those who believe that education
can be accurately measured by standardized tests, these assessment results bolster the case
of an education system in crisis.
Again, is there a crisis in education in the United States? The Nation at Risk report
and connected standardized tests paint a bleak picture of the education system in the United
States. Paradoxically, many Americans are still satisfied with their childs school. Recent
surveys in New York report 95% of parents report overall satisfaction with the school
system, (Fertig, 2012) and in Washington, D.C. more than 80% of the parents report
satisfaction with their students teachers (DCPS, 2011). The wide range of beliefs brought
forth by educational interest groups confounds clarity about the problems and solutions in
education. Even without clarity on the current state of education, policy makers continue to
proffer a myriad of root causes for problems in education including poverty, failing
schools, and most recently, and the effectiveness of individual educators.
History of Root Causes: Poverty, Failing Schools, In-effective Teachers
In the 1960s poverty was considered the root cause of low academic performance
and President Lyndon Johnson made education a part of his vision of the Great Society and
4


the War on Poverty: A third place to build the Great Society is in the classrooms of
America. There your children's lives will be shaped. Our society will not be great until
every young mind is set free to scan the farthest reaches of thought and imagination
(Johnson, 1964). Johnson worked with Congress to pass the Elementary and Secondary
Education Act (ESEA) of 1965 as a way to improve schools by targeting poverty.
Focusing on this root cause elevated services for children, but also created a culture of
excuse making and low expectations for children in poverty.
In 2002, the No Child Left Behind Act (NCLB) made further adjustments to Title I
of the ESEA and placed the root cause of low student achievement on failing schools
(Stein, 2004). The intent of this revision of the ESEA was to adjust the focus from serving
poor children to serving all children by holding schools to a high level of standards and
accountability. This idea of school accountability came to fruition in the 1970s when the
ESEA was up for reauthorization, and lawmakers sought evidence that the money spent in
schools was yielding results (Stein, 2004, p. 57). NCLB, in short, held schools accountable
for results rather than children.
... [NCLB] does not require schools to label children for services, .. .it posits
the problem of poor performance as an institutional, rather than
individual, concern; it creates accountability mechanisms for student
performance through average yearly progress.. .Finally, it removes past
incentives for identifying a deficient or defective population of students.
(Stein, 2004, p. 133)
Under NCLB States have the authority to set standards and create assessments. In
Colorado, school report cards are provided to the community to ensure public awareness
5


of how children at individual schools performed on state assessments. At the beginning of
each school year, teachers are given the results of their students performance on state
tests the previous year, and this type of reporting was a first step in rating individual
teachers. Ineffective educators are now considered a root cause of low student
achievement on these standardized tests (Stein, 2004;Wong & Rothman, 2009). Focusing
on this root cause elevated the discussion about gaps in student learning but also created a
culture of competition and financial rewards rather than a culture of collaboration and
student rewards (Brookfield, 2005).
Nation wide, a current trend in policy making is improving the quality of teachers
and principals as they are now thought to be a root cause of low performing students.
Research conducted by Sander and Rivers (1996) found that, in school, the teacher is the
most important factor of student achievement, and this finding compelled the State of
Colorado to consider effective educators. In 2010, Colorado Governor Bill Ritter
convened the State Council on Educator Effectiveness (SCEE) to study the educators and
their role in root causes of poor performance in schools. In its final report, The Council
promised to provide every child with an effective teacher and an effective school leader
(SCEE, 2011). One direct result of the work of the Council was the passage of
Colorados Senate Bill 10-191 that focused attention on teacher preparation, teacher
ongoing development, and teacher assessment. Senate Bill 10-191 propelled the state of
Colorado towards a new focus on Educator Effectiveness and a new educator evaluation
system. All students in Colorado will have effective teachers in their classrooms and
effective leaders for their schools. Evaluation provides teachers and principals with clear
expectations for their performance and with ongoing feedback and support needed to
6


improve performance (SCEE, 2011). The SCEE focused on P-20 standards alignment in
the state education system and grappled with the questions: What makes an effective
teacher? What makes an effective principal? How do we measure educator effectiveness?
The Councils goals included an evaluation system by which teachers are:
Evaluated using multiple fair, transparent, timely, rigorous, and valid
methods, at least 50 percent of which is determined by the academic
growth of their students, afforded a meaningful opportunity to improve
their effectiveness, provided the means to share effective practices with
other educators statewide. (Engdahl, 2010)
Under this new law, teachers who do not meet the effectiveness standards for two years
in a row go back to non-probationary status, (SCEE, 2011),yet there is no specific
provision to improve the effectiveness of such a teacher or to improve the classroom
experience for his/her students. Focusing on this root cause elevates the importance of
naming the qualities of an effective educator, but will this focus cause schools to
implement the collaborative professional learning required to improve a teachers
practice?
Because of the stated goals of the SCEE and knowledge and research about
professional learning outlined in the literature review of this document, Colorado schools
have an immense opportunity to use this new Educator Effectiveness law to rethink and
revise professional learning, and to catapult the system into one that benefits all students.
Currently, many schools do not spend ample resources training teachers, nor do they
design teacher learning based on substantiated learning theory, and more focus needs to
be placed on practice in the classroom (Borko, 2004; MET, 2012).
7


The Measures of Effective Teaching Project (MET, 2012), funded by the Gates
Foundation claims:
The nations collective failure to invest in high-quality professional
feedback to teachers is inconsistent with decades of research reporting
large disparities in student learning gains in different teachers classrooms
(even within the same schools). The quality of instruction matters. And
our schools pay too little attention to it. (p. 2)
Like the schools in the MET study, Colorado schools are likely to have similar
discrepancies in both the quality of professional learning and in the levels of student
achievement between classrooms. Without emphasis on professional learning, the design
of this new system assumes that labeling a teacher on a rubric itself will increase that
teachers effectiveness (Darling-Hammond, 2013; MET, 2012). The hope of this study is
to discover how professional development is planned and implemented in the new
educator effectiveness system, and how districts plan to support a teachers improvement
once they have been labeled by the teacher evaluation rubric. This knowledge will inform
other educators as they grapple to create an educator effectiveness system this school
year.
Framed in both critical theory (Brookfield, 2005; Mezirow,1981) and socio-
cultural theory (Hartley, 2009; Tharpe, Estrada, Dalton, & Yamauchi, 2000; Warschauer
1997) the intent of this study was to gather data about the beliefs and the practice
influencing the opportunities for feedback, support, and professional learning that will
follow the teacher evaluation process. In data analysis, the merit of a feedback, support,
and professional learning will be ascertained based on the principles of these theories.
8


Critical theory requires that solutions to this yet undefined problem of effective educators
consider the complexity of the job and reject any solution that simply labels a teacher as
effective or ineffective, a model akin to labeling products on an assembly lines in
factories as effective or ineffective (Denning, 2011; Hargreaves & Fullan, 2012; Smith,
1998). Fortunately, the Colorado Educator Effectiveness Law indicates an appreciation
for the complexity of teaching by including language that brings attention to the need for
evaluation followed by ongoing feedback and quality support (SCEE, 2011). Feedback
and support in education has tended to be oversimplified and low quality (Borko, 2004,
Darling-Hammond, 2013), and critical theory requires that data collected in this study be
scrutinized against the research of quality professional learning outlined in the literature
review of this document. Quality professional learning, designed through the lens of
socio-cultural theory, requires that teachers learn collaboratively by analyzing actual
classroom practice and by applying new learning to their own classroom (Bransford et al,
2000; Tharpe, Estrada, Dalton, & Yamauchi, 2000; Warschauer, 1997). Sociocultural
theory also places value in the apprentice learning model described by Lave and Wegner
(1991) and therefore, the study will seek data about how schools are using expert teachers
as models in these social learning settings. Darling-Hammond (2013) stresses the
importance of professional learning in the evaluation process:
It is important to link both formal professional development and job-
embedded learning opportunities to the evaluation system. Evaluation
alone will not improve practice. Productive feedback must be
accompanied by opportunities to learn. Evaluations should trigger
continuous goal-setting for areas teachers want to work on, specific
9


professional development supports and coaching, and opportunities to
share expertise, as a part of recognizing teachers strengths and needs.
(p.99)
Darling-Hammonds (2013) advisement on quality evaluation mirrors beliefs included in
socio-cultural theory. In this study, professional learning was deemed high quality if it
allowed teachers to collaborate in meaningful ways to improve their planning,
assessment, and instruction. The ultimate goal of this study was to understand
professional learning practices that move beyond simple models of teacher and student
learning and towards effective models of expert teachers, creating social learning
environments that motivate teachers to improve their practice to engage more students in
the classroom.
Research Questions
The study explored the following research questions:
1. How is professional development planned and implemented in the new educator
effectiveness system?
2. What are the perspectives of district leaders on teacher learning as an aspect of the
educator effectiveness system?
3. How do districts plan to support a teachers improvement once they have been
rated on the teacher evaluation rubric?
4. What role are expert teachers expected to take in districts in the educator
effectiveness system?
10


Significance of Study
This study was significant because effective educators are a critical component in
a students civil right to a quality education, one that will give them access to political,
economic, and personal power in the world. In the 1954 United States Supreme Court
Decision, Brown v. Board, Chief Justice Earl Warren argued for the civil right to
education:
Compulsory school attendance laws and the great expenditures for
education both demonstrate our recognition of the importance of education
to our democratic society. . Today it is a principal instrument in
awakening the child to cultural values, in preparing him for later
professional training, and in helping him to adjust normally to his
environment. In these days, it is doubtful that any child may reasonably be
expected to succeed in life if he is denied the opportunity of an education.
Such an opportunity, where the state has undertaken to provide it, is a
right which must be made available to all on equal terms. . (Landmark
Cases, 2013)
In the spirit of this ruling, all students in Colorado are entitled to effective educators who
facilitate learning experiences to ensure access to power in our society and who recognize
the individual needs and strengths of each child. Student power in society comes from an
understanding of social constructs such as the common justice system, the common
military, and the common economic systems in the United States, and requires that all
students build the reading, writing, and speaking skills required to communicate their
needs to those in power (Goldin-Dubois, 1999). Students are also entitled to be in schools
11


where leaders have a clear vision for student learning and support teachers in a climate
that promotes perpetual learning (SCEE, 2011). Further, the Colorado Constitution
ensures .. All Colorado children equal access to thorough and uniform educational
opportunities (Lobato v. Colorado, 2013). This notion of all students having equal
access to a high quality education should be the beacon for the educator effectiveness
concept and law in Colorado. Finally, the study was also significant because Colorado
had just mandated that an Educator Effectiveness System be implemented in every
school.
Local Context
This study took place in two school districts in a large metropolitan area. District
A was in the third year of implementation of a new teacher evaluation system and created
its own evaluation tool. District B was in the first year of implementation of a new
teacher evaluation system and adopted the state evaluation tool. Both school districts
value teacher leadership as evident by their teacher leadership academies.
12


CHAPTER II
THEORETICAL FRAMEWORK
The quest for educator effectiveness has a long history (Markley, 2004); however,
the codification of educator effectiveness in a new law will likely cause rapid change in
the education system. In the next school year, educators and policy makers are called to
think quickly and carefully about how to enact this new educator effectiveness law, and
implementation responsibility falls largely on the school districts (SCEE, 2011).
Conceivably, educators and policy makers implementing this new law have many
different perspectives about educator effectiveness, and applying a theoretical framework
may bring clarity and alignment to this rapidly changing problem of practice. To frame
my thinking about educator effectiveness in this study, I examined the theoretical
frameworks of critical theory and sociocultural theory.
Critical theory is a collection of beliefs that hold that humans need to think and
reflect on the beliefs that are driving daily actions and be aware of the hegemony of
capitalism that drives all that we do in life and subsequently in education (Brookfield,
2005). Looking through the critical theory lens, I am able to see how some policy makers
may look at educator effectiveness from a point of view which values the individualism,
capitalism, and oversimplification of issues in education (Hargreaves & Fullan, 2012;
Smith, 1998). Because of my bias as a long time educator and staff developer, I focused
on research and solutions that recognize that education is complex and has multifaceted
solutions. I rejected the idea that schools need to be run like a business that promotes the
individual and competition. Alternatively, in this study, I placed value on the collective
approach of teacher learning supported by sociocultural theory (Brookfield, 2005;
13


Hartley, 2009; Peck, Gallucci, Sloan, &Lippincott, 2008) which posits that learning takes
place in social settings and bolsters the notion that teachers must work together to
become more effective educators (Warschauer, 1997). Further, Bransford et al. (2000)
supports that sociocultural theory is the framework for the apprenticeship model of
learning that lends credence to utilization of teacher leaders as a strategy to guide the
learning of apprentice teachers. My penchant towards sociocultural theory and the power
of teacher collaboration leads me to believe teacher learning is paramount to student
learning, and I believe that thoughtful teachers can learn together to improve their
practice and become more effective for children.
Critical Theory
Critical theory encompasses a web of ideas that encourage humans to think deeply
about and reflect on the beliefs that are driving daily work and actions (Mezirow,1981).
One facet of critical theory is the notion that capitalism, sometimes unconsciously, drives
thinking and action in most western political systems including education policy. To
illustrate this point, Karl Marx wrote that capitalism works to concentrate the majority of
the resources to the few and enslaves the others; this desire for resources dominates all
social relationships (Brookfield, 2005). Stephen Brookfield (2005) builds on the ideas of
Marx and defines critical theory in four parts: 1) The exchange of goods in the economy
generates a series of tensions created by the desire of some people for emancipation and
the wish of others to prevent this desire being realized (p. 23); 2) People need
knowledge to free themselves from the oppression of capitalism; 3) Human beings are
not just objects in critical theory, instead humans embrace the theory and want to live in a
world that embodies this theory; 4) The theory creates a vision for how people might live
14


more collectively in the future. Brookfield (2005) asserts that the education system is
dominated by the individualism of capitalism and explains that adult development is
collectively formed, and one person cannot be economically free if another is not. Critical
theory forces me be watchful of ideas and systems related to the new educator
effectiveness law that encourage teachers to compete against one another for high ratings
on a rubric or for higher pay at the expense of working towards the collective good of
increasing achievement and engagement for all students. As I conducted this research
through the lens of critical theory, I was critical of ideas and solutions that fail to
acknowledge the complexities of educating humans and try to fix schools by running
them like factories or businesses.
Critical Theory, Education, Capitalism
The capitalist view of the world influences the structure and culture of both
student and adult learning in schools (Brookfield, 2005; Hargreaves & Fullan, 2012;
Smith, 1998). According to Brookfield (2005), critical theory posits that examining
education through a capitalist viewpoint allows educators and policy makers to
.. Do something highly complex (help humans learn) within a system that is
organized according to bureaucratic rationality and modes of factory production.
Such system ignores complexity and assumes, for example that learning takes
place at predictable times each week, in the same location, and follows the
rationale of a curriculum divided into discrete and manageable units, (p. 6)
An illustration of the oversimplifications of education can be connected to the
comparison of technical problems to adaptive challenges. In short, one person can easily
fix a technical problem with a simple solutiona student has no lunch money, lend her
15


two dollars. An adaptive challenge is multifaceted, complex, involves many people and
organizations, and seeks a long-term solutiona student has no lunch money, the
educator determines she is living in poverty and does not have access to healthy food on
many days. Consult a network of experts to develop a plan for long-term resolution to her
hunger problem. This model cautions leaders against employing technical fixes for
adaptive challenges (Heifetz & Linsky, 1997). In my opinion, applying a capitalist model
to fix schools and improve teacher performance is a technical fix and will fail to bring
about long term, sustainable solutions for students.
Critical theory requires critical thinkers to become aware of capitalisms
influences on the education system that is often compared to a factory system. Frank
Smith, in his book The Book of Learning and Forgetting (1998), traces capitalisms roots
in education back to the influence the military had over factories and subsequently the
influence factories had over schools:
This was the militaristic model that invaded the one-room schoolhouse.
What specifically went into schools was walls-(or rather barricades).
Students would no longer be mixed up together, but instead would be
grouped according to age and ability, all to be treated the same way and
expected to learn together throughout their school careers, until they
graduateda standardized, predictable, and reliable product, (p. 47)
Frequently, students and adults within the K-12 system are taught in the same format,
with the same curriculum, in the same time frame, and are measured by the same
assessments (Denning, 2011). Like a factory, teaching in some schools and districts has
been reduced to scripts and step-by-step procedures and ignores the students for the sake
16


of uniformity of the system (Denning, 2011; Hargreaves & Fullan, 2012). The factory
model is not a match for the complex process of educating humans (Denning, 2011;
Hargreaves & Fullan, 2012; Smith, 1998) because so many moving parts are required to
educate individual students and teachers.
Further, critical theory requires critical thinkers to examine the influence of
capitalism that stretches beyond K-12 education and into higher education. Without a
college degree, students have little possibility of earning a living wage in our capitalist
society. In order to gain entrance into a college or university, students must be exposed to
a common curriculum of knowledge and skills which prepare them to pass a college
entrance exam created by a corporation. As a result of this system, such corporations
dictate curriculum in schools (PBS, 2013). Further bolstering the need to be critical about
the influence of capitalism in the education system is the fact that the same corporations
that create the college entrance exams also guide curriculum and sell standardized testing
packages to K-12 school systems. Such corporations are estimated to earn between $400
and $700 million dollars a year in the standardized test and curriculum market (PBS,
2013). For example, College Board creates and scores the SAT college entrance exam,
creates the Advanced Placement exams, and is creating the new Common Core State
Standards as well as the supporting curriculum materials such as Spring Board
(CollegeBoard.org, 2013). The influence and power College Board and other businesses
have on education is enormous and well beyond the power of a classroom teacher,
student, parent, or principal.
Critical Theory in Educator Effectiveness
As the country and the state of Colorado begin to implement the new educator
17


effectiveness law, policy makers, educators, and community members must be critical of
the beliefs and theories that beget the definition of an effective educator. As a nation, we
are not in agreement about the purpose of education nor the qualities of an effective
educator; this ambiguity in framing of the problem does not lend itself to solutions. One
belief about education is that an effective educator builds the self-esteem of a child or
inspires great musicians and artists (Smith, 1998). Another belief, held by Thomas
Jefferson, is that effective educators build citizens and such education is necessary for
self-government (Jefferson, 1789). The adoption of the Common Core State Standards is
evidence of the current belief that an effective educator causes growth in student learning
on commonly agreed upon content and skills standards (Common Core, 2013). The belief
that is most closely aligned with capitalistic priorities is the idea that effective educators
cause children to gain the knowledge and skills that will allow the children to sustain the
economy and this idea is termed Value-Added.
Value-Added models seek to understand how effective teachers cause student
achievement of specific knowledge and skills that create an educated citizenry who add
to a viable economy as measured through standardized test scores. These models measure
the effect of a school and a teacher on a student achievement by using one or more years
of student test scores combined with other variables (National Research Council and
National Academy of Education, 2010). Researchers from the National Bureau of
Economic Research assert, students assigned to high-VA teachers are more likely to
attend college, attend higher-ranked colleges, earn higher salaries, live in higher SES
neighborhoods, and save more for retirement (Chetty, Friedman, & Rockoff, 2011, p. 1).
In this line of thinking, the objective of an educator is to cause students to grow each year
18


and to ensure that the student becomes more financially viable in our capitalistic society.
This judgment fails to mention other reasons one might seek an education. In my opinion,
educational stakeholders need to be critical of the perspectives and beliefs that are driving
the implementation of the new educator effectiveness law so that sound policy is created
for all students. Such sound policy would include a broader vision for the purpose of
education, which would require critical thinkers to measure student growth using a
variety of assessments beyond standardized tests. Further, the vision of an effective
educator would be broadened, and an educators value would be measured beyond
students standardized tests scores, which are not sensitive enough to determine a
teachers value.
One idea in critical theory holds that adults (in this case educators) become more
effective when they work together collectively (Brookfield, 2005), and without careful
thinking, the Educator Effectiveness Law may cause competition among teachers that is
counter productive to the collective focus of meeting the needs of all students in a school
(Hargreaves & Fullan, 2012). For example, high stakes evaluation may not be productive
for increasing the effectiveness of educators, because it might prevent educators from
sharing strategies and information that might give one teacher an advantage over another
in the evaluation system (Hargreaves & Fullan, 2012). Effective educators must be wary
of the perils of a competition-based system and look toward a more collective and united
approach to helping all students grow (Brookfield, 2005). In my opinion, policy makers,
educators, and community members need to abandon the capitalistic view of education
and seek a new framework to think about creating more effective teachers in the
classrooms one such framework is sociocultural theory.
19


Sociocultural Theory
Sociocultural theory is a web of ideas attributed to the work of Vygotsky and
posits that learning takes place in social settings before it becomes internalized (Tharpe,
Estrada, Dalton, & Yamauchi, 2000); Sociocultural theory is essentially the theory of
development (Vygotsky, 1978) (Tharpe et al., 2000, p. 9), and this theory is related to
effectiveness because it provides a framework to think about change and development of
humans (Tharpe et al., 2000). Vygotsky (1978) writes, The transformation of
interpersonal processes into an intrapersonal one is the result of a long series of
developmental events (p. 57), and this lends credence to the adaptive challenge of
educating both teachers and students. Warschauer (1997) described:
Vygotsky further believed that this development principally took place through a
form of apprenticeship learning; interaction with teachers or peers allowed students to
advance through their zone of proximal development (i.e., the distance between what
they could achieve by themselves and what they could achieve when assisted by others).
(No page)
Although Vygotsky did not create sociocultural theory, many scholars have linked the
import of collaborative work, between teachers or students, to his work (Bransford,
Brown, Cocking, Donavan, Pelligrino, 2000; Tharpe et al., 2000; Warschauer, 1997).
Lave and Wenger (1991) built on the work of Vygotsky and described an apprenticeship
model of learning as a method for the novice to be mentored by the expert (Bransford et
al., 2000), and this notion provides support for using highly effective teachers as leaders
for their colleagues.
20


Sociocultural Theory in Education
Sociocultural theory is a cornerstone of thoughtful learning structures and holds
that learning is cultural and social rather than an individual experience (Kozulin, Gindis,
Ageyev, Miller, 2003). Learning environments that promote a sense of community
through a common culture of safety, experiment, interaction, and teacher and peer
feedback, enhance student learning experiences (Bransford, et al., 2000). Bransford, et
al., (2000) explain In such a community, students might help one another to solve
problems by building on each others knowledge, asking questions to clarify explanations,
and suggesting avenues that would move a the group towards its goal (Browne &
Campione, 1994) (Bransford, et al. 2000, p. 25). Additionally, Kozulin, et. al, (2003)
synthesized ten years of classroom observations and found that students in collaborative
classrooms increase understanding of concepts, rather than simply acquire knowledge. In
a collaborative classroom, discussions are carefully designed, and the teacher
differentiates uses probing questions to meet the individual needs in the discussion.
Thoughtful 21st Century teachers embrace sociocultural theory when students use
technology as a tool to collaborate and make meaning. For example, Bonk (1998)
asserted that technology alone does not increase student learning however, when the tool
is used for collaboration it engenders the cognitive and emotional connections required to
ensure learning. Sociocultural theory calls for safe learning environments that value the
process of learning without fear of mistake, growth through peer feedback, and learning
amongst peers, and these same concepts must be applied when considering the growth of
effective educators.
21


Sociocultural Theory in Educator Effectiveness
Evaluation alone will not increase effectiveness of an educator rather,
professional learning, framed in this theory of social construction of knowledge, will
promote learning among teachers (Darling-Hammond, 2013). In investigations of teacher
learning and change by Peck, Gallucci, Sloan & Lippincott (2009), a connection is made
between collaborative teacher development and the sociocultural theory: We conclude
that the sociocultural learning theories.. .may offer a useful theoretical framework for
interpreting complex social processes underlying organizational renewal, innovation, and
change (p. 16). To illustrate this point, Peck et al. (2009) found that in cases of new state
policies and mandates, individual teachers developed creative implementation ideas, but
policies did not take hold until a collective group of faculty embraced the change in
practice, and further, this change process almost never occurred in the presence of an
administrator or leader. Collaborative teacher learning that brings about teacher change is
described in the literature review and may take many forms including mentoring,
apprenticeship relationships between expert and novice, peer laboratories and teacher led
in-service (Bransford et al., 2000; Darling-Hammond, 2013; NBPTS, 2011). To ensure
that an effective teacher is in each Colorado classroom, CDE has outlined guidelines for
principals to conduct teacher evaluations (CDE, Fact Sheet, 2013); however, research
supports that teacher change comes from collaboration among teachers and often away
from administrators (Hargraves & Fullan, 2012; Peck et al, 2009). School districts, in
implementation of the new state mandate and in pursuit of effective educators, must
include collaboration as a pathway to teacher growth, change, and effectiveness.
22


Research has been conducted on effective educators, professional learning, and
teacher leaders, and critical theory and sociocultural theory provided valuable
frameworks to review the relevant literature for this study. In accordance with these
theories, research studies were included in this study if they recognized the complexity of
teaching and learning and if they promoted collaborative teacher learning.
23


CHAPTER III
LITERATURE REVIEW
Literature Review: Evaluations; Professional Learning; Teacher Leaders
As Colorado schools build a plan to have an effective educator in each classroom,
policy makers and educators have a host of research studies to inform their decisions on
topics including effective evaluation, effective professional learning, and effective
utilization of teacher leaders. The literature for this study included thirty peer-reviewed
research articles and seven books that describe research on the topics related to this
proposed study. Research studies were considered a good match for this literature review
if ideas presented harmonized with critical theory and sociocultural theory; More than
twenty studies were rejected because they did not meet this criteria. Instead, they offered
simple solutions to the complex idea of changing a teachers practice or because studies
did not talk about the collaborative nature of teacher learning. This literature review
begins by examining research and recommendations for effective evaluation, then
identifies and describes components of effective professional learning, and finally gives
examples of emerging practices in training and utilizing teacher leaders.
Effective Evaluations
To be successful, the new state evaluation system must cause educators to be
more effective (SCEE, 2011). To accomplish this goal, CDE requires that evaluation is
ongoing and includes an educator self-assessment, a review of annual goals and
performance plan, a mid year review of annual goals and performance plan, a final rating
goal setting and performance planning for the next school year (CDE, Fact Sheet, 2013).
A longitudinal study will have to be conducted to determine if the Colorado evaluation
24


steps create more effective teachers, but in the present, the requirements, as written
above, do not specify the constitution of post evaluation learning opportunities that likely
influence the probability of educators becoming more effective.
Educator effectiveness is a nation-wide challenge of practice, and a wide variety
of educational stakeholders and scholars have crafted recommendations for the
implementation of a quality teacher evaluation system (MET, 2012;Munoz, Prather, &
Stronge, 2011). Nearly all research in this literature review cautions that teacher
evaluation must be based on multiple data points, including teacher observation, student
surveys, and student growth scores (Chetty, Friedman, & Rockoff, 2011; MET, 2012;
Munoz, Prather, & Stronge, 2011). The National Board of Professional Teacher
Standards (NBPTS), suggests that a new evaluation system is necessary but must be
implemented correctly by convening the right stakeholders, specifying what is to be
measured, defining the process of measuring, clarifying how the measures will be applied
consistently, defining the evaluation process, and defining ongoing support (NBPTS,
2011). The New Teacher Projects (TNTP) (2012) report on evaluation describes the
importance of treating all teachers as individuals and therefore differentiating the
evaluation process based on student and teacher needs (TNTP, 2010). Research
conducted by the Measures of Effective Teaching project (MET) (2012), which is
sponsored by the Bill and Melinda Gates Foundation, found that teacher evaluators must
be well trained and show their ability to use the instrument before using it in a high stakes
setting. The MET project further found that to ensure a reliable rating, each teacher
would require four observations, each conducted by a different observer (MET, 2012).
Another data point, student surveys have been shown to render reliable and valid teaching
25


ratings (Kyriakides, 2005; MET, 2012). School districts in Colorado can utilize the
findings of these research studies as they design their teacher evaluation cycles and
evaluation tools.
Student growth scores are another possible measure of an educators
effectiveness. A teachers purpose is to help students learn and it seems reasonable to
measure a teachers effectiveness using student growth scores; however, studies show
mixed reviews about the validity of this practice (Goe, Bell, & Little, 2008; National
Research Council and National Academy of Education, 2010: Steele, Hamilton, &
Stecher, 2010). Value added models measure the effect of schools and teachers on a
student achievement by using one or more years of student test scores and other variables
connected to learning (National Research Council and National Academy of Education,
2010). Some researchers posit that too many variables exist to reliably connect a teacher
to a student test score (Goe, Bell, & Little, 2008; National Research Council and National
Academy of Education, 2010: Steele, Hamilton, & Stecher, 2010). Other researchers find
value added scores are a valid and should be used as one measure of a teachers
effectiveness (Chetty, Friedman, & Rockoff, 2011; National Research Council and
National Academy of Education, 2010: Steele, Hamilton, and Stecher, 2010). Regardless
of their belief in the validity of using test scores to measure a teachers effectiveness,
most of the experts mentioned above do believe that student achievement scores must be
combined with other measures of teacher effectiveness to yield a valid and reliable rating.
Effective Professional Learning
Learning Forward (2013), a national consortium for staff development, proposed
a definition of professional learning to be used in the reauthorization of the Elementary
26


and Secondary Education Act: The term professional development means a
comprehensive, sustained, and intensive approach to improving teachers and principals
effectiveness in raising student achievement (Learning Forward, 2013). The work of
many researchers has helped to illuminate specific characteristics of effective
professional learning and these include being participant driven, being data driven, giving
specific feedback on clear targets, modeling, and coaching. These notions are described
in the literature review.
Participant Driven. Participant driven indicates that learning is based on the
stated needs and preferences of teachers. Professional learning must be planned based on
the needs of teachers, be facilitated through teacher-centered instruction, and be grounded
in teacher inquiry, reflection, and experimentation (Allen, 2007; Cross, 2012; Darling-
Hammond & McLaughlin, 1995). Hewitt and Weckstein, (2012) offer an illustration of
this practice from teachers in Oakwood, Ohio who improved their practice by self-
selecting professional learning goals based on student assessment results. Following data
analysis, teachers in Oakwood choose from a menu of professional learning experiences
including joining a specialized professional learning community (PLC) or observing an
expert teacher (Hewitt & Weckstein, 2012). Thibodeau (2008) found when teachers have
choice in PLC membership, choice in PLC outcomes, and choice in PLC norms, PLCs
are most effective in making instructional practices that benefit students.
Data Driven. Data driven is the practice of using results from student
assessments to plan professional learning (Thompson, n.d.). Successful PLCs examine
student work including standardized tests, writing samples, and other formative
assessment; which provides direction to meet student needs and subsequently,
27


professional learning needs (Cross, 2012; Darling-Hammond & McLaughlin, 1995;
Smith, Johnson, &Thompson, 2012). Student data drives discussion in small PLCs, where
one teacher presents a successful strategy used to render quality work, and student data is
utilized to plan a school-wide workshops on a school improvement goal (Smith, Johnson,
&Thompson, 2012).
Specific Feedback on Clear Targets. A clear target refers to a teacher learning
goal that is based on data and followed by a teacher action for growth (i.e., participating
in a lesson study) (Learning Forward, 2013). Professional learning should be aligned to
targets which enhance student outcomes, are mutually agreed upon among the teacher,
the evaluator, and the staff developer, are grounded in learning research, and are aligned
to the school or district evaluation rubric (Cross, 2012;Leaming Forward, 2013; MET,
2013). More specifically, Cross (2012) provides an illustration of this concept from
instructional coaches in Fort Lupton, Colorado who began their work by pouring through
research on adult learning which lead to the co-creation of a professional learning rubric
(Cross, 2012). The targets should include both content targets and process targets (i.e.,
literacy or critical thinking) and should be discretely worded so that follow up feedback is
actionable and can cause a change in practice (Cross, 2013;MET, 2013). Additionally,
Wiliam (2012) advises that specific feedback requires an atmosphere of trust, a focus on
the data rather than the person, and a focus on aspects that are within the recipients
control.
Modeling. Traditionally, teaching is a profession that takes place in the isolation
of peers, but promising practices including peer observation and teaching laboratories
allow teachers to learn more about their craft from a primary source or model (Darling-
28


Hammond & McLaughlin, 2013; NBPTS, 2011). Peer observations take place in many
forms and are often informal, formative, and non-evaluative (Center for Teaching and
Learning, 2013). In peer observation, teachers gain insight into their own practice when
they watch expert teachers teach children, and reflect on the practice in the observation
(Darling-Hammond & McLaughlin, 1995; NBPTS, 2011). Further research about peer
observation conducted by City (2011) found that well-planned peer observations might
focus on a specific guiding question or problem of practice that turns the observation into
a collective problem solving experience. City (2011) also discovered that peer
observation is different than evaluation because all participants learn from the experience
and change practice based on the observations and subsequent debrief. Laboratory (lab)
classrooms may take place in many forms but often feature one teacher who models
exemplary practice in one or more aspects of teaching and learning (Center for Teaching
and Learning, 2013). Lab classrooms provide support through modeling and directly
connect to the work of teachers and their students (Darling-Hammond, & McLaughlin,
1995). They also utilize teacher leaders as role models and mentors for effective practice
(NBPTS, 2011).
Coaching. Instructional coaching has many forms, but often includes one-on-one
conversations between a teacher and a non-evaluative coach around the topics of
planning, assessment, and instruction (Center for Teaching and Learning, 2013). Kraft
and Balzer (2012) found that teachers who received instructional coaching were more
effective for students than those teachers who did not. In another example, Cornett and
Knight (2008) found that teachers who just attend a workshop had a 15% implementation
29


rate of the new skill learned. In contrast, teachers who attend a workshop and have follow
up coaching have an 85% implementation rate of a new skill.
Professional Development Increases Student Achievement
Research has shown that investing in teacher learning causes gains in student
learning (REL, 2007; Slabine, 2012; Taylor & Tyler, 2011) and increased student
learning is the ultimate goal of the educator effectiveness concept (SCEE, 2011). Because
learning is complex, it is challenging to connect one teachers work to one students
learning: The connection seems intuitive. But demonstrating it is difficult (Regional
Education Laboratory [REL], 2007, p. iii). Taylor and Tyler (2011) found that student
achievement increases in years of teacher evaluation and especially in the three years
following evaluation if investments in professional learning are made and quality
feedback is given. A case study in Duval County Public Schools in Florida presented by
Slabine (2012) explains the school district has made professional learning its top district
priority and reports a consequent increase in student achievement. The professional
learning plan for the district includes placing professional learning as the top priority,
district level coordination of all professional learning opportunities, active pursuit of
professional learning grants, time during the school day for teacher collaboration, and a
professional learning accountability board that uses progress monitoring tools to track
progress of professional learning (Slabine, 2012). The Shultz Center for Teaching and
Leadership, a partner organization of Duvall Public Schools, reports a significant
difference in student achievement on the Florida State Reading test as teachers increased
their days in professional learning (Slabine, 2012).
30


Effective Professional Development Led by Teacher Leaders
The New Educator Effectiveness Law calls for an increased use of teacher
leaders; Colorado State Senator Michael Johnston, the author of the Educator
Effectiveness Law in Colorado wrote (2013):
SB 191 recognizes that there are gaps in the teacher career ladder. Besides
taking an administrative position, highly effective teachers are often
without options if they want to remain in the classroom while assuming
leadership roles. SB 191 empowers the Governors Council to create
rewarding career ladders for these educators, ensuring that highly effective
teachers will have compensated opportunities to become leaders in their
field by recording and disseminating their best practices to educators
across the state.
The law, in its final version, did not include a plan for highly effective teachers to
become leaders, but Colorado can learn from the work of many school districts that are
building the skills of teacher leaders and using this expertise to improve instruction.
Research suggests that principals need help to provide an effective educator in
every classroom, and this distributive model of leadership could include expert teachers
leaders (Barth 1990; Fink, 2013; Hallinger, 2003; Lambert, 1998). The label expert or
highly effective educator is nebulous in Colorado because most districts are just
beginning to evaluate teachers with the new educator effectiveness rubric this school year
(SCEE, 2011). Regardless of this complexity, it is likely that there are expert teachers in
all school districts that can be leveraged now to increase the effectiveness of all teachers
in all classrooms. Sociocultural theory supports leveraging expert teachers as leaders in
31


the implementation of the new educator effectiveness system. The apprentice model of
learning, introduced by Lave and Wegner (1991), which calls for experts to work along
side novices as a way for both to improve their craft (Bransford et al., 2000), is a useful
model for this work. Additionally, when principals collaborate with expert teachers to
design and implement professional learning, teachers in the building tend to have more
engagement and trust in the learning activities (York-Barr & Duke, 2004).
There are benefits to using teacher leaders, but also, there are consequences when
teacher leaders are not utilized; To illustrate this, Curtis (2013) warns that, When
districts fail to use high performing teachers to support their colleagues growth and
development, administrators and coaches remain overburdened, and teachers as well as
many students are left lacking individualized support (p. 1). Individually increasing the
effectiveness of every educator in Colorado is a heavy lift, and teacher leadership is a
possible leverage point to meet this demand. To ensure that every child has an effective
teacher in the classroom, principals and expert-teacher leaders must collaborate to design
professional learning experiences for all teachers. Additionally, to ensure that expert
teachers remain in the classroom, principals and districts must provide a career path for
expert teachers that utilizes their skill set and provides challenges that keep them engaged
in classroom work.
Principals and Expert Teachers As Partners in Professional Learning
Even if a principal has the time and the expertise for instructional leadership, a
positive school culture may require that principals lean on expert teachers to create and
sustain strong professional learning experiences for all teachers. Principals are ultimately
responsible for evaluation, feedback, and post evaluation support (SCEE, 2011),
32


however, research shows that they must partner with teacher leaders, who are effective
educators, to realize the goal of an effective educator in every classroom (York-Barr &
Duke, 2004). Principals have a wide array of responsibilities that siphon time each day
and some principals lack the expertise to provide instructional leadership; Fink (2012)
found in a survey of close to 3,000 principals, most self reported being a novice or
emerging in instructional leadership in the areas of purpose, engagement, assessment,
curriculum, and classroom climate. Hallinger (2003) examined the issue of instructional
leadership among principals and found .. .any intention to provide instructional
leadership, especially in secondary schools, is complicated by the fact that in many cases
principals have less expertise than the teachers whom they supervise (p. 331). An
additional factor, a study by Horng, Klasik, and Loeb (2009) explained that principals
may lack the time to focus on instruction. Their yearlong study of principals in Florida
exemplifies this issue and found that high school principals spent 5.8% of their day on
instructional tasks, middle school principals spent 8.38% on instructional tasks,
elementary principals spent 9.3% of their time on instructional tasks. In light of this
research, effective teachers must be used as leverage to increase the learning of all
students.
Gaps in the Research
Taken together, the research reviewed names the characteristics and practices of
quality professional learning, shows that quality professional learning has caused student
achievement, and calls for an increased use of teacher leaders in professional learning.
The research described in this literature review does not include examples of how schools
use teacher learning in evaluation and educator effectiveness systems. This study sought
33


to understand how school systems move beyond a rating on an evaluation tool and
towards post-evaluation, collaborative professional learning that increases the
effectiveness of educators; Therefore, the question for this research study remained, how
is professional development planned and implemented in the new educator effectiveness
system? (Borko, 2004; Hargreaves & Fullan, 2012; MET project, 2012). Additionally,
Colorado did not include a career ladder for effective teachers in the final version of its
law and a second question remains, what role are expert teachers expected to take in
districts in the educator effectiveness system? This study sought to understand the ways
districts plan to use teacher leaders, who understand the complexity of educating students
and teachers, in their educator effectiveness systems.
34


CHAPTER IV
METHODOLOGY
Research Design
The purpose of this study was to understand how schools and districts are
considering professional learning as a vehicle to increase the effectiveness of educators.
To reach this understanding, research was conducted in two school districts to discover
the plans for post evaluation professional learning. In both districts teacher leaders,
evaluators, and district administrators participated in a questionnaire and an interview
(Creswell, 2014; Gliner, Morgan, & Leech, 2009) and the combination of these methods
will seek the answers to these research questions: How is professional development
planned and implemented in the new educator effectiveness system? What are the
perspectives of district leaders on teacher learning as an aspect of the educator
effectiveness system? How do districts plan to support a teachers improvement once
they have been rated on the teacher evaluation rubric? What role are expert teachers
expected to take in districts in the educator effectiveness system?
Study Participants
To seek answers to my research questions, I used a purposive sample (Gliner,
Morgan, & Leech, 2009) of 112 teacher leaders in two school districts who shared their
thinking about educator effectiveness and post evaluation professional learning through a
questionnaire. Next, I interviewed three central office administrators in both districts to
gain their perspective on teacher learning within the new teacher evaluation system and
35


to understand their thinking about the data rendered from the teacher leader
questionnaire.
These participants who have different types of jobs (teacher leaders and central
office administrators) offered diverse perspectives on educator effectiveness and the
professional learning that followed an evaluation rating (Creswell, 2014; Merriam, 2009).
The teacher leaders in the study offered insight into the research questions because they
understand quality instruction, they have been evaluated, and they have received specific
training on effective professional learning through the teacher leadership academies.
Central office administrators provided insight for the research questions because they
collaborated with state officials to gain understanding on the legal mandate of the
educator effectiveness law, and they have experience in planning how a large school
system manages teacher accountability and professional learning within educator
effectiveness and the new teacher evaluation system.
Choosing Expert Teacher Leaders
Almost 8,000 teachers are employed in School District A and B and a participant
selection process will narrow the sample to 112 teachers who provided insight into the
research questions. At the time research was conducted, not all teachers were rated on the
state educator effectiveness rubric; therefore, I could not choose a highly effective
teacher based on this measure for my study. Instead, I asked the leaders of the Teacher
Leadership Academies to distribute the questionnaire to teacher leaders and attempted to
collect data at least 100 teacher leaders to respond to the Educator Effectiveness Staff
Development Support Questionnaire (Appendix A).
36


Interviewing District Administrators
Interviews with central office administrators were grounded in the data from the
results of the teacher leader questionnaire. Once questionnaire results were analyzed,
central office administrators, in both districts (n=6), including the Director of Teacher
Development and Leadership, the Director of the Peer Observers, the Manager of the
Teacher Leadership Academy, the Director of Educator Effectiveness, the Director of the
Teacher Leadership Academy, and the Director of Professional Learning were
interviewed to gain additional perspective on trends from the questionnaire and beliefs in
educator effectiveness, post evaluation teacher learning, and teacher leadership. This
combination of survey, interview, and diverse job type triangulated the data and give
broader insight into my research questions (Creswell, 2014; Merriam, 2009).
Data was collected in two large, school districts; School District A is considered
urban, has 155 schools, and reports the following demographics: ethnicity (15% Black,
58% Brown, and 20% White), free and reduced lunch (72%), and 35% of students are
ELL. School District B is considered suburban, has 61 schools, and reports the following
demographics: ethnicity (12% Black, 18% Brown, and 56% White, 18% Asian), free and
reduced lunch (27%), and 8.8% of students are ELL.
District A and District B, were selected because they are in different stages of
implementation of the educator effectiveness system and this difference yielded a wider
range of perspectives on the subject. In 2010, two years before the educator effectiveness
state mandate, District A began development on an educator effectiveness system and
created a new evaluation tool. District B is in its first year of implementation and chose
to use the state evaluation tool. Additionally, both District A and District B were selected
37


because both place value on teacher leadership as evident by their teacher leadership
academies and one of the aims of this study was to collect data about how teacher leaders
are being leveraged to increase the effectiveness of all teachers (Creswell, 2014). Table
2.1 list study participants.
Table 2.1
Study Participants
District A
Questionnaire
789 Teacher leaders were contacted
84 Teacher leaders Responded
Total Questionnaires from Both
Districts =112
Interviews
District A
Central Office: Director of Teacher
Leadership Academy n=l
Central Office: Director of Professional
Learning n=l
Central Office: Director of Educator
Effectiveness n=l
Total Interviews in District A=3
Total Interviews in both districts=6
District B
69 Teacher leaders were contacted
27 Teacher leaders responded
District B
Central Office: Director of Teacher Leadership
Academy n=l
Central Office: Director of Professional
Learning n=l
Central Office: Director of Educator
Effectiveness n=l
Total Interviews in District B=3
Data Collection
Educators faced with the challenges and opportunities of educator effectiveness
shared their professional experiences in the data collection process through a
questionnaire and interviews. Both quantitative and qualitative data was collected to
answer the four research questions and this variety of sources allows for triangulation of
data (Creswell, 2014; Johnson, Onwuegbuzie, & Turner, 2007; Merriam, 2009). Results
38


from the quantitative questionnaire of teacher leaders helped me to be more pointed in
my interview questions with the central office administrators (Johnson, Onwuegbuzie, &
Turner, 2007). Qualitative interview data was thick, explained questionnaire results in
from a different perspective, and captured the individual experience and perspectives that
exist for those who coordinate the Educator Effectiveness, the new teacher evaluation,
and professional learning at a district level (Merriam, 2009).
Questionnaire
An electronic questionnaire gathered quantitative data from a sample of teacher
leaders to determine beliefs about the educator effectiveness evaluation tool and to collect
data about the characteristics of post evaluation professional learning (Gliner, Morgan, &
Leech, 2009; Creswell, 2014). The close-ended questionnaire (Gliner, Morgan, & Leech,
2009) was designed to gather information to answer the research questions. Specifically,
participants were asked about the presence of professional learning practices described in
the literature review. Each participant completed the questionnaire presented in Survey
Monkey, on their own computers, and on their own time. The questionnaire took less
than fifteen minutes to complete. The questionnaire had 13 statements pertaining to
educator effectiveness and or professional learning. Respondents choose from an ordinal
scale with responses including: strongly disagree, disagree, agree, strongly agree,
unsure, and N/A (in District B) (Gliner, Morgan, & Leech, 2009). Questions were
modeled after a questionnaire created for District B and were adapted from a professional
learning questionnaire created at the University of Missouri (Missouri, 2006). Three
teacher leaders in District B gave feedback on the readability and relevancy of the
questionnaire and revisions were made (Creswell, 2014). Results from the questionnaire
39


informed interview questions and in the interviews, I asked participants about trends in
the questionnaire data (Johnson, Onwuegbuzie, & Turner, 2007). The questionnaire is
included in Appendix A.
Interviews
Interviews capture behavior, feelings, or how people interpret the world
(Merriam, 2009, p.88) and are flexible enough to capture the complexity of experiences
that likely exists in the educator effectiveness process. The interview questions were
semi-structured (Merriam, 2009) as some questions were asked of all participants based
on questionnaire analysis, the theoretical framework and the literature review of this
study (Merriam, 2009), and some questions came organically from conversation. In the
interviews, participants were asked to explain their perspective on the problem or
matter of educator effectiveness and explain the role professional learning played in
their personal educator effectiveness framework.
The interviews were designed with participants needs in mind. Interviews took
place in December and January at a time and location convenient for the participant. The
participants had time constraints and each interview lasted less than one hour. Interviews
were recorded on three different audio devices and transcribed as soon as possible after
the interview. As I listened to the audio, multiple times for accuracy, I transcribed using
methods suggested by Merriam (2009). The interview protocols are included in Appendix
B. The data collection plan is illustrated in Table 2.2.
40


Table 2. 2. Data collection plan
Date
Questionnaire
Interview
Participant
Follow up
November 2013
December 2013
January 2014
112 Participants
Invite to follow
up meeting in
spring
Manger of Teacher Invite
Leadership to
Academy, District follow
A; Director of up
Teacher Leadership meeting
Academy, District B in
spring
Director of Teacher Development and Leadership, District A; Director of Professional Learning, District B
Director of Peer Observers, District A; Director of Educator Effectiveness, District B
Data Analysis
The data analysis plan for this study was built on a foundation of critical theory,
sociocultural theory, the four research questions, and the literature review and this
foundation was the lens through which data is analyzed (Creswell, 2014; Kawulich, 2004;
Saldana, 2008Y). To ensure more study validity, the data analysis plan was steeped the
ideas of many researchers including Creswell (2014), Gliner, Morgan, & Leech (2009),
Johnson, Onwuegbuzie, & Turner, (2007), Kawulich (2004), Miles and Huberman
(1994), Merriam (2009), Punch (2004), Saldana, (2008) and their thinking gave me a
41


structure to navigate through this topic that I am connected with both personally and
professionally.
Questionnaire Analysis
Gliner, Morgan, & Leech (2009) suggest Exploratory studies of a new topic
may just describe what people say or feel about that topic(p. 90) and therefore, in this
study, this new topic of educator effectiveness was described in part, through a
Descriptive Research Approach. A Descriptive Research Approach includes several
variables that are not compared to each other but instead are described through a variety
of methods including averages, percentages, and frequency distributions (Gliner, Morgan,
& Leech 2009). To illustrate this concept, teachers in this study reported their beliefs that
The district has a to plan support teachers once a teacher has been rated on the new
evaluation tool and In post evaluation learning, participants are involved in determining
topic and content of the session but these two items will not be compared to each other.
Data analysis began with determining the frequency with which 112 teacher
leaders reported that teacher learning is part of educator effectiveness and that evaluation
is followed by support. Once this data was collected, I created a spreadsheet that showed
the frequency and percentage (Gliner, Morgan, & Leech, 2009) with which teachers
leaders reported professional learning was planned and implemented with the following
characteristics from the literature review: participant driven, data driven, clear targets,
specific feedback on targets, modeling, and coaching. Because the ordinal data was not
likely be normally distributed, for each item, a percentage will be calculated to show the
frequency with which a teacher reports an experience strongly disagree, disagree, agree,
strongly agree, unsure, and N/A (in District B). In this study, use of practices in literature
42


review is the desired state and therefore considered the highest standard of practice.
During analysis, I kept track of questions that were raised about the data and seek
information on these questions in the interviews with district administrators.
Questionnaire
How will schools mid districts support a teachers improvement once they have been labeled by the teacher
evaluation rubric?
How is professional development planned mid implemented in the new educator effectiveness system?
Look for
teacher
learning as an
element of
professional
learning:
frequency and
report % of
each response.
i
Figure 3. Questionnaire Analysis
43


Interview Analysis
Data Reduction. After each interview, I listened to the audio several times and
created a transcript (Merriam, 2009; Kawulich, 2004). Once transcripts were created, I
will began coding which is the first stage of analysis (Merriam, 2009; Kawulich, 2004;
Punch, 2004; Creswell, 2014). My analysis plan was based on the work of Miles and
Huberman (1994) (as cited by Punch, 2004), which includes three stages, data reduction,
data display, and data conclusion. Data reduction occurs throughout the analysis process
when the researcher codes data, links data to other data, and then when the researcher
uses data to create a theory (Miles & Huberman, 1994). When I first read the word data
reduction, it had a negative connotation for me because it sounded like an attempt to
break or take away from participant responses. I then thought about the process of
reduction in cooking and how the process transforms raw material into a new, interesting
product. In my analysis, I remained open to all possible categories but based on the
theoretical framework and literature review of this study, I coded (use data reduction) the
interview transcripts with these anticipated codes: teacher learning part of educator
effectiveness(Literature Review), types of professional learning (Literature Review),
use teacher leaders in professional learning, attention to complexity of learning
(Theoretical Framework), attention to teacher social learning (Theoretical Framework)
(Creswell, 2014; Merriam 2009). I did not restrict myself to these codes and 63 codes
emerged during analysis. Once data was coded, I began the next part of data reduction,
the categorizing process.
Data Categorizing. Once data was coded I began the categorization process. I
color coded interview text and cut and pasted excerpts from each of the six interviews
44


into a table with the 63 categories. I then connected each category to my theoretical
framework and research questions. During this process, I drew a graphic display of
categories that were connected to my questions and the theoretical framework. I also
drew graphics of themes that recurred with the six participants (Kawulich, 2004; Saldana,
2004). Once categories were compiled and labeled, I began to think about relationships
between categories. Relationships between categories, become the basis of assertions
made in the later steps of analysis (Creswell, 2014; Merriam, 2009).
Data Display. Punch (2004) writes that Miles and Huberman (1994) value
creating visual representations of qualitative data throughout the analysis process and
they propose, Data displays organize, compress, and assemble information(Punch, p.
174). As I was planning the methodology for this study, my thinking became more clear
when I created a graphic display for the analysis plan (Kawulich, 2004; Saldana, 2004).
Miles and Huberman (1994) (as cited in Punch, 2004) further propose that this graphic
display of data changes over time as understanding of the data becomes more focused. To
begin my analysis, I created of a data display that has categories with arrows between
them and each link or connection is explained in the display (See Figure 2.4). During this
process, I drew a graphic display of categories that were connected to my questions and
the theoretical framework. I also drew graphics of themes that recurred with the six
participants (Kawulich, 2004; Saldana, 2004).
45


Drawing and Verifying Conclusions. The third stage of data analysis is drawing
and verifying conclusions (Miles & Huberman, 1994 as cited in Punch, 2004).
Conclusions are often drawn early in the process and then solidified and verified
throughout data analysis (Punch, 2004). Because the educator effectiveness process is in
its nascent stages of implementation, I hesitated to make conclusions and rather will
make assertions about where the process is at current state. Three categories emerged as
possible findings including: 1) Teacher learning is a desired state of the educator
effectiveness system in District A and B but is not yet implemented. 2) District A is using
teacher leaders to coach, to provide professional learning, to provide feedback, and to
evaluate teachers to increase the likelihood of teacher growth. District B has the desire to
use teacher leaders but is not yet strategically using them in such roles. 3) Systemic issues
block District A and B from using evaluation tools to yield data from teacher
observations to inform and plan professional learning. After arriving at these three big
findings, I went back through all of the data to verify that I had enough data to support
these findings (Miles and Huberman, 1994 as cited in Punch, 2004). I reorganized data by
cutting and pasting into tables that provided evidence for these initial findings. Next, I
described my initial findings in more than 50 pages of text and through this process, I
reorganized the categories again into two main topics: 1) the disconnect between the
teacher evaluation system and the teacher growth system 2) The selection, training, and
utilization of teacher leaders.
46


Data Reduction
Category
Data Display
Anticipated Codes and Categories
Open to any emerging codes/or categories
Teacher learning part of educator
effectiveness
Types of professional learning
Use of Teacher Leaders
Complexity of learning (Theoretical
Framework)
Social learning (Theoretical Framework)
Drawing adapted from
Saldana 2008
V /
Possible Outcomes Interviews
Understandings of and
Assertions about...
How professional development
planned and implemented in the
new educator effectiveness
system; How teacher learning
framed as a dimension of
educator effectiveness; How
schools and districts plan to
support a teachers
improvement once they have
been labeled by the teacher
evaluation rubric; Roles expert
teachers take in schools in the
educator effectiveness system.
Figure 2.4 Interview Analysis Data Display
47


Issues of Reliability and Validity
Issues of validity and reliability have been considered in the research design. The
most problematic issue was that sample of expert teachers was not random or
representative of all teachers or teacher leaders in the two school districts (Gliner,
Morgan, & Leech, 2009). Many expert teachers have participated in the teacher
leadership academies in years prior to the study and more will participate in future
yearstheir voices were not included in this study. Selecting teachers from the
leadership academies was the best method available for this study because when data
collection begins, teachers were not yet rated on the state educator effectiveness system.
Beyond the issues associated with sampling, other precautions have been taken to
address issues of validity and reliability. The study is framed in both critical theory and
socio-cultural theory, which identify and explain my lens for conducting research
(Kawulich, 2004; Merriam, 2009). Methods in this study employ multiple sources of
data including interviews (n=6) and a questionnaire distributed to teacher leaders
(=100) across two school districts and three type of jobs (Creswell, 2014; Merriam,
2009).
Conclusion
Rivers and Sanders (1996) found that the teacher is the most important factor, in
school, for student achievement. Hargreaves and Fullan (2012) call this finding the most
abused finding in educational research. However, it is a commonly held belief and is
driving substantial reforms in policy and practice. Quite frankly, it is difficult to disagree
that great teachers are powerful. Therefore, this proposed study is driven by the desire to
48


ensure all students have the opportunity to work with an effective teacher who helps them
gain the reading, writing, and speaking skills that are required to access power in the
world. To reach this desired state, this study seeks to understand the ways that two
school districts are implementing the state mandated Educator Effectiveness Law.
To prepare for this study, I reviewed literature of many experts on professional
learning, evaluation, and teacher leadership so I might understand critical lessons learned
about post-evaluation teacher learning in schools beyond District A and District B.
Darling-Hammond (2013) is one expert who is concerned about teacher learning in
evaluation:
Virtually everyone agrees that teacher evaluation in the United States needs an
over haul. Existing systems rarely help teachers improve or clearly distinguish
those who are succeeding from those who are struggling. The tools that are used do
not always represent the important features of good teaching. It is nearly
impossible for principals, especially in large schools, to have sufficient time or
content expertise to evaluate all of the teachers they supervise, much less to
address the needs of some teachers for intense instructional support. And many
principals have not had access to professional development and support they need
to become expert instructional leaders and evaluators of teaching. Thus,
evaluation, in its current form often contributes little to either to teacher learning or
to accurate, timely information for personnel decisions, (p. I)
Through this study, I hope to learn how School District A and School District B
are using teacher learning as an aspect of educator effectiveness. As I capture and
49


analyze the thinking of study participants through the questionnaire and interviews, I
hope to compare their thinking to research completed outside of study so that new
knowledge is gained about effectively implementing an educator effectiveness system
that considers teacher learning as one aspect of the process. Ultimately, by discerning the
current practices of educator effectiveness in these two districts, this study hopes to
provide clear data and assertions that help all educators make the best possible decisions
about educator effectiveness and positively impact the educational experiences for all
students.
50


CHAPTER IV
FINDINGS AND IMPLICAITONS
Findings
The Educator Effectiveness Law intended to increase student achievement by
ensuring that every classroom has an effective teacher. To ensure that an effective
teacher is in every Colorado classroom, CDE outlined guidelines to implement a new
teacher evaluation system followed by post evaluation support (CDE, Fact Sheet, 2013).
District leaders in this study, with valuable knowledge of adult learning theory and
practice, grappled with the connection between the new evaluation system and what is
known about teacher growth. The findings of this study aim to explain how six district
leaders and the teacher leaders they support have attempted to navigate through the new
state mandate and work toward the complex goal of coupling a teacher evaluation
system and a teacher growth system.
The primary finding of this study is that in both districts, post evaluation learning
did not exist in the implementation of the new evaluation system. In other words, teachers
were not necessarily learning as a result of evaluation. District leaders in this study had a
deep foundation of knowledge and experience with sociocultural theory and adult
learning principles. With this foundation, district leaders knew that to ensure an effective
educator in all classrooms, their systems needed to move beyond teacher evaluation. My
analysis found that a gap between teacher accountability and teacher growth existed in
these districts for two reasons: A) The lack of communication between evaluators and
professional developers and B) The capacity of an evaluator to provide post evaluation
professional learning.
51


The second major finding was that leaders in District A relied on teacher leaders
(TLs) as a major strategy to increase the effectiveness of all teachers and District B
aspired to do the same. However, district leaders concluded the teacher leader program
could be more intentionally implemented. This reliance on TLs matched the sociocultural
beliefs held by participants who considered peer learning an effective method to grow the
skills of teachers. Examination of these findings elucidated the struggle to harmoniously
combine teacher accountability and teacher growth into a system that ultimately results in
more students having access to quality education.
Less than half of TLs in this study indicated that their school/district had an
articulated plan to support a teachers post evaluation learning and interestingly, each
district leader countered that TLs gave districts too much credit. In District A, 47.06% of
teacher leaders surveyed reported that their school/di strict had an articulated plan for post
evaluation support. In District B, 33.34% of TLs surveyed reported that their school had a
plan for post evaluation support. TLs responses did not indicate a high level of
agreement with the existence of post evaluation support, yet all six-district leaders
believed TLs rated the district too high on this question. If the goal of educator
effectiveness in these districts was to use a new evaluation tool, the task was complete,
but if the goal of educator effectiveness was increasing a teachers effectiveness with
students, the task was ongoing. District leaders knew what the qualities of professional
learning associated with evaluation looked like and knew their teachers were not getting
it. The following describe these findings in detail by first focusing on the depth of
knowledge of district leaders, then on the lack of connection between teacher evaluation
52


and teacher growth, and finally on the use teacher leaders as strategy to increase the
effectiveness of all educators.
District Leaders Understand Adult Learning
District leaders in this study had experience and knowledge in creating more
effective educators. Each participant described quality teacher learning using similar
language and with characteristics substantiated by literature on quality adult learning
including incorporating coaching (Center for Teaching and Learning, 2013; Kraft and
Balzer, 2012), utilizing peer modeling (City, 2011; Darling-Hammond & McLaughlin,
1995; NBPTS, 2011), using data (Cross, 2012; Darling-Hammond & McLaughlin,
1995;Smith, Johnson, &Thompson, 2012), giving choice (Allen, 2007; Cross, 2012
Darling-Hammond & McLaughlin, 1995), and giving clear feedback (Cross, 2012;
Learning Forward, 2013; MET, 2013). Additionally, each participants spoke with
language connected to ideas of the sociocultural theories that characterize learning as
social, ongoing, and complex (Bransford et al, 2000; Tharpe, Estrada, Dalton, &
Yamauchi, 2000; Warschauer, 1997). Each of these district leaders understood that
having an effective educator in every classroom was a complex task and framed post
evaluation teacher learning as a critical aspect of the educator effectiveness system.
These district leaders recognized there was no quick fix for educator effectiveness
and recognized that teachers improve their practice through collaborative, social learning.
Ellen1, the Director of the Teacher Leadership Academy in District B, posited We tend
to think the presence of a document, i.e. teacher evaluation tool, will someway magically
transform what has been apparent... on the part of the teacher practitioner or on the part
1 All names in the study are pseudonyms
53


of an administrator.. .nothing could be more incorrect. District leaders in this study
believed teacher learning and collaboration connected to student needs was paramount
for a growth system. Table 1 illustrates how district leaders believed a teachers
effectiveness is increased and shows the connection between their statements and
literature on adult learning, sociocultural theory, and critical theory.
54


Table 1
Sample language from district leader interviews that indicate connection to literature on adult learning and theoretical
frameworks of this study
District Leader Excerpt from interview Connection literature on Connection
adult learning theoretical
frameworks
Pruitt, Director of Peer Observers, District A I think small actionable feedback, so frequent and actionable feedback that comes from someone they deem as reliable Quality feedback (Learning Forward, 2013; MET, 2013; Cross, 2012). Sociocultural theory; peer collaboration and feedback (Bransford et al, 2000; Tharpe, Estrada, Dalton, & Yamauchi, 2000; Warschauer, 1997)
John, Manager of Teacher Leadership Academy, District A [Offerings] that are more community oriented and engaging with a body of peers with opportunities for consistent reflection and reapplication and adjustment over time Participant driven professional learning communities (Darling- Hammond & McLaughlin, 1995; Cross, 2012; Allen, 2007) Sociocultural theory; peer collaboration and feedback followed by time for self-reflection to make meaning (Bransford et al, 2000; Tharpe, Estrada, Dalton, &. Yamauchi, 2000; Warschauer, 1997)
Terri, Director of Teacher Development and Leadership, District A How is it that we can develop a system that gives teachers feedback about where they have strengths, where they have growth areas, and then providing opportunities for them to continue to grow? Clear feedback on targets (Learning Forward, 2013; MET, 2013; Cross, 2012) Sociocultural theory; social construction of knowledge (Borko, 2004, Darling- Hammond, 2013)
Lorie, Director of Educator Effectiveness, District B Having that ongoing support to implement whatever the learning is. Ongoing support with peer coaching; Cornett & Knight (2008) Socio-cultural framework (Bransford et al, 2000; Tharpe, Estrada, Dalton, & Yamauchi, 2000; Warschauer, 1997)
Ellen, Director of Teacher Leadership Academy, District B What data are you using to inform not only your initial investment in this work, but your monitoring and your outcomes? Data-driven professional learning (Smith, Johnson, &Thompson, 2012; Darling- Hammond & McLaughlin, 1995; Cross, 2012) Critical theory; using data to in valid ways drive action (Brookfield, 2005)
Jane, Director of Professional Learning, District B A professional learning community looking at student work, for example, when theres more people participating the threat, the feelings of fear arent as prevalent, because everybody is in it together Professional learning community (Hewitt & Weckstein, 2012) Sociocultural theory; safe learning environments (Bonk,1998)
55


Teacher Learning not Connected to Evaluation at the District Level
All six-district leaders reported that no specific plan for teacher learning
connected to the evaluation tool existed. Specifically, there was no clear connection
between data yielded from the evaluation tools and post evaluation professional learning
offered to teachers. Without an explicit plan for teacher learning tied to the tools,
participants concluded the new evaluation system was not different from the old system.
Lorie, the Director of Educator Effectiveness in District B, quipped Frankly [this is] a
practice that weve done in District B, its really not that different. The districts already
had an evaluation tool and these leaders believed teacher learning existed beyond the use
of an evaluation tool.
Lack of communication between evaluators and professional developers. In
both districts professional developers and evaluators did not have an effective
communication system and most often evaluation data and professional learning
knowledge existed in separate silos. District leaders reported that principals, the
evaluators, were not always aware of professional learning opportunities available to
support teacher learning on a specific indicator. Pruitt, The Director of Peer Observers
District A, illustrates this communication gap:
[I dont know] if theres been any sort of support for building leaders on... what
... .are you offering your teachers after their evaluation to figure out what is
productive... I think the systems are not connected.
Terri, The Director of Teacher Leadership and Development from District A,
explained that principals may not be aware of professional learning in their own building:
.. .it depends on what the school leader has in place and what they might know is even
56


available at their school. Theres a need to have more insight into how they can tap into
and understand what structures are in place for that teacher. Jane, the Director of
Professional Learning in District B, reported her district also suffered a lack of
communication between evaluators and professional developers. Jane explained I cant
guarantee with all certainty that everything at every school is being implemented with
fidelity and that it will make a difference to the students. Participants in the study
reported that a teacher could not count on communication between their evaluator and
those who planned professional learning.
Another impediment to communication, study participants reported that
professional developers had no way to create data driven professional learning connected
to teacher scores on the evaluation tools. Pruitt explained that the teachers association
closely protected evaluation data, and consequently professional learning was planed
without a link to educator effectiveness. One of Pruitts goals was to move educator
effectiveness from an accountability system to a growth system; Her objective was to
create a system where Teacher Effectiveness Coaches (TECs) use evaluation data
collected by Peer Observers (POs) to focus teacher coaching cycles. This objective did
not come to fruition because evaluation data could not be utilized by anyone but
evaluators. In a more favorable example, participants explained that schools in District A
reported a collective score on teacher evaluations prompting each school to choose two
focus areas for teacher learning. This is a promising first step towards a greater
connection between the evaluation system and teacher learning.
Evaluator capacity to give quality feedback. District leaders understood that
feedback on the indicators on the evaluation tools would help teachers increase their
57


effectiveness. Even more, district leaders thought the indicators might serve as a common
language of instruction for the district. Unfortunately, they reported that many
administrators lacked the time, the pedagogical experience, and practice to give good
feedback. This is consistent with literature on instructional leadership skills of principals
(Fink, 2012; Hallinger, 2003; Horng, Klasik, and Loeb, 2009). Terri, District A, and
Lorie, District B, explained that evaluators might have the ability to give feedback, but
they did not have the time to follow up in a timely manner with the multitude of teachers
on their caseload. Ellen, District B, explained that many teachers had difficulty
understanding evaluator feedback under the old tool and that the new, more complex tool
may make it even more difficult to receive actionable feedback. Lorie, District B, posited
that the whole new system hinged on an evaluators ability to give actionable feedback.
She recounted multiple stories of reviewing evaluations with principals and I can tell
you.. .almost with 99% [accuracy] youd read their evaluations and they were glowing.
Lorie further explained that although the new state tool was specifically designed to
elevate feedback, improvement would not occur unless evaluators improved their skill at
giving the type of feedback that guides a teacher towards increased effectiveness. In this
study district leaders surmised that a principals capacity to give feedback to teachers was
limited, and leaning on teacher leaders to help provide feedback was a promising practice
in the quest to have an effective teacher in every classroom.
Teachers Lead Growth of Peers
In its original draft, the Educator Effectiveness Law called for the construction of
a teacher career lattice that would give teachers leadership opportunities while
simultaneously keeping them in the classroom; this career lattice did not come to fruition.
58


This lapse in implementation might be a critical flaw in the new educator effectiveness
system and to illustrate this point, Curtis (2013) warns that, When districts fail to use
high performing teachers to support their colleagues growth and development,
administrators and coaches remain overburdened, and teachers as well as many students
are left lacking individualized support (p. 1). In the spirit of sociocultural theory,
participants in this study believed that teacher leadership is an essential strategy for
collaborative, thoughtful, teacher learning, as well as a leverage point to successfully
implement the new educator effectiveness system. The literature on teacher leadership
supports the validity of the district leaders beliefs about these new roles (Curtis, 2013;
Leithwood & Jantzi, 2006; Moreau, Dumouchel, & Sallafranque-St-Louis, 2012; York-
Barr & Duke, 2004) and even asserts that teachers often have a higher level of
collaboration and growth when an administrator is not in the room (Hargraves & Fullan,
2012; Peck et al, 2009). TLs hold various roles in District A and District B, but just
having TLs does not increase the effectiveness of all teachers. In interviews, leaders in
both districts self reflected and reported that they could be more strategic in selecting,
training, and using TLs.
Roles of teacher leaders in District A. District A was at the forefront of use of
TLs. Their advantage was due to access to grant money, early implementation of an
educator effectiveness evaluation tool, and the sponsorship of the District Superintendent.
During the time of the study, the Superintendent expressed his confidence in the strategy
of TLs in an internal district communication that read No other highly-skilled,
knowledge-based profession has our traditional organizational structure, with one leader
responsible for coaching and supervising 30 to 40 professionals. Our profession will
59


greatly benefit from smaller teams with greater collaboration and peer learning
(Superintendent District A, 2014). TLs in District A held a variety of roles that aspired to
increase the effectiveness of all teachers. These roles included Peer Observers (POs),
Teacher Effectiveness Coaches (TECs, not related to technology), both of which were
funded and trained by the central office, and school based TLs. Such use of TLs indicated
a district commitment to sociocultural approach of peer learning and collaboration.
Peer Observers (POs) evaluated and gave feedback to teachers on their caseload
twice a year using their districts evaluation tool. This year, POs evaluated 2,382 teachers
(about 1/3 of the teaching force); these evaluations were ancillary to principal
evaluations. Peer observers entered scores into a centralized database but did not
collaborate or calibrate scores with principals. The PO observation model varied each
year. This year it included teachers new to the district, teachers who were below a certain
score cut point last year, and those seeking TL positions. PO observations added another
data point in the body of evidence utilized for a teachers evaluation.
Teacher Effectiveness Coaches (TECs), another central office TL role, also
aspired to increase the effectiveness of all teachers. Each of the 50 TECs worked in two
schools and assisted the leadership teams and the teachers in achieving the goals on the
school Unified Improvement Plan (UIP). The Mission of the TEC Team is to increase the
achievement of all students by building capacity for educators growth, teachers as
leaders, and sustainable systems. TECs work in 82 of the 153 schools in the district to
provide ongoing coaching that is proven to increase teacher effectiveness in
implementing specific skills (Cornett & Knight, 2008).
60


Of the TLs in District A who participated in the questionnaire, 41.67% reported
that their post evaluation professional learning included an observation from a peer or
expert teacher. This indicated that POs and TECs provided services to less than half of
the TLs in the district. To close this gap in services District A began to employ TLs in
buildings. TLs in these roles lead small teams of teachers at the school level and support
them in their learning and evaluation needs. John, the Manager of the Teacher Leadership
Academy, described the role as end to end support and evaluation and explained that
teacher leaders coached, observed, gave feedback, co-planned lessons, and facilitated
team planning. Interestingly, some TLs in District A served as both coach and evaluator,
and this combination is at the heart of quandary of connecting teacher learning and
teacher growth. The school based TL structure was in its nascent stages at the time of this
study, and district leaders were aware of the need to define and shape the specific role of
school based TLs in order to make better use of this resource.
Roles of teacher leaders in District B. TLs in District B did not have roles
explicitly tied to the educator effectiveness evaluation system; however, all three District
B leaders in this study believed TLs could be utilized to increase teacher effectiveness.
Jane, District B, believed that her district needed to offer more roles to teacher leaders
because expert teachers know instruction best. Ellen, District B, explained
Unfortunately I think thats where District B probably... could do a better job of
following through in that [Using TLs to promote educator effectiveness] realm.
Specifically, Ellen believed that teacher leaders might be used to run professional
learning communities (PLCs) and to model specific skills such as close readings and
math critiques. Lorie, District B, touted the effectiveness of the First Year Teacher
61


Mentor Program that supported new teachers and also offered leadership positions to
expert teachers. Lorie believed more money would be required to increase the probability
of using TLs in the role of instructional coach. Going forward, District B might utilize
their 69 Teacher Leaders in specific roles to increase post evaluation support for teachers
and might consider some TL strategies being implemented in District A. Table 2 provides
a summary of teacher leader roles, selection, and training in both districts.
62


Table 2
Roles of Teacher Leaders
Role
Description
Selection
Training
Teacher
Effectiveness Coach
(TEC), District A
52 TECs in District
Coach works in two schools,
make agreements with principals
meet UIP; Facilitates PLCs;
planned and facilitated
professional learning; coaches
teachers in non-evaluative setting;
builds capacity in building teacher
leaders; collaborated with district
level instructional superintendents
who supervise principals; Does
not have access to data from
evaluation tool unless offered by
teacher
Successful teacher minimum
5 years, successful coaching
experiences, rigorous
interview process
Centrally trained in coaching,
facilitation, standards,
assessment, equity, ELL,
content, and other district
initiatives. Attend two- week
institute in summer, minimum
of four hours a week. Trained
in national coaching and
facilitation models. One on one
coaching visits from program
director
Peer Observer (PO),
District A
43 POs in District
Teachers on special assignment;
Go into classroom watch an entire
lesson, record evidence of teacher
and student behaviors at that time,
align the evidence to new
evaluation tool, assign scores
based on where that evidence
aligns; conduct a 30 minute
feedback conversation after both
evaluations
School based teacher Part time teacher and part time
leaders, District A teacher leader; role varies widely
to meet school need; might coach
1,200 in District teachers, lead PLCs, plan with
teachers, facilitate PD, some in
pilot schools evaluated teachers
on certain parts of new tool;
sometimes school leaders utilize
them, sometimes they do not
Hired based on content areas
and sometimes based on their
own score on the new
evaluation tool (this is
available for some applicants
and not others), rigorous
interview process
No consistent selection
process, some voted in and
some chosen by principals;
some have to show a
minimum score on
evaluation tool
First Year Teacher
Mentor, District B
7 in District
Teachers on special assignment;
Assigned a caseload of 15-25 first
year teachers, helped new to
profession teachers with planning,
assessment, instruction, classroom
management, and any other needs;
non-evaluative position.
Considered most master
teacher by principal; rigorous
interview process
School based TLs
District B
No exact number; 69
completed the TL
academy
Role varied widely to meet school
need; might coach teachers, lead
PLCs, plan with teachers,
facilitate PD; sometimes school
leaders utilized them, sometimes
they do not
No consistent selection
process, some voted in and
some chosen by principals
Centrally trained, 100 hours on
observation and feedback at
beginning of each school year,
and additional throughout the
year. One on one coaching
visits from program director
Training varied; two day
summer institute on standards
and leadership skills; attend
once monthly cohort facilitated
by Teacher Effectiveness
Coaches on standards and
leadership; some attend
additional training on scoring
on evaluation tool
Centrally trained in coaching,
facilitation, standards,
assessment, equity, ELL,
content, and other district
initiatives a minimum of four
hours a week. Trained in
national coaching models. One
on one coaching visits from
program director
Training varied; attend 100
hours of training in teacher
leadership academy over two
school years; topics included
data literacy, equity, coaching,
facilitation, assessment
63


Recruiting teacher leaders. School based TLs are a valuable resource (Barth
1990; Fink, 2013;Hallinger, 2003;Lambert, 1998) and yet there is not a strategic system
for selecting teacher leaders in either district. Ellen, District B, recruited participants by
speaking at principal and leadership meetings and through word of mouth. John, District
A, pointed out The identification and selection, right now [of TLs], its all over the
place. Some schools vote people in, some schools, its only the principal who is selecting
people, some schools have this very thoughtful process. It is possible that because
school based TLs were not strategically selected, they were also not strategically used
because principals did not learn about TLs through an interview process. John hoped the
selection process could be differentiated based on the exact role of the teacher leader.
Leaders recruiting school based TLs could learn from the PO and TEC selection
processes. For example, POs are selected through a rigorous interview process in which
they must provide evidence of content area expertise and have earned high scores on
personal evaluations. TECs, also managed by the central office, must provide evidence of
experience in successfully coaching teachers and leading professional learning
communities in their interviews. If a district invests in teacher leaders, they must also
invest in a careful selection process. Without a quality TL selection and recruitment
process, TLs may not add value to the educator effectiveness system.
Ensure strategic training of teacher leaders. Like principals and teachers, TLs
require training and support to be effective. The roles of POs, TECs, and First Year
Teacher Mentors had been established for more than three years, and these TLs
participated in ongoing, quality professional learning provided by the central office. POs
had about 100 hours of professional learning on observation and feedback at the
64


beginning of each school year and continued to practice their skill in professional
learning sessions throughout the school year. Both First Year Teacher Mentors and TECs
participated in a minimum of four hours per week (about 200 hours per school year) of
professional learning around standards, coaching, facilitation, planning, and assessment.
The number of hours of professional learning does not always equate to quality learning;
however professional learning offered to these three TL roles used practices represented
in the literature on quality adult learning. This included incorporating coaching (Center
for Teaching and Learning, 2013; Kraft & Balzer, 2012), utilizing peer modeling (City,
2011 ;Darling-Hammond & McLaughlin, 1995; NBPTS, 2011), using data (Cross,
2012;Darling-Hammond & McLaughlin, 1995; Smith, Johnson, &Thompson, 2012),
giving choice (Allen, 2007;Cross, 2012;Darling-Hammond & McLaughlin, 1995), and
giving clear feedback (Cross, 2012; Learning Forward, 2013; MET, 2013). Professional
learning for TL roles was facilitated in large and small group settings and followed by
coaching sessions facilitated by program directors. TLs in these three roles engaged in
more training than principals had around standards, assessment, planning, coaching,
giving feedback, and facilitating. Their training practices can be used as a model for TLs
in all schools.
School based TLs, in both districts, received fewer hours of training, but the sum
of the description of their learning experience was congruent with practices of quality
adult learning. TLs in District A were trained in small cohort groups that meet two full
days before school started and once a month there after, about 32 hours a school year.
TECs facilitated each of the cohorts of approximately 10 teacher leaders. TLs in District
B received about 60 hours of training per school year around standards, data literacy,
65


equity, and coaching. Ellen and other content experts led the work of these cohorts. None
of the TL training was explicitly connected to data yielded from the educator
effectiveness evaluation rubric. TLs in both districts participated in some group
professional learning but did not receive feedback on their specific performance in their
roles. Understanding that principals do not always have the capacity to provide learning
support to teachers, it is likely they lack capacity to support TLs. Therefore, a systems for
training needs to be thoughtfully considered and implemented to ensure TLs are equipped
to provide the best learning opportunities for teachers.
Essentially, this research suggests that because each of these six district leaders
had deep knowledge of how to grow effective teachers, it was difficult to implement the
educator effectiveness system because they felt it emphasized teacher accountability
rather than teacher growth. The paramount challenge faced by district leaders in the study
was to actualize a connection between teacher evaluation and teacher growth. A
connection they knew was necessary to fully implement the Educator Effectiveness Law.
Implications
Drawing from the results of this study it appears that to realize the ideal of an
effective educator in every classroom, school districts might employ two strategies: 1)
Thoughtfully connect the often disjointed systems of teacher accountability and teacher
growth; 2) Carefully select, train, and utilize expert teacher leaders as an additional
support for all educators. Contrary to the spirit of this law, the truth is, in two large
Colorado school districts, professional learning is not planned or implemented as a part of
the Educator Effectiveness system.
66


A first step towards connecting accountability and growth is underway in District
A where the new evaluation tool is used as a common language of instruction. All district
leaders in this study believed the teacher performance indicators on their evaluation tools
were valid and worthy of pursuing, and the indicators on the rubrics were used as the
common language of good instruction in the districts. District A hoped this common
language, spelled out in the indicators of the evaluation tool, becomes the foundation of
all school and district professional learning opportunities including: professional learning
community (PLCs) work, coaching sessions, feedback conversations, and in-service days.
Terri, District A, articulated her belief in using the tool as a common language by
making sure that our teachers have an understanding and a common definition of what it
means to be effective. Ideally, district leaders would not accept grants, programs, or
curriculum unless it matched to the common language of the district. Additionally the
interview processes for all district educators might be based on evidence of understanding
and use of notions held in the indicators, which is the practice in some teacher leader
selection in District A. Using the evaluation tool as a common language of teaching and
learning could be a strategy to move educator effectiveness beyond accountability and all
six participants showed some inclination towards this practice.
To further connect these systems of accountability and growth, schools might use
the data yielded from the new evaluation tool to plan professional learning and actualize
this common language of instruction. Professional developers should always use data to
plan formal course offerings tied to specific indicators on the evaluation tool (Cross,
2012; Learning Forward, 2013; MET, 2013). If this were the case, evaluators could
strategically offer learning choices to teachers connected to indicators on the rubric. Jane,
67


District B, explained her desire that the rubric will drive professional learning that we
will need to have in our district, at least thats my hope. Additionally, professional
developers could use change in evaluation scores as one data point to track the
effectiveness of professional learning offerings. One final benefit of such a system, when
school and district leaders use data to inform decisions, adjust professional development,
and individualize learning for teachers, they are modeling good practice for teachers and
their work with students.
Using data from the evaluation tool to plan for teacher growth is complex.
Because these systems have long been separated, evaluation has never really been viewed
as a learning experience and actually, at times, has been seen as a threat. Sociocultural
theory tells us that it is difficult for learning to take place under such conditions. This
history of mistrust between evaluators and teachers has likely led to the prohibition of
sharing evaluation data. Even if data sharing is for the benefit of planning professional
learning, it is still forbidden by district policy and by employment law. A current example
of this discord between evaluation and growth is illustrated this school year in which the
new evaluation system coincides with new standards and assessment implementation; this
situation requires a teacher to experiment and innovate to improve their craft while
simultaneously grappling with taking risks in a higher stakes evaluation system. Quality
professionals demand accountability in their work and teachers are no exception, but the
accountability offered through this new evaluation system has higher stakes than in the
past. This culture may make teachers even less inclined to share their evaluation data.
Any plan to use of evaluation data for professional learning will have to be thoughtfully
constructed by multiple stakeholders. Both teacher accountability and teacher growth will
68


be enhanced by strategic use evaluation data used for planning individualized
professional learning.
The second strategy to connect the accountability system to a growth system is to
employ and train expert teacher leaders who can help principals provide feedback and
provide quality professional learning. Much research asserts that principals do not have
the time and in some cases, the expertise to provide quality professional learning to
increase the effectiveness of educators (Barth 1990; Fink, 2013; Hallinger, 2003;
Lambert, 1998; York-Barr, J. & Duke, K., 2004 ). An uncritical approach to educator
effectiveness would accept that a principal could evaluate a teacher one time, hold a
feedback meeting, and expect educator growth. In accordance with the ideas of
sociocultural learning theories, TLs, who are experts in their craft and content, could be
used to model and train other teachers. This is promising practice is not yet fully
actualized in either districts because TLs have been named but not empowered with
specific roles and quality training.
Simply naming TLs would be an uncritical approach to educator effectiveness.
More suitably, TLs must be carefully chosen, trained, and utilized in schools; However
district leaders within this study wrestled with these complexities. District leaders
admitted that the teacher leader selection process was less than strategic; for example,
some TLs were nominated by principals, some were voted in, and some self selected into
the position. Not all teachers have the skill to lead another in instruction, and therefore
teacher leaders must be carefully screened, interviewed, and selected based on multiple
data points including articulation of sociocultural learning beliefs, personal scores on the
district evaluation tool, coaching disposition, and the ability to create strong relationships
69


with adults in the school. To improve the selection process, teacher leaders might be
chosen based on school needs; for example, one school may need a math expert or one
school may need an expert in giving feedback. Schools would interview with these
specific roles in mind. Once selected, TLs would be a valuable resource for teachers, but
only if they have ongoing training in coaching, giving feedback, data literacy, planning,
assessment, and instruction. In District A, Teacher Effectiveness Coaches provided much
of the ongoing training for school based TLs.
Finally teacher leaders need to be utilized effectively in their schools, and this is
perhaps one of the biggest challenges faced by district and teacher leaders. Like any
other valuable resource, principals need training on how to effectively use teacher leaders
and teacher leaders need to know how to make their skills fit the needs of the school. In
District A, some TLs are being used as both evaluators and coaches and this dichotomy is
at the heart of the complexity of combining teacher accountability with teacher growth.
Keeping this complexity in mind, the work Glickman (2002) would suggest proceeding
with caution if using a TL as both a coach and an evaluator. Data in this study suggests
that to make teacher leadership effective, principals need to give up some of the absolute
control of their buildings and be willing to distribute the leadership to expert teachers.
Perhaps, if TLs are better selected and trained, principals may be more willing to share
leadership and this second strategy of teacher leadership could be realized.
Connecting the accountability system and the growth system is complex but a
worthy goal to pursue; A goal that will require thoughtful planning, a focus on adult
learning for all educators, and the intentional and consistent use of the common language
offered by the evaluation tools. Most critically, the integration of evaluation and growth
70


requires a commitment to building strong relationships of trust between teachers and
evaluators. It also requires time for teachers and evaluators to co-create a belief system
about this new purpose and outcomes of evaluationthese critical steps cannot be
circumvented or rushed. Sociocultural theory helps educators understand that the State
and subsequently a district, cannot honestly call an evaluation tool a growth tool unless
teachers, the learners in this case, agree it is so. As the Educator Effectiveness system
endures, further research is required to determine if a connected system of teacher
accountability and teacher growth causes more effective educators or if educators are
better served if the systems remain unconnected. Ultimately, students would benefit from
a thoughtful district theory of the integration of accountability and growth as they
experience both firsthand each day in the classroom. Perhaps if the adults in a school
system can figure out how to fruitfully couple these two complex concepts, educators can
better design a students classroom experience and ensure that accountability through
assessment actually causes student growth.
71


REFERENCES
Allen, D. (2006). The push to excellence: teachers focus on professional learning to lift
student achievement. Journal of Staff Development, 27(1), 56-60.
Auerbach, E. R. (1993). Reexamining English only in the ESL classroom. TESOL
Quarterly, 27, 9-32. doi: 10.2307/3586949
Barth, R. (1990) Improving schools from within. San Francisco, CA: Jossey-Bass.
Berg, J. EL, Miller, L. R., & Souvanna, P. (2011). Boston shifts learning into high gear.
Certificate program accelerates student learning by building teacher capacity.
Journal of Staff Development, 32(3), 33-36.
Borko, H. (2004). Professional development and teacher learning: Mapping the terrain.
Educational Researcher, 22(8), 3-15.
Bransford, J., Brown, A., & Cocking, R. (2000). How people learn: Brain, mind,
experience, and school (Expanded ed.). Washington, DC: National Academy
Press.
Brookfield, S. (2005). The power of critical theory: Liberating adidt learning and
teaching. San Francisco, CA: Jossey-Bass.
Center for Teaching and Learning (2013). Teaching resources. University of Minnesota.
Retrieved from http://wwwl.umn.edu/ohr/teachleam/resources/peer/guidelines/
Chetty, R., Friedman, J. N., & Rockoff, J. E. (December, 2011). The long-term impact of
teacher: Teacher value-added and student outcomes in adulthood (Working Paper
17699). Cambridge, MA: National Bureau of Economic Research. Retrieved from
http://www.nber.org/papers/wl7699.
72


City, E. (2011). Learning from instructional rounds. Educational Leadership, 69(2),
36-41.
Clemans, A., Berry, A., & Loughran, J. (2012). Public anticipation yet private realization:
The effects of using cases as an approach to developing teacher leaders.
Australian Journal of Education, 56(3), 287-302.
College Board. (2013). College board. Retrieved from www.collegeboard.org
Colorado Department of Education. (May 2013). Educator effectiveness: A Colorado
priority (CDE Fact Sheet). Retrieved from
http://www.cde.state.co.us/sites/default/files/documents/educatoreffectiveness/do
wnloads/factsheets/statemodel .pdf
Cornett, J., & Knight J. (2009). Coaching approaches and perspectives. Thousand Oaks,
CA: Corwin Press.
Creswell, J. W. (2003). Research design: Qualitative, quantitative, and mixed method
approaches (2nd ed.). Thousand Oaks, CA: Sage Publications.
Cross, J. (2012). Lasting legacy for achievement: Colorado coaches boost teacher
effectiveness with a rubric for professional learning. Journal of Staff
Development, 33(5), 36-44.
Curtis, R. (2013). Finding a new way: Leveraging teacher leadership to meet
unprecedented demands. Retrieved from
http://www.aspeninstitute.org/publications/fmding-new-way-leveraging-teacher-
leadership-meet-unprecedented-demands#sthash.r3eRYde3.dpuf
Darling-Hammond, L. (2013). Getting teacher evaluation right: What really matters for
effectiveness and improvement. New York, NY: Teachers College Press.
73


Darling-Hammond, L., & McLaughlin, M. W. (1995). Policies that support professional
development in an era of reform. Phi Delta Kappan, 76(8), 597-604.
DCPS. (2011, October 24.). Washington DC Public Schools. Survey: DCPS stakeholder
sentiment improving in several categories. Retrieved from http://dc.gov/DCPS/
Denning, S. (2011, May). The single best idea for reforming K-12 education. Forbes
Magazine. Retrieved from www.forbes.eom/sites/stevedenning/2011/09/01/the-
single-best-idea-for- reforming-k-12-education/
Engdahl, T. (2010, March). Teacher effectiveness council starts 18-month run. Education
News Colorado. Retrieved from
http://www.ednewscolorado.org/2010/03/ll/3679-teacher- effectiveness-council-
starts-18-month-run
Fertig, B. (2013, July 23). Annual school survey shows parents are more satisfied than
teachers. School Book. Retrieved from http://www.wnyc.org/story/302090-
annual-schools-survey-shows-parents-more-satisfied-than-teachers/
Fink, S. (October, 2012/ School and district leaders as instructional experts: What we
are learning. Center for Educational Leadership, University of Washington.
Retrieved from
http://depts.washington.edu/uwcel/5d/level_of_expertise_article_revised_oct2012
.pdf
Goe, L., Bell, C., & Little, O. (2008). Approaches to evaluating teacher effectiveness: A
research synthesis. Washington, DC: National Comprehensive Center for Teacher
Quality.
74


Goldin-Dubois. (1999). Increasing participation in American politics (Masters thesis,
University of Colorado at Denver).
Hallinger, P. (2003). Leading educational change: Reflections on the practice of
instructional and transformational leadership. Cambridge Journal of Education,
53(3), 329-351.
Hanushek, E. A., & Lindseth, A. A. (2009). Schoolhouses, courthouses, andstatehouses:
Solving the funding-achievement puzzle in America's public schools. Princeton,
NJ: Princeton University Press.
Hargreaves, A., & Fullan, M. (2012). Professional capital: Transforming teaching in
every school. New York: Teachers College Press.
Hargreaves, A., & Shirley, D. (2009). The persistence of presentism. Teachers College
Record, 111(11), 2505-2534.
Hartley, D. (2009). Education policy, distributed leadership and socio-cultural theory.
Educational Review, 61(2), 139-150.
Heifetz, R. A., & Laurie, D. L. (1999, January-February). The work of leadership.
Harvard Business Review, 75(1), 124-134.
Hewitt, K. K., & Weckstein, D. K. (2012). Programs in practice: Differentiated
instruction: Begin with teachers! Kappa Delta Pi Record, 48(1), 35-46.
Horng, E., Klasik, D., & Loeb, S. (2010). Principal's time use and school effectiveness.
American Journal of Education, 116(4), 491-523.
Jefferson, T. (1789, January 8). [Letter to Richard Price], Copy at Monticello.
75


Johnson, L. (1964, January 8). President Lyndon B. Johnson's annual message to the
Congress on the State of the Union. Retrieved from
http://www.lbjlib.utexas.edu/johnson/archives.hom/speeches.hom/640108.asp
Johnson, R., Onwuegbuzie, A. J., & Turner, L. (2007). Toward a definition of mixed
methods research. Journal of Mixed Methods Research, 7(2), 112-133.
Johnson, S. (n.d). Why teachers must have an effective evaluation system. American
Federation of Teachers. Retrieved from
www. danielsongroup. org/. ../Why%20Teachers%20Must%20Have%20an...
Johnston, M. (2013). SB 10-191: Great teachers & leaders. Retrieved from
http ://www.mikej ohnston.org/issues/ sb-10-191/
Kawulich, B. (2004). Data analysis techniques in qualitative research. Retrieved at
www.eeraonline.org/journal/files/2004/JRE_2004_08_Kawulich. pdf
Knight, J. (2007). Instructional coaching: A partnership approach to improving
instruction. Thousand Oaks, CA: NSDC.
Kozulin, A., Gindis, B., Ageyev, V., & Miller, S. (2003). Vygotskys educational theory
in cultural context. Cambridge, United Kingdom: Press Syndicate of the
University of Cambridge.
Kraft, M., & Blazar, D. (2013, Spring). Improving teacher practice: Experimental evidence
on individualized teacher coaching. Presented at the 2013 SREE Spring
Conference.
Kyriakides, L., Creemers, B., & Antoniou, P. (2009). Teacher behaviour and student
outcomes: Suggestions for research on teacher training and professional
development teaching. Teacher Education, 25(1), 12-23.
76


Labato v. State of Colorado. (2013, April 16). Colorado Department of Law. John W.
Suthers, Attorney General. Case can be retrieved at
http://www.coloradoattorneygeneral.gov/departments/state_services/education/lob
ato
Ladson-Billings, G. (2009). The dreamkeepers. San Francisco, CA: Jossey-Bass.
Ladson-Billings, G. (1995). Toward a theory of culturally relevant pedagogy. American
Educational Research Journal, 32(3), 465-491.
Landmark Cases. (2013). Key excerpts from the majority opinion, Brown I (1954).
Retrieved from
http ://www. streetlaw.org/en/Page/ 519/Key_Excerpts_from_the_Maj ority_
Lambert, L. (1998). How to build leadership capacity. Education Week, 55(7), 17-19.
Learning Forward. (2013). Standards for professional learning. Retrieved from
http://leamingforward.Org/standards-for-professional-lVlearning#.UaId24KhBdQ
Learning Forward. (2013). Definitions for professional learning. Retrieved from
http://leamingforward.Org/standards-for-professional-llearning#.UaId24KhBdQ
Leithwood, K., & Jantzi, D. (2006). Transformational school leadership for large-scale
reform: Effects on students, teachers, and their classroom practices. School
Effectiveness and School Improvement, 17(2), 201-227.
Leclerc, M., Moreau, A., Dumouchel, C., & Sallafranque-St. Louis, F. (2012, December
10). Factors that promote progression in schools functioning as professional
learning community. International Journal of Education Policy & Leadership,
7(1), 1-14.
77


Lloyd, K., Tienda, M., & Zajacova, A. (2001). Trends in educational achievement of
minority student since Brown us. Board of Education. Retrieved at
theop.princeton.edu/reports/misc/trends_in_ed.pdf
Markley, T. (2004). Defining the effective teacher: Current arguments in education.
Retrieved from www.usca.edu/essays/volll2004/markey.pd
Marshall, K. (2005, June). Time to rethink teacher evaluation. Phi Delta Kappan, 56(10),
730.
Marzano, R, J., & Frontier, T, (2011), Effective supervision supporting the art and
science of teaching. Alexandria, VA: ASCD.
Merriam, S. B. (2009). Qualitative research: A guide to design and implementation. San
Francisco: Jossey-Bass.
The MET Project. (2012). Gathering feedback for teaching, combining high-quality
observations with student surveys and achievement gains. Retrieved from
http ://www.metproj ect. org/downloads/MET_Gathering_F eedbackPractionerBri
ef.pdf
The Met Project. (2010). Initial findings from the measures of effective teaching project.
Retrieved from httpd/wwwmetproject.org/downloads/Preliminary_Finding-
Policy_Brief.pdf
Meyer, D., Madden, D., & McGrath, D. J. (2004). English language learner students in
U.S. public schools 1994 and 2000. Washington, DC: National Center for
Education Statistics, U.S. Dept, of Education, Institute of Education Sciences.
Mezirow, J. (1981, September). A critical theory of adult learning and education. Adult
Education Quarterly, 32, 3-24.
78


Missouri Department of Elementary and Secondary Education. (2006, Revised). Missouri
professional development guidelines for student success. Retrieved from
http://www.sikeston.kl2.mo.us/profdev/270484.pdf
Mongiello, P., Brady, D., Johnson, G., & Berg, J. EL, Cao, L.; Gorodetsky, V., Liu, J.,
Weiss, G., & Yu, P. S. (2009). Strength training: Institutes pump up teachers'
roles as instructional leaders. Journal of Staff Development, 30(4), 20-24.
Munoz, M., Prather, J., & Stronge, J. (2011). Exploring teacher effectiveness using
hierarchical linear models: Student- and classroom-level predictors and cross-year
stability in elementary school reading. Planning and Changing, 42(3/4), 241-273.
National Research Council and National Academy of Education. (2010). Getting value
out of value-added: Report of a workshop. Washington, DC: Center for
Education, Division of Behavioral and Social Sciences and Education.
National Board for Professional Teaching Standards. (2011). Getting it right: A
comprehensive guide to developing and sustaining teacher evaluation and
support system. Arlington, VA: National Board for Professional Teaching
Standards.
National Center for Educational Statistics. (2013). [Website for National Center for
Educational Statistics], Retrieved from nces.ed.gov/
National Commission on Excellence in Education. (1983). Nation at risk (Report to the
Nation and Secretary of Education United States Department of Education).
Author.
79


National Governors Association Center for Best Practices & Council of Chief State
School Officers. (2010). Common Core State Standards. Washington, DC:
Authors.
The New Teacher Project. (2010, October 5). Teacher evaluation 2.0. Retrieved from
http://tntp.Org/ideas-and-innovations/view/teacher-evaluation-2.0
The New Teacher Project [TNTP], (2012, July 30). The irreplaceables, understanding
the real retention crisis in Americas urban schools. Retrieved from
http://tntp.org/ideas-and-innovations/view/the-irreplaceables-understanding-the-
real-retention-crisis
Owens, C. (2008). Leading without leaving the classroom. National Staff Development
Council, 29(3), 57-60.
PBS. (2013). Frontline: The testing industrys big four. Retrieved from
http://www.pbs.org/wgbh/pages/firontline/shows/schools/testing/companies.html
Peck, C. A., Gallucci, C., Sloan, T., & Lippincott, A. (2009). Organizational learning and
program renewal in teacher education: A socio-cultural theory of learning,
innovation and change. Educational Research Review, 4(1), 16-25.
Punch, K. (2009). Introduction to research methods in education. London: Sage.
Regional Education Laboratory [REL], (2007). Reviewing the evidence on how teacher
professional development affects student achievement. Educational Researcher,
53(8), 3-15.
Saldana, J. (2009). The coding manual for qualitative researchers. Thousand Oaks, CA:
Sage Publications.
80


Sanders, W., & Rivers, J. (1996). Cumulative and residual effects of teachers on future
student academic achievement (Research progress report). In University of
Tennessee Value-Added Assessment Center, Knoxville, TN. Retrieved March 28,
2001, from http://mdkl2.org/practices/ensure/tva/ tva_2.html
Slabine, M. A. (2012). 841 miles of square commitment. District wide plans makes
professional learning a priority. Journal of Staff Development 33, 22-27.
Slavin, R. E., & Cheung, A. (2005). A synthesis of research on language of reading
instruction for English Language Learners. Review of Educational Research,
75(2), 247-284.
Smith, F. (1998). The book of learning andforgetting. New York: Teachers College
Press.
Smith, R., Johnson, M., &Thompson, K. (2012). Data, our gps. Education Leadership,
69(5), 56-59.
State Council for Educator Effectiveness. (2011, April 13). Report and recommendations.
Denver, CO: State Board of Education.
Steele, J., Hamilton, L., & Stecher, B. (2010, December 1). Incorporating student
performance measures into teacher evaluation systems. Santa Monica, CA: Rand
Corporation.
Stein, S. J. (2004). The culture of education policy. New York: Teachers College Press.
Taylor, E., & Tyler J. (2011). The effect of evaluation on performance: Evidence from
longitudinal student achievement data of mid-career teachers. Cambridge, MA:
National Bureau of Economic Research.
81


Tharp, R. G., Estrada, P., Dalton, S., & Yamauchi, L. (2000). Teaching transformed:
Achieving excellence, fairness, inclusion, and harmony. Boulder, CO: Westview
Press.
Thibodeau, G. (2011, June). A content literacy collaborative study group: High school
teachers take charge of their professional learning. Journal of Adolescent & Adidt
Literacy, 52(1), 54-64. doi:10.1598/JAA.52.1.6
Thompson, L. (n.d.). Principles of data driven instruction. Retrieved from epd-
mh.com/leadership/pdfs/Principles of Data Driven Instruction.pdf
Torres, Z. (June, 2013). Disparities in Douglas County schools teacher evaluations draw
fire. Denver Post, ztorres@denverpost.com
Vygotsky, L. S. (1978). Mind in society: The development of higher psychological
processes. Cambridge, MA: Harvard University Press.
Wiliam, D. (2012). Feedback: Part of a system. Educational Leadership, 70(1), 30-34.
Warschauer, M. (1997). A sociocultural approach to literacy and its significance for In K-
12. In J. Murphy & R. Sanders (Eds.), Nexus: The convergence of research &
teaching through new information technologies (pp. 88-97). Durham, NC:
University of North Carolina.
Wong, K. K., & Rothman, R. (2009). Clio at the table: Using history to inform and
improve education policy. New York: Peter Lang.
York-Barr, J. & Duke, K. (2004). What do we know about teacher leadership? Findings
from two decades of scholarship. Review of Educational Research Fall, 74,
255-316.
82


APPENDIX A
TEACHER LEADER SURVEY
District A
The purpose of this survey is to gather information about the characteristics of professional learning that follows an
evaluation on the LEAP framework. Thank you for your participation.
1. The district/school has an articulated plan to support teachers once a teacher has been
rated on LEAP.
Strongly Disagree Disagree Agree Strongly Agree Unsure
r r r r r
2. Post-evaluation professional learning is planned as a response to student data.
Strongly Disagree Disagree Agree Strongly Agree Unsure
r r r r r
3. In post-evaluation professional learning, participants are involved in determining the
topic and content of the session.
Strongly Disagree Disagree Agree Strongly Agree Unsure
r r r c r
4. In post-evaluation professional learning, the facilitator is knowledgeable and has
credibility with the participants. Strongly Disagree Disagree Agree Strongly Agree Unsure
r r c r r
5. In post-evaluation professional learning, the program includes a variety of activities
designed for adult
learners (active engagement, use of prior knowledge, working in teams, real world
applications, choice of activities).
Strongly Disagree Disagree Agree Strongly Agree Unsure
r r r r r
6. In post-evaluation professional learning, the outcomes of the learning activity include
specific changes in
teachers classroom practices.
Strongly Disagree Disagree Agree Strongly Agree Unsure
c r c r r
7. In post-evaluation professional learning, the activity includes continued support and
follow-up activities
(frequent and ongoing sessions, expectations for implementation, not one time events).
Strongly Disagree Disagree Agree Strongly Agree Unsure
r c r c c
83


District B
The purpose of this survey is to gather information about the characteristics of professional learning that follows an
evaluation on the new state educator effectiveness rubric. Thank you for your participation.
* 1. My school has an articulated plan to support teachers once a teacher has been rated
on the new teacher effectiveness rubric.
Strongly Disagree
C
Disagree
r
Agree
C
Strongly Agree
r
Not sure
r
N/A
r
*2. At my school, post-evaluation professional learning is planned as a response to
student data.
Strongly Disagree
r
Disagree
r
Agree
r
Strongly Agree
r
Unsure
c
N/A
r
*3. At my school, in post-evaluation professional learning, participants are involved in
determining the topic and content of the session.
Strongly Disagree
r
Disagree
r
Agree
r
Strongly Agree
r
Unsure
r
N/A
r
*4. At my school, in post-evaluation professional learning, the facilitator is
knowledgeable and has credibility with the
participants.
Strongly Disagree
r
Disagree
r
Agree
r
Strongly Agree
r
Unsure
r
N/A
r
*5. At my school, in post-evaluation professional learning, the program includes a variety
of activities designed for adult learners (active engagement, use of prior knowledge,
working in teams, real world applications, choice of activities).
Strongly Disagree
r
Disagree
r
Agree
r
Strongly Agree
r
Unsure
r
N/A
r
*6. At my school, in post-evaluation professional learning, the outcomes of the learning
activity include specific changes in teachers classroom practices.
Strongly Disagree
r
Disagree
r
Agree
r
Strongly Agree
r
Unsure
r
N/A
r
*7. At my school, in post-evaluation professional learning, the activity includes continued
support and follow-up activities (frequent and ongoing sessions, expectations for
implementation, not one time events).
Strongly Disagree
C
Disagree
r
Agree
r
Strongly Agree
r
Unsure
r


APPENDIX B
INTERVEIW PROTOCOLS
Semi Structured Interview Protocol #1
Protocol for Director of Teacher Leadership Academy
You are the director of the Teacher Leadership Academy in your school and in the district. I am
going to ask you some questions about your role as a leader in your district, your beliefs about
professional development, your role in professional development, and your experience with the
new teacher evaluation system.
We will start with some general questions about your experience in schools
1. Tell me about your work as an educator
Probes:
How long have you been in this role? Why did you seek this role? Where have
you worked? What is your teaching experience?
What are your responsibilities in this role?
What are some of your beliefs about student learning? Teacher learning? Principal
learning?
Colorado and your school district are implementing a new educator effectiveness system to meet
the requirements mandated by the educator effectiveness law. For this study, I am focusing on the
teacher side of the system and the next few questions are about educators effectiveness, teacher
learning, and the new evaluation system.
2. Explain the issue of teacher effectiveness from your perspective
Probing questions:
Why is educator effectiveness an initiative in your district?
What role does the new evaluation system play in the educator effectiveness
initiative?
What actions should schools take to increase teacher effectiveness?
What do you believe makes teachers more effective for students?
How is teacher learning part of your thinking about increasing teacher
effectiveness?
How are educators in schools involved in the district decision-making process
around educator effectiveness?
What sort of professional learning can be designed to support teachers once they
have been labeled?
3. Explain the issue of teacher effectiveness from your school districts point of view
85


Probing questions:
What are the reasons your school district is implementing a new educator
effectiveness system?
What actions is your district going to take to improve educator effectiveness?
How is professional learning connected to educator effectiveness in your district?
What action steps does your district/school need to take to meet these goals?
4. In your experience, what causes teachers to increase their effectiveness for all students?
Now, I would like to move to your work in the Teacher Leadership Academy and your work with
teacher leaders.
5. Tell me about the teacher leadership academy
Probes:
What value do you place on teacher leaders?
How and why is the academy in existence?
What are the goals of the academy?
How do you recruit and retain teachers in the academy?
How do teacher leaders help realize the vision for the district?
What sort of training have you received around using teacher leaders?
How do you cultivate relationships with teacher leaders?
How do you remain connected with teachers who have left the academy?
How do you assess the value the academy has had on teachers, students, and
principals?
How do you collaborate with principals about utilization of teacher leaders?
How do you collaborate with other district leaders and departments about the utilization
of teacher leaders?
Expert teachers, who are also building leaders, may play a role in helping all teachers to become
more effective.
6. What roles and responsibilities do teacher leaders hold in schools?
Probes:
Are their roles and responsibilities you wish you could add for teacher leaders? Could
remove?
In your experience, how are teacher leaders being utilized to improve student and teacher
learning in their buildings?
I am wondering about your beliefs and practice in professional learning and how they connect to
your work with teacher leaders and the learning experiences for the academy.
7. In your experience, what are some characteristics of quality professional learning?
Probes:
What is the best professional learning you have experienced? The worst?
Does professional learning the academy embody these named characteristics?
86


What do you believe about how adults leam? What causes teachers to change
practice?
How do you connect your beliefs about PD to your work in the teacher academy?
8. How do you plan professional learning in the teacher academy?
Probes:
Who do you collaborate with to plan for the teacher academy?
What are the ways you use student data to plan for professional learning?
What are the ways you use district goals to plan for professional learning?
How do you know if professional learning is working?
How do you solicit and use feedback from teachers about professional learning?
9. Do you believe the new evaluation system will improve teacher effectiveness?
Probes:
How does the feedback on the rubric lead to teacher change?
What sort of in building and district professional learning are in place to support teachers
once they have been labeled?
What role do you envision teacher leaders having to improve teacher effectiveness in
their buildings?
10. What are the challenges of the educator effectiveness initiative? What are the strengths
of the educator effectiveness initiative?
11.1 want to talk to get your impressions about some data I collected from teacher leaders
in your district.
Conclusion:
Thank you so much for your time and your thinking today. I will share the results of my study
with you in the spring.
87


Semi Structured Interview Protocol #2
Protocol for Director of Professional Learning/Director of Teacher Development
You are the director of professional learning in the district. I am going to ask you some questions
about your role as a leader in your district, your beliefs about professional development and
teacher leaders, your role in professional development, and your experience of the new teacher
evaluation system.
We will start with some general questions about your experience as an educator
1. Tell me about your work as an educator
Probes:
How long have you been in this role? Why did you seek this role? Where have
you worked? What is your teaching experience?
What are your responsibilities in this role?
What are some of your beliefs about student learning? Teacher learning? Principal
learning?
Colorado and your school district are implementing a new educator effectiveness system to meet
the requirements mandated by the educator effectiveness law. For this study, I am focusing on the
teacher side of the system and the next few questions are about educators effectiveness, teacher
learning, and the new evaluation system.
2. Explain the issue of teacher effectiveness from your perspective
Probing questions:
Why is educator effectiveness an initiative in your district?
What role does the new evaluation system play in the educator effectiveness
initiative?
What actions should schools take to increase teacher effectiveness?
What do you believe makes teachers more effective for students?
How is teacher learning part of your thinking about increasing teacher
effectiveness?
How are educators in schools involved in the district decision-making process
around educator effectiveness?
What sort of professional learning can be designed to support teachers once they have
been labeled?
3. Explain the issue of teacher effectiveness from your school districts point of view
Probing questions:
What are the reasons your school district is implementing a new educator
effectiveness system?
What role does the new evaluation system play in the educator effectiveness
initiative?
What actions is your district going to take to improve educator effectiveness?
88


How is professional learning connected to educator effectiveness in your district?
What action steps does your district/school need to take to meet these goals?
4. In your experience, what causes teachers to increase their effectiveness for all students?
Now I would like to focus on professional learning from a district point of view and how you
connect to the work that occurs with teachers in schools.
5. Tell me about the professional learning structures in your district
Probes:
What are the goals of professional learning in the district?
How does professional learning look in a site based district?
How does professional learning help realize the vision for the district?
How do you manage enhance teacher learning from the district level?
How do you collaborate with principals about professional learning?
How do you collaborate with other district leaders and departments about the professional
learning?
How is professional learning connected to educator effectiveness?
Expert teachers, who are also building leaders, may play a role in helping all teachers to become
more effective.
6. What roles and responsibilities do teacher leaders hold in schools?
Probes:
What value do you place on teacher leaders?
How are teacher leaders being used both at the district and building levels to
increase educator effectiveness?
The State Council on Educator Effectiveness advises that teachers need ongoing feedback and
support needed to improve performance. I would like to move towards the support teachers get
once they are labeled on the rubric.
7. In your experience, what are some characteristics of quality professional learning?
Probes:
What is the best professional learning you have experienced? The worst?
Does professional learning in your district embody these named characteristics?
What do you believe about how adults leam? What causes teachers to change
practice?
How do you connect your beliefs about PD to your work in the office of professional
learning?
8. Do you believe the new evaluation system will improve teacher effectiveness?
Probes:
89


How does the feedback on the rubric lead to teacher change?
What sort of building and district professional learning is in place to support
teachers once they have been labeled?
What role do you envision teacher leaders having to improve teacher effectiveness in
their buildings?
9. What are the challenges of the educator effectiveness initiative? What are the strengths of
the educator effectiveness initiative?
10.1 want to talk to get your impressions about some data I collected from teacher leaders
in your district.
Conclusion:
Thank you so much for your time and your thinking today. I will share the results of my study
with you in the spring.
90


Semi Structured Interview Protocol #3
Protocol for Director of Educator Effectiveness/ Peer Observer
You are the director of Educator Effectiveness in the district. I am going to ask you some
questions about your role as a leader in your district, your beliefs about professional development
and teacher leaders, your role in professional development, and your experience of the new
teacher evaluation system.
We will start with some general questions about your experience as an educator
1. Tell me about your work as an educator
Possible probing questions:
How long have you been in this role? Why did you seek this role? Where have you
worked? What is your teaching experience?
What are your responsibilities in this role?
What are some of your beliefs about student learning? Teacher learning? Principal
learning?
2. Explain the issue of teacher effectiveness from your perspective
Possible probing questions:
Why is educator effectiveness an initiative in your district?
What role does the new evaluation system play in the educator effectiveness
initiative?
What actions should schools take to increase teacher effectiveness?
What do you believe makes teachers more effective for students?
How is teacher learning part of your thinking about increasing teacher
effectiveness?
How are educators in schools involved in the district decision-making process
around educator effectiveness?
What sort of professional learning can be designed to support teachers once they have
been labeled?
3. Explain the issue of teacher effectiveness from your school districts point of view
Possible probing questions:
What are the reasons your school district is implementing a new educator
effectiveness system?
What actions is your district going to take to improve educator effectiveness?
How is professional learning connected to educator effectiveness in your district?
What action steps does your district/school need to take to meet these goals?
4. In your experience, what causes teachers to increase their effectiveness for all students?
5. Tell me about the Educator Effectiveness structures in your district
Possible probing questions:
91


What are the goals of Educator Effectiveness in the district?
How does Educator Effectiveness help realize the vision for the district?
How do you manage Educator Effectiveness from the district level?
How do you collaborate with principals about Educator Effectiveness?
How do you collaborate with other district leaders and departments about the Educator
Effectiveness?
How is professional learning connected to Educator Effectiveness?
The State Council on Educator Effectiveness advises that teachers need ongoing feedback and
support needed to improve performance. I would like to move towards the support teachers get
once they are labeled on the rubric.
6. In your experience, what are some characteristics of quality professional learning?
Possible probing questions:
What is the best professional learning you have experienced? The worst?
Does professional learning in your district embody these named characteristics?
What do you believe about how adults learn? What causes teachers to change
practice?
How do you connect your beliefs about PD to your work in the office of Educator
Effectiveness?
As you think increasing teacher effectiveness, I wonder about how the district might utilize expert
teachers, who are also building leaders, to provide support post evaluation
7. What roles and responsibilities do teacher leaders hold in schools that help to increase
educator effectiveness?
Possible probing questions:
What value do you place on teacher leaders in connection to Educator Effectiveness?
How are teacher leaders being used both at the district and building levels to
increase Educator Effectiveness?
8. How do you plan for professional learning connected to Educator Effectiveness in the
district?
Possible probing questions:
How do you plan for professional learning connected to Educator Effectiveness?
What role does student data hold in Educator Effectiveness?
How do you know if the Educator Effectiveness structures are working?
How do you solicit and use feedback from teachers about Educator Effectiveness?
How do you solicit and use feedback from principals about Educator Effectiveness?
For this study, I am focusing on the teacher side of the system and the next few questions are
about how teacher evaluation affects an educators effectiveness. You may refer to the evaluation
rubric to answer these questions.
9. Do you believe the new evaluation system will improve teacher effectiveness?
92


Full Text

PAGE 1

EDUCATOR EFFECTIVENESS IN COLORADO: THE CONNECTION BETWEEN EVALUATION AND PROFESSIONAL LEARNING by COLLEEN L. O'BRIEN B.A., Colorado State University, 1991 M.S., University of Colorado Denver, 1994 A thesis submitted to the Faculty of the Graduate School of the University of Colorado Denver in partial fulfillment of the requirements for the degree of Doctor of Education Leadership for Educational Equity 2014

PAGE 2

ii 2014 COLLEEN L. O'BRIEN ALL RIGHTS RESERVED

PAGE 3

iii This thesis for the Doctor of Education degree by C olleen L. O'Brien has been approved for the Leadership for Educational Equity by Kara Viesca Chair Carolyn Haug Rodney Blunck May 1, 2014

PAGE 4

iv O'Brien, Colleen L. ( EdD, Leadership in Equity) Educator Effectiveness in Colorado: Following Evaluation with high Quality Learning Facilitated by Teacher Leaders Dissertation Directed by Assistant Professor Kara Viesca ABSTRACT Purpose: The purpose of this study was to understand how schools and districts plan professional learning that provides the ongoing feedback and support needed to improve teacher performance under the new Colorado Educator Effectiveness Law. The study focused on t he professional learning that follows a teacher's rating on the new educator effectiveness evaluation tool and on the roles teacher leaders have in the educator effectiveness system. The ultimate goal of this study was to understand and share professional learning practices implemented in the educator effectiveness system so that all students achieve at higher levels and are more engaged in classrooms. Research Design: The study includes district l eader interviews and a teacher l eader questionnaire, and t he combination of these methods identified and explain ed how schools and districts use the new educator effectiveness law as an opportunity to support a teacher's development post evaluation. This mixed methods study includes 111 teacher l eaders from teacher leadership academies in two school districts and the d istrict l eaders that support their work including the Directors of The Teacher Leadership Academy, the Directors of Professional Learning, and the Directors of Educator Effectiveness, the Manager of th e Teacher Leadership Academy, The Director of Peer Observers, and the Director of Teacher Development and Leadership. The primary finding of this study is that in both districts, post evaluation learning did not exist in the implementation of the new evalu ation system. In other words, teachers were not necessarily learning as a result

PAGE 5

v of evaluation. The second major finding was that leaders in District A relied on teacher leaders (TLs) as a major strategy to increase the effectiveness of all teachers and D istrict B aspired to do the same. However, district leaders concluded the teacher leader program could be more intentionally implemented. The form and content of this abstract are approved. I recommend its publication. Approved: Kara Viesca

PAGE 6

vi Dedicated to Robert and William

PAGE 7

vii TABLE OF CONTENTS CHAPTER I. INTRODUCTION .1 Problem of Practice1 Research Questions ..10 Significance of Study ...11 Local Context ...12 II. THEORETICAL FRAMEWORK ...13 Critical Theory.15 Sociocultural Theory....20 III. LITERATURE REVIEW ... 2 4 IV. METHODOLOGY ...35 Research Design... 35 Study Participants.35 Data Collection.39

PAGE 8

viii Data Analysis42 Conclusion 50 V. FINDINGS AND IMPLICATIONS .52 Findings.....52 Im plications. 68 REFERENCES . 73 APPENDIX..76 A. Tea cher Leader Survey.. .76 B. Inte rview Protocols. 88

PAGE 9

1 CHAPTER I INTRODUCTION Problem of Practice Education is a civil right. It is essential for one to enjoy intellectual, cultural, and financial freedom, and to attain the benefits of health, wellness, and political power. Regrettably, too many st udents in the United States lack the educational experience required to enjoy such rights and furthermore find it difficult to be eligible to study in college or to gain access to political and social power (Goldin Dubois, 1999; National Center for Educati on Statistics, 2013). Educating a human is an incredibly complex endeavor and multiple factors influence human learning. Parents, educators, policy makers, and community members cannot always ag ree on the purpose of education or define specific problems t hat need to be addressed in education. Those who think critically about education realize that focusing on one aspect of education as the root cause of the problem, i.e., poverty, health, schools, standards, racism, teachers, parents, is an oversimplificat ion of a complex process and, in reality, all of these factors influence a child's success in school (Brookfield, 2005). This study recognizes that teacher quality is one critical element of effective education (Sanders & Rivers, 1996). In the 2013 2014 school year, Colorado is embarking on another education solution intended to improve student learning and is instituting a new educator evaluation system intended to create more effective teachers. The State Council on Educator Effectiveness defines

PAGE 10

2 an eff ective educator as one who knows content, establishes a learning environment, facilitates learning, demonstrates leadership, reflects on practice, and demonstrates that students have grown academically as a result of their class (SCEE, 2011). To measure ef fective teachers in these six domains of teacher performance, the Colorado Department of Education (CDE) is crafting a quality standards rubric one aspect of educator effectiveness. Once a teacher is given a rating on the rubric, each school and district i s responsible to provide the feedback and support that improves the teacher's performance the second aspect of educator effectiveness (SCEE, 2011). Although the state is very clear on the criteria for an effective teacher, no criteria exists for the feedba ck and professional learning required to improve teacher performance, and there is no indication about who will lead teachers towards a higher level of effectiveness. A rating on a rubric alone will not improve the quality of instruction and is an oversimp lification of the process of teacher growth. Further, it will not promote the collaboration required to improve teacher learning (Darling Hammond, 2013). To realize the goal of an effective educator in every classroom, Colorado policy makers and school le aders must place more emphasis on quality professional learning structures (Darling Hammond, 2013; MET, 2012 ). Is There a Crisis in Education? Is there a crisis in education in the United States? Many Americans speak of a crisis in education, and yet other Americans are satisfied with the education system. In 1983, the National Commission on Excellence in Education explained their perspective on a crisis in our country. Our Nation is at risk. Our once unchallenged preeminence in commerce, industry, science, and technological innovation is being overtaken by competitors throughout the world. This report is concerned with only one of

PAGE 11

3 the many cause s and dimensions of the problem, but it is the one that undergirds American prosperity, security, and civility. We report to the American people that while we can take justifiable pride in what our schools and colleges have historically accomplished and co ntributed to the United States and the well being of its people, the educational foundations of our society are presently being eroded by a rising tide of mediocrity that threatens our very future as a Nation and a people. What was unimaginable a generatio n ago has begun to occur -others are matching and surpassing our educational attainments. ( National Commission on Excellence in Education, 1983, p. 3) The conclusions drawn in A Nation at Risk' are further supported by a number of educational assessments that measure student achievement of academic standards. The National Assessment of Educational Progress (NAEP), an exam that measures the educational progress of U.S. students in math, science, and reading, reveals that there has been minimal gain in stud ent progress in reading since the 1960s (National Center for Education Statistics, 2013). NAEP also reports a significant variance in the quality of education between states. For example, longitudinal data over the last forty years reveals 36% of students in Mississippi scored basic or above in math compared to 77% of students in Maine and North Dakota and 78% of students in Iowa (National Center for Education Statistics, 2013). The Third International Mathematics and Science Study (TIMSS) reports that in both math and science, U.S. students consistently decline in their performance on the TIMMS as their time in school increases, and perform below their international counterparts in both math and science in 12 th grade (National Center for Education

PAGE 12

4 Statisti cs, 2013). Although the racial achievement gap is beginning to narrow, when student assessment data is disaggregated by race, black and brown children perform at a significantly lower level than do their white classmates (Loyd, Tienda, Zajacova, 2001; Nat ional Center for Education Statistics, 2013). Adding another layer of complexity to student achievement, more than eleven million U.S. students have a primary language other than English spoken at home, yet instruction in the United States remains mostly m onolingual (Auerbach, 1993; Meyer, Madden, McGrath, 2004, National Center for Education Statistics, 2013 ; Slaven & Cheung, 2005). For those who believe that education can be accurately measured by standardized tests, these assessment results bolster the ca se of an education system in crisis. Again, is there a crisis in education in the United States? The Nation at Risk report and connected standardized tests paint a bleak picture of the education system in the United State s. P aradoxically, many Americans are still satisfied with their child's school. Recent surveys in New York report 95% of parents report overall satisfaction with the school system, (Fertig, 2012) and in Washington, D.C. more than 80% of the parents report satisfaction with their students teachers (DCPS, 2011). The wide range of beliefs brought forth by educational interest groups confounds clarity about the problems and solutions in education. Even without clarity on the current state of education, policy makers continue to proffer a myr iad of root causes for problems in education including poverty, failing schools, and most recently, and the effectiveness of individual educators. History of Root Causes: Poverty, Failing Schools, In effective Teachers In the 1960's poverty was considered the root cause of low academic performance and President Lyndon Johnson made education a part of his vision of the Great Society and

PAGE 13

5 the War on Poverty: "A third place to build the Great Society is in the classrooms of America. There your children's lives will be shaped. Our society will not be great until every young mind is set free to scan the farthest reaches of thought and imagination" (Johnson, 1964). Johnson worked with Congress to pass the Elementary and Secondary Education Act (ESEA) of 1965 as a way to improve schools by targeting poverty. Focusing on this root cause elevated services for children, but also created a culture of excuse making and low expectations for children in poverty. In 2002, the No Child Left Behind Act (NCLB) made further adjustments to Title I of the ESEA and placed the root cause of low student achievement on failing schools (Stein, 2004). The intent of this revision of the ESEA was to adjust the focus from serving poor children to serving all children by holding schools to a high level of standards and accountability. This idea of school accountability came to fruition in the 1970s when the ESEA was up for reauthorization, and lawmakers sought evidence that the mon ey spent in schools was yielding results (Stein, 2004, p. 57). NCLB, in short, held schools accountable for results rather than children. [NCLB] does not require schools to label children for services, it posits the "problem" of poor performance as an in stitutional, rather than individual, concern; it creates accountability mechanisms for student performance through average yearly progressFinally, it removes past incentives for identifying a deficient or defective population of students. (Stein, 2004, p 133) Under NCLB States have the authority to set standards and create assessments. In Colorado, school report cards are provided to the community to ensure public awareness

PAGE 14

6 of how children at individual schools performed on state assessments. At the be ginning of each school year, teachers are given the results of their student's performance on state tests the previous year, and this type of reporting was a first step in rating individual teachers. Ineffective educators are now considered a root cause of low student achievement on these standardized tests (Stein, 2004;Wong & Rothman, 2009). Focusing on this root cause elevated the discussion about gaps in student learning but also created a culture of competition and financial rewards rather than a cultur e of collaboration and student rewards (Brookfield, 2005). Nation wide, a current trend in policy making is improving the quality of teachers and principals as they are now thought to be a root cause of low performing students. Research conducted by Sander and Rivers (1996) found that, in school, the teacher is the most important factor of student achievement, and this finding compelled the State of Colorado to consider effective educators. In 2010, Colorado Governor Bill Ritter convened the State Council o n Educator Effectiveness (SCEE) to study the educators and their role in root causes of poor performance in schools. In its final report, The Council promised to provide every child with an effective teacher and an effective school leader (SCEE, 2011). One direct result of the work of the Council was the passage of Colorado's Senate Bill 10 191 that focused attention on teacher preparation, teacher ongoing development, and teacher assessment. Senate Bill 10 191 propelled the state of Colorado towards a new focus on Educator Effectiveness and a new educator evaluation system. All students in Colorado will have effective teachers in their classrooms and effective leaders for their schools. Evaluation provides teachers and principals with clear expectations f or their performance and with ongoing feedback and support needed to

PAGE 15

7 improve performance" (SCEE, 2011). The SCEE focused on P 20 standards alignment in the state education system and grappled with the questions: What makes an effective teacher? What makes an effective principal? How do we measure educator effectiveness? The Council's goals included an evaluation system by which teachers are: Evaluated using multiple fair, transparent, timely, rigorous, and valid methods, at least 50 percent of which is determined by the academic growth of their students, afforded a meaningful opportunity to improve their effectiveness, provided the means to share effective practices with other educators statewide. (Engdahl, 2010) Under this new law, teachers who do not m eet the effectiveness standards for two years in a row go back to non probat ionary status, (SCEE, 2011), yet there is no specific provision to improve the effectiveness of such a teacher or to improve the classroom experience for his/her students. Focusing on this root cause elevates the importance of naming the qualities of an effective educator, but will this focus cause schools to implement the collaborative professional learning required to improve a teacher's practice? Because of the stated goals of th e SCEE and knowledge and research about professional learning outlined in the literature review of this document, Colorado schools have an immense opportunity to use this new Educator Effectiveness law to rethink and revise professional learning, and to ca tapult the system into one that benefits all students. Currently, many schools do not spend ample resources training teachers, nor do they design teacher learning based on substantiated learning theory, and more focus needs to be placed on practice in the classroom (Borko, 2004; MET, 2012).

PAGE 16

8 The Measures of Effective Teaching Project (MET, 2012), funded by the Gates Foundation claims: The nation's collective failure to invest in high quality professional feedback to teachers is inconsistent with decades o f research reporting large disparities in student learning gains in different teachers' classrooms (even within the same schools). The quality of instruction matters. And our schools pay too little attention to it. (p. 2) Like the schools in the MET stud y, Colorado schools are likely to have similar discrepancies in both the quality of professional learning and in the levels of student achievement between classrooms. Without emphasis on professional learning, the design of this new system assumes that lab eling a teacher on a rubric itself will increase that teacher's effectiveness (Darling Hammond, 2013; MET, 2012) The hope of this study is to discover how professional development is planned and implemented in the new educator effectiveness system, and h o w districts plan to support a teacher's improvement once they have been labeled by the teacher evaluation rubric This knowledge will inform other educators as they grapple to create an educator effectiveness system this school year. Framed in both critic al theory ( Brookfield, 2005; Mezirow,1981 ) and socio cultural theory ( Hartley, 2009; Tharpe, Estrada, Dalton, & Yamauchi, 2000; Warschauer 1997) the intent of this study was to gather data about the beliefs and the practice influencing the opportunities fo r feedback, support, and professional learning that will follow the teacher evaluation process. In data analysis, the merit of a feedback, support, and professional learning will be ascertained based on the principles of these theories.

PAGE 17

9 Critical theory req uires that solutions to this yet undefined problem of effective educators consider the complexity of the job and reject any solution that simply labels a teacher as effective or ineffective, a model akin to labeling products on an assembly lines in factori es as effective or ineffective (Denning, 2011; Hargreaves & Fullan, 2012; Smith, 1998). Fortunately, the Colorado Educator Effectiveness Law indicates an appreciation for the complexity of teaching by including language that brings attention to the need fo r evaluation followed by ongoing feedback and quality support (SCEE, 2011). Feedback and support in education has tended to be oversimplified and low quality (Borko, 2004, Darling Hammond, 2013), and critical theory requires that data collected in this stu dy be scrutinized against the research of quality professional learning outlined in the literature review of this document. Quality professional learning, designed through the lens of socio cultural theory, requires that teachers learn collaboratively by a nalyzing actual classroom practice and by applying new learning to their own classroom (Bransford et al, 2000; Tharpe, Estrada, Dalton, & Yamauchi, 2000; Warschauer, 1997). Sociocultural theory also places value in the apprentice learning model described by Lave and Wegner (1991) and therefore, the study will seek data about how schools are using expert teachers as models in these social learning settings. Darling Hammond (2013) stresses the importance of professional learning in the evaluation process: I t is important to link both formal professional development and job embedded learning opportunities to the evaluation system. Evaluation alone will not improve practice. Productive feedback must be accompanied by opportunities to learn. Evaluations should trigger continuous goal setting for areas teachers want to work on, specific

PAGE 18

10 professional development supports and coaching, and opportunities to share expertise, as a part of recognizing teachers strengths and needs. (p.99) Darling Hammond's (2013) advise ment on quality evaluation mirrors beliefs inc luded in socio cultural theory. In this study, professional learning was deemed high quality if it allowed teachers to collaborate in meaningful ways to improve their planning, assessment, and instruction. The ultimate goal of this study was to understand professional learning practices that move beyond simple models of teacher and student learning and towards effective models of expert teachers, creating social learning environments that motivate teachers to im prove their practice to engage more students in the classroom. Research Questions The study explored the following research questions: 1. How is professional development planned and implemented in the new educator effectiveness system? 2. What are the perspectives of district leaders on teacher learning as an aspect of the educator effectiveness system? 3. H ow do districts plan to support a teacher's improvement once they have been rated on the teacher evaluation rubric? 4. What role are expert teachers expected to take in districts in the educator effectiveness system?

PAGE 19

11 Significance of Study This study was significant because effective educators are a critical component in a student's civil right to a quality education, one that will give the m access to political, economic, and personal power in the world. In the 1954 United States Supreme Court Decision, Brown v. Board, Chief Justice Earl Warren argued for the civil right to education: Compulsory school attendance laws and the great expenditu res for education both demonstrate our recognition of the importance of education to our democratic society. . Today it is a principal instrument in awakening the child to cultural values, in preparing him for later professional training, and in helpin g him to adjust normally to his environment. In these days, it is doubtful that any child may reasonably be expected to succeed in life if he is denied the opportunity of an education. Such an opportunity, where the state has undertaken to provide it, is a right which must be made available to all on equal terms. . (Landmark Cases, 2013) In the spirit of this ruling, all students in Colorado are entitled to effective educators who facilitate learning experiences to ensure access to power in our society and who recognize the individual needs and strengths of each child. Student power in society comes from an understanding of social constructs such as the common justice system, the common military, and the common economic systems in the United States, and requires that all students build the reading, writing, and speaking skills required to communicate their needs to those in power (Goldin Dubois, 1999). Students are also entitled to be in schools

PAGE 20

12 where leaders have a clear vision for student learning and s upport teachers in a climate that promotes perpetual learning (SCEE, 2011). Further, the Colorado Constitution ensures "All Colorado children equal access to thorough and uniform educational opportunities" (Lobato v. Colorado, 2013). This notion of all st udents having equal access to a high quality education should be the beacon for the educator effectiveness concept and law in Colorado. Finally, t he study was also significant because Colorado had just mandated that an Educator Effectiveness System be impl emented in every school. Local Context This study took place in two school districts in a large metropolitan area. District A was in the third year of implementation of a new teacher evaluation system and created its' own evaluation tool. District B was in the first year of implementation of a new teacher evaluation system and adopted the state evaluation tool. Both school districts value teacher leadership as evident by their teacher leadership academies.

PAGE 21

13 CHAPTER II THEORETICAL FRAMEWORK The quest for educator ef fectiveness has a long history (Markley, 2004) ; however the codification of educator effectiveness in a new law will likely cause rapid change in the education system. In the next school year, educators and policy makers are called to think qui ckly and carefully about how to enact this new educator effectiveness law and implementation responsibility falls largely on the school districts (SCEE, 2011). Conceivably, educators and policy makers implementing this new law have many different perspect ives about educator effectiveness, and applying a theoretical framework may bring clarity and alignment to this rapidly changing problem of practice. To frame my thinking about educator effectiveness in this study, I examined the theoretical frameworks of critical theory and sociocultural theory. Critical theory is a collection of beliefs that hold that humans need to think and reflect on the beliefs that are driving daily actions and be aware of the hegemony of capitalism that drives all that we do in li fe and subsequently in education (Brookfield, 2005). Looking through the critical theory lens, I am able to see how some policy makers may look at educator effectiveness from a point of view which values the individualism, capitalism, and oversimplificatio n of issues in education ( Hargreaves & Fullan, 2012; Smith, 1998). Because of my bias as a long time educator and staff developer, I focused on research and solutions that recognize that education is complex and has multifaceted solutions. I rejected the i dea that schools need to be run like a business that promotes the individual and competition. Alternatively, in this study, I placed value on the collective approach of teacher learning supported by sociocultural theory ( Brookfield, 2005 ;

PAGE 22

14 Hartley, 2009; Pe ck, Gallucci, Sloan, &Lippincott, 2008 ) which posits that learning takes place in social settings and bolsters the notion that teachers must work together to become more effective educators (Warschauer, 1997). Further, Bransford et al. (2000) supports that sociocultural theory is the framework for the apprenticeship model of learning that lends credence to utilization of teacher leaders as a strategy to guide the learning of apprentice teachers. My penchant towards sociocultural theory and the power of teac her collaboration leads me to believe teacher learning is paramount to student learning, and I believe that thoughtful teachers can learn together to improve their practice and become more effective for children. Critical Theory Critical theory encompass es a web of ideas that encourage humans to think deeply about and reflect on the beliefs that are driving daily work and actions ( Mezirow,1981). One facet of critical theory is the notion that capitalism, sometimes unconsciously, drives thinking and action in most western political systems including education policy. To illustrate this point, Karl Marx wrote that capitalism works to concentrate the majority of the resources to the few and enslaves the others; this desire for resources dominates all social r elationships (Brookfield, 2005). Stephen Brookfield (2005) builds on the ideas of Marx and defines critical theory in four parts: 1) The exchange of goods in the economy "generates a series of tensions created by the desire of some people for emancipation and the wish of others to prevent this desire being realized" (p. 23); 2) People need knowledge to free themselves from the oppression of capitalism; 3) Human beings are not just objects in critical theory, instead humans embrace the theory and want to liv e in a world that embodies this theory; 4) The theory creates a vision for how people might live

PAGE 23

15 more collectively in the future. Brookfield (2005) asserts that the education system is dominated by the individualism of capitalism and explains that adult de v elopment is collectively formed, and one person cannot be economically free if another is not. Critical theory forces me be watchful of ideas and systems related to the new educator effectiveness law that encourage teachers to compete against one another for high ratings on a rubric or for higher pay at the expense of working towards the collective good of increasing achievement and engagement for all students. As I conducted this research through the lens of critical theory, I was critical of ideas and so lutions that fail to acknowledge the complexities of educating humans and try to fix schools by running them like factories or businesses. Critical Theory Education, Capitalism The capitalist view of the world influences the structure and culture of bo th student and adult learning in schools ( Brookfield, 2005; Hargreaves & Fullan, 2012; Smith, 1998). According to Brookfield (2005), critical theory posits that examining education through a capitalist viewpoint allows educators and policy makers to Do something highly complex (help humans learn) within a system that is organized according to bureaucratic rationality and modes of factory production. Such system ignores complexity and assumes, for example that learning takes place at predictable times each week, in the same location, and follows the rationale of a curriculum divided into discrete and manageable units. (p. 6) An illustration of the oversimplifications of education can be connected to the comparison of technical problems to adaptive cha llenges. In short, one person can easily fix a technical problem with a simple solution a student has no lunch money, lend her

PAGE 24

16 two dollars. An adaptive challenge is multifaceted, complex, involves many people and organizations, and seeks a long term soluti on a student has no lunch money, the educator determines she is living in poverty and does not have acce ss to healthy food on many days. C onsult a network of experts to develop a plan for long term resolution to her hunger problem. This model cautions lead ers against employing technical fixes for adaptive challenges ( Heifetz & Linsky, 1997). In my opinion, applying a capitalist model to fix schools and improve teacher performance is a technical fix and will fail to bring about long term, sustainable solutio ns for students. Critical theory requires critical thinkers to become aware of capitalism's influences on the education system that is often compared to a factory system. Frank Smith, in his book The Book of Learning and Forgetting (1998), traces capital ism's roots in education back to the influence the military had over factories and subsequently the influence factories had over schools: This was the militaristic model that invaded the one room schoolhouse. What specifically went into schools was walls ( or rather barricades). Students would no longer be mixed up together, but instead would be grouped according to age and ability, all to be treated the same way and expected to learn together throughout their school careers, until they graduated a standardi zed, predictable, and reliable product. (p. 47) Frequently, students and adults within the K 12 system are taught in the same format, with the same curriculum, in the same time f rame, and are measured by the same assessments (Denning, 2011). Like a facto ry, teaching in some schools and districts has been reduced to scri pts and step by step procedures and ignor es the students for the sake

PAGE 25

17 of uniformity of the system (Denning, 2011; Hargreaves & Fullan, 2012). The factory model is not a match for the comple x process of educating humans (Denning, 2011; Hargreaves & Fullan, 2012; Smith, 1998) because so many moving parts are required to educate individual students and teachers. Further, critical theory requires critical thinkers to examine the influence of capitalism that stretches beyond K 12 education and into higher education. Without a college degree, students have little possibility of earning a living wage in our capitalist society. In order to gain entrance into a college or universit y, students must be exposed to a common curriculum of knowledge and skills which prepare them to pass a college entranc e exam created by a corporation. A s a result of this system, such corporations dictate curriculum in schools (PBS, 2013). Further bolste ring the need to be critical about the influence of capitalism in the education system is the fact that the same corporations that create the college entrance exams also guide curriculum and sell standardized testing packag es to K 12 school systems. S uch c orporations are estimated to earn between $400 and $700 million dollars a year in the standardized test and curriculum market (PBS, 2013). For example, College Board creates and scores the SAT college entrance exam, creates the Advanced Placement exams, an d is creating the new Common Core State Standards as well as the supporting curriculum materials such as Spring Board (CollegeBoard.org, 2013). The influence and power College Board and other businesses have on education is enormous and well beyond the pow er of a classroom teacher, student, parent, or principal. Critical Theory in Educator Effectiveness As the country and the state of Colorado begin to implement the new educator

PAGE 26

18 effectiveness law, policy makers, educators, and community members must be cr itical of the beliefs and theories that beget the definition of an effective educator. As a nation, we are not in agreement about the purpose of education nor the qualities of an effective educator; this ambiguity in framing of the problem does not lend it self to solutions. One belief about education is that an effective educator builds the self esteem of a child or inspires great musicians and artists (Smith, 1998). Another belief, held by Thomas Jefferson, is that effective educators build citizens and su ch education is necessary for self government (Jefferson, 1789). The adoption of the Common Core State Standards is evidence of the current belief that an effective educator causes growth in student learning on commonly agreed upon content and skills stand ards (Common Core, 2013). The belief that is most closely aligned with capitalistic priorities is the idea that effective educators cause children to gain the knowledge and skills that will allow the children to sustain the economy and this idea is termed Value Added. Value Added models seek to understand how effective teachers cause student achievement of specific knowledge and skills that create an educated citizenry who add to a viable economy as measured through standardized test scores. T hese models measure the effect of a school and a teacher on a student achievement by using one or more years of student test scores combined with other variables ( National Research Council and National Academy of Education, 2010). Researchers from the National Bureau of Economic Research assert, "s tudents assigned to high VA teachers are more likely to attend college, attend higher ranked colleges, earn higher salaries, live in higher SES neighborhoods, and save more for retirement" ( Chetty, Friedman, & Rockoff, 2011, p.1) In this line of thinking, the objective of an educator is to cause students to grow each year

PAGE 27

19 and to ensure that the student becomes more financially viable in our capitalistic society. This judgment fails to mention other reasons one might seek an e ducation. In my opinion, educational stakeholders need to be critical of the perspectives and beliefs that are driving the implementation of the new educator effectiveness law so that sound policy is created for all students. Such sound policy would inclu de a broader vision for the purpose of education, which would require critical thinkers to measure student growth using a variety of assessments beyond standardized tests. Further, the vision of an effective educator would be broadened, and an educator's v alue would be measured beyond students' standardized tests scores, which are not sensitive enough to determine a teacher's value. One idea in critical theory holds that adults (in this case educators) become more effective when they work together collec tively (Brookfield, 2005), and without careful thinking, the Educator Effectiveness Law may cause competition among teachers that is counter productive to the collective focus of meeting the needs of all students in a school (Hargreaves & Fullan, 2012). Fo r example, high stakes evaluation may not be productive for increasing the effectiveness of educators, because it might prevent educators from sharing strategies and information that might give one teacher an advantage over another in the evaluation system (Hargreaves & Fullan, 2012). Effective educators must be wary of the perils of a competition based system and look toward a more collective and united approach to helping all students grow (Brookfield, 2005). In my opinion, policy makers, educators, and c ommunity members need to abandon the capitalistic view of education and seek a new framework to think about creating more effective teachers in the classrooms one such framework is sociocultural theory.

PAGE 28

20 Sociocultural Theory Sociocultural theory is a web of ideas attributed to the work of Vygotsky and posits that learning takes place in social settings before it becomes internalized (Tharpe, Estrada, Dalton, & Yamauchi, 2000); "Sociocultural theory is essentially the theory of development (Vygotsky, 1978)" (Tharpe et al., 2000, p. 9), and this theory is related to effectiveness because it provides a framework to think about change and development of humans (Tharpe et al., 2000). Vygotsky (1978) writes, "The transformation of interpersonal processes into an intrapersonal one is the result of a long series of developmental events" (p. 57), and this lends credence to the adaptive challenge of educating both teachers and students. Warschauer (1997) described: Vygotsky further believed that this development pr in cipally took place through a form of apprenticeship learning; interaction with teachers or peers allowed students to advance through their zone of proximal development (i.e., the distance between what they could achieve by themselv es and what they could ac hieve when assisted by others). (No page) Althoug h Vygotsky did not create socio cultural theory, many scholars have linked the import of collaborative work, between teachers or students, to his work (Bransford, Brown, Cocking, Donavan, Pelligrino, 2000; Th arpe et al., 2000; Warschauer, 1997). Lave and Wenger (1991) built on the work of Vygotsky and described an apprenticeship model of learning as a method for the novice to be mentored by the expert (Bransford et al., 2000), and this notion provides support for using highly effective teachers as leaders for their colleagues.

PAGE 29

21 Sociocultural Theory in Education Sociocultural theory is a cornerstone of thoughtful learning structures and holds that learning is cultural and social rather than an individual exper ience ( Kozulin, Gindis, Ageyev, Miller, 2003). Learning environments that promote a sense of community through a common culture of safety, experiment, interaction, and teacher and peer feedback, enhance student learning experiences (Bransford, et al., 200 0). Bransford, et al., (2000) explain "In such a community, students might help one another to solve problems by building on each others knowledge, asking questions to clarify explanations, and suggesting avenues that would move a the group towards its goa l (Browne & Campione, 1994)" (Bransford, et al. 2000, p. 25). Additionally, Kozulin, et. al, (2003) synthesized ten years of classroom observations and found that students in collaborative classrooms increase understanding of concepts, rather than simply acquire knowledge. In a collaborative classroom, discussions are carefully designed, and the teacher differentiates uses probing questions to meet the individual needs in the discussion Thoughtful 21 st Century teachers embrace sociocultural theory when students use technology as a tool to collaborate and make meaning. For example, Bonk (1998) asserted that technology alone does not increase student learning however, when the tool is used for collaborati on it engenders the cognitive and emotional connections required to ensure learning. Sociocultural theory calls for safe learning environments that value the process of learning without fear of mistake, growth through peer feedback, and learning amongst pe ers, and these same concepts must be applied when considering the growth of effective educators.

PAGE 30

22 Sociocultural T heory in Educator Effectiveness Evaluation alone will not increase effectiveness of an educator rather, professional learning, framed in thi s theory of social construction of knowledge, will promote learning among teachers (Darling Hammond, 2013). I n investigations of teacher learning and change by Peck, Gallucci, Sloan & Lippincott (2009), a connection is made between collaborative te acher de velopment and the socio cultural theory: We conclude that the sociocultural learning theoriesmay offer a useful theoretical framework for interpreting complex social processes underlying organizational renewal, innovation, and change" (p. 16). To illustra te this point, Peck et al. (2009) found that in cases of new state policies and mandates, individual teachers developed creative implementation ideas, but policies did not take hold until a collective group of faculty embraced the change in practice, and f urther, this change process almost never occurred in the presence of an administrator or leader. Collaborative teacher learning that brings about teacher change is described in the literature review and may take many forms including mentoring, apprenticesh ip relationships between expert and novice, peer laboratories and teacher led in service (Bransford et al., 2000; Darling Hammond, 2013; NBPTS, 2011 ). To ensure that an effecti ve teacher is in each Colorado c lassroom, CDE has outlined guidelines for princi pals to conduct teacher evaluations (CDE, Fact Sheet, 2013) ; however, research supports that teacher change comes from collaboration among teachers and often away from administrators (Hargraves & Fullan, 2012; Peck et al, 2009). School districts, in implem entation of the new state mandate and in pursuit of effective educators, must include collaboration as a pathway to teacher growth, change, and effectiveness.

PAGE 31

23 Research has been conducted on effective educators, professional learning and teacher leaders, and critical theory and sociocultural theory provided valuable frameworks to review the relevant literature for this study. In accordance with these theories, research studies were included in this study if they recognized the complexity of teaching and le arning and if they promoted collaborative teacher learning.

PAGE 32

24 CHAPTER III LITERATURE REVIEW Literature Review: Evaluations; Professional Learning; Teacher Leaders As Colorado schools build a plan to have an effective educator in each classroom, policy makers and educators have a host of research studies to inform their decisions on topics including effective evaluation, effective professional learning, and effective utilization of teacher leaders. The literature for this study included thirty peer revie wed research articles and seven books that describe research on the topics related to this proposed study. Research studies were considered a good match for this literature review if ideas presented harmonized with critical theory and sociocultural theory; More than twenty studies were rejected because they did not meet this criteria. Instead, t hey offered simple solutions to the complex idea of changing a teachers practice or because studies did not talk about the collaborative nature of teacher learning. This literature review begins by examining research and recommendations for effective evaluation, then identifies and describes components of effective professional learning, and finally gives examples of emerging practices in training and utilizing teache r leaders. Effective Evaluations To be successful, the new state evaluation system must cause educators to be more effective (SCEE, 2011). To accomplish t his goal, CDE requires that evaluation is ongoing and includes an educator self assessment, a review of annual goals and performance plan, a mid year review of annual goals and performance plan, a final rating goal setting and performance planning for the next school year (CDE, Fact Sheet, 2013). A longitudinal study will have to be conducted to determin e if the Colorado evaluation

PAGE 33

25 steps create more effective teachers, but in the present, the requirements, as written above, do not specify the constitution of post evaluation learning opportunities that likely influence the probability of educators becoming more effective. Educator effectiveness is a nation wide challenge of practice, and a wide variety of educational stakeholders and scholars have crafted recommendations for the implementation of a quality teacher evaluation system (MET, 2012;Munoz, Prat her, & Stronge, 2011). Nearly all research in this literature review cautions that teacher evaluation must be based on multiple data points, including teacher observation, student surveys, and student growth scores (Chetty, Friedman, & Rockoff, 2011; MET, 2012; Munoz, Prather, & Stronge, 2011). The National Board of Professional Teacher Standards (NBPTS), suggests that a new evaluation system is necessary but must be implemented correctly by convening the right stakeholders, specifying what is to be measu red, defining the process of measuring, clarifying how the measures will be applied consistently, defining the evaluation process, and defining ongoing support (NBPTS, 2011). The New Teacher Project's (TNTP) (2012) report on evaluation describes the import ance of treating all teachers as individuals and therefore differentiating the evaluation process based on student and teacher needs (TNTP 2010). Research conducted by t he Measures of Effective Teaching project (MET) (2012), which is sponsored by the Bill and Melinda Gates Foundation, found that teacher evaluators must be well trained and show their ability to use the instrument before using it in a high stakes setting. The MET project further found that to ensure a reliable rating, each teacher would requ ire four observations, each conducted by a different observer (MET, 2012). Another data point, student surveys have been shown to render reliable and valid teaching

PAGE 34

26 ratings ( Kyriakides, 2005; MET, 2012). School districts in Colorado can utilize the findin gs of these research studies as they design their teacher evaluation cycles and evaluation tools. Student growth scores are another possible measure of an educator's effectiveness. A teacher's purpose is to help students learn and it seems reasonable to measure a teacher's effectiveness using student growth scores; however, studies show mixed reviews about the validity of this practice ( Goe, Bell, & Little, 2008; National Research Council and National Academy of Education, 2010: Steele, Hamilton, & Stech er, 2010) Value added models measure the effect of schools and teachers on a student achievement by using one or more years of student test scores and other variables connected to learning ( National Research Council and National Academy of Education, 2010 ). Some researchers posit that too many variables exist to reliably connect a teacher to a student test score ( Goe, Bell, & Little, 2008; National Research Council and National Academy of Education, 2010: Steele, Hamilton, & Stecher, 2010). Other research ers find value added scores are a valid and should be used as one measure of a teacher's effectiveness ( Chetty, Friedman, & Rockoff, 2011; National Research Council and National Academy of Education, 2010: Steele, Hamilton, and Stecher, 2010). Regardless o f their belief in the validity of using test scores to measure a teacher's effectiveness, most of the experts mentioned above do believe that student achievement scores must be combined with other measures of teacher effectiveness to yield a valid and reli able rating. Effective Professional Learning Learning Forward (2013), a national consortium for staff development, proposed a definition of professional learning to be used in the reauthorization of the Elementary

PAGE 35

27 and Secondary Education Act: "The term professional development' means a comprehensive, sustained, and intensive approach to improving teachers' and principals' effectiveness in raising student achievement'' (Learning Forward, 2013). The work of many researchers has helped to illuminate specifi c characteristics of effective professional learning and these include being participant driven, being data driven, giving specific feedback on clear targets, modeling, and coaching. These notions are described in the literature review. Participant Drive n. P articipant driven indicates that learning is based on the sta ted needs and preferences of teachers. Professional learning must be planned based on the needs of teachers, be facilitated through teacher centered instruction, and b e grounded in teacher in quiry, reflection, and experimentation ( Allen, 2007; Cross, 2012; Darling Hammond & McLaughlin, 1995). Hewitt and Weckstein, (2012) offer an illustration of this practice from teachers in Oakwood, Ohio who improved their practice by self selecting professional learning goals based on student assessment results. Following data analysis, teachers in Oakwood choose from a menu of professional learning experiences including joining a specialized professional learning community (PLC) or observing an expert teacher ( Hewitt & Weckstein, 2012). Thibodeau (2008) found when teachers have choice in PLC membership, choice in PLC outcomes, and choice in PLC norms, PLCs are most effective in making instructional practices that benefit students. Data Driven. Data driven is the practice of using results from student assessments to plan professional learning (Thompson, n.d.). Successful PLCs examine stude nt work including standardized tests, writing samples, and other formative assessment; which provides direction to meet student needs and subsequently,

PAGE 36

28 professional learning needs ( Cross, 2012; Darling Hammond & McLaughlin, 1995; Smith, Johnson, &Thompson 2012). Student data drives discussion in small PLCs, where one teacher presents a successful strategy used to render quality work, and student data is utilized to plan a school wide workshops on a school improvement goal ( Smith, Johnson, &Thompson 2012). Specific Feedback on Clear Targets. A clear target refers to a teacher learning goal that is based on data and followed by a teacher action for growth (i.e., participating in a lesson study) (Learning Forward, 2013). Professional learning should be al igned to targets which enhance student outcomes, are mutually agreed upon among the teacher, the evaluator, and the staff developer, are grounded in learning research, and are aligned to the school or district evaluation rubric (Cross, 2012;Learning Forwar d, 2013; MET, 2013) More specifically, Cross (2012) provides an illustration of this concept from instructional coaches in Fort Lupton, Colorado who began their work by pouring through research on adult learning which lead to the co creation of a profess ional learning rubric (Cross, 2012). The targets should include both content targets and process targets (i.e., literacy or critical thinking) and should be discretely worded so that follow up feedback is actionable and can cause a change in practice ( Cros s, 2013;MET, 2013) Additionally, Wiliam (2012) advises that specific feedback requires an atmosphere of trust, a focus on the data rather than the person, and a focus on aspects that are within the recipient's control. Modeling. Traditionally, teaching is a profession that takes place in the isolation of peers, but promising practices including peer observation and teaching laboratories allow teachers to learn more about their craft from a primary source or model ( Darling

PAGE 37

29 Hammond & McLaughlin, 2013; NBPT S, 2011). Peer observations take place in many forms and are often informal, formative, and non evaluative (Center for Teaching and Learning, 2013). In peer observation, teachers gain insight into their own practice when they watch expert teachers teach ch ildren, and reflect on the practice in the observation ( Darling Hammond & McLaughlin, 1995; NBPTS, 2011). Further research about peer observation conducted by City (2011) found that well planned peer observations might focus on a specific guiding question or problem of practice that turns the observation into a collective problem solving experience. City (2011) also discovered that peer observation is different than evaluation because all participants learn from the experience and change practice based on t he observations and subsequent debrief. Laboratory (lab) classrooms may take place in many forms but often feature one teacher who models exemplary practice in one or more aspects of teaching and learning (Center for Teaching and Learning, 2013). Lab class rooms provide support through modeling and directly connect to the work of teachers and their students ( Darling Hammond, & McLaughlin, 1995). They also utilize teacher leaders as role models and mentors for effective practice (NBPTS, 2011). Coaching. Inst ructional coaching has many forms, but often includes one on one conversations between a teacher and a non evaluative coach around the topics of planning, assessment, and instruction (Center for Teaching and Learning, 2013). Kraft and Balzer (2012) found t hat teachers who rec eived instructional coaching were more effective for students than those teachers who did not. In another example, Cornett and Knight (2008) found that teachers who just attend a workshop had a 15% implementation

PAGE 38

30 rate of the new skill l earned. In contrast, teachers who attend a workshop and have follow up coaching have an 85% implementation rate of a new skill. Professional Development Increases Student Achievement Research has shown that investing in teacher learning causes gains in s tudent learning (REL, 2007; Slabine, 2012; Taylor & Tyler, 2011) and increased student learning is the ultimate goal of the educator effectiveness concept (SCEE, 2011). Because learning is complex, it is challenging to connect one teacher's work to one stu dent's learning: "The connection seems intuitive. But demonstrating it is difficult" (Regional Education Laboratory [REL], 2007, p. iii). Taylor and Tyler (2011) found that student achievement increases in years of teacher evaluation and especially in the three years following evaluation if investments in professional learning are made and quality feedback is given. A case study in Duval County Public Schools in Florida presented by Slabine (2012) explains the school district has made professional learning its top district priority and reports a consequent increase in student achievement. The professional learning plan for the district includes placing professional learning as the top priority, district level coordination of all professional learning opportu nities, active pursuit of professional learning grants, time during the school day for teacher collaboration, and a professional learning accountability board that uses progress monitoring tools to track progress of professional learning (Slabine, 2012). T he Shultz Center for Teaching and Leadership, a partner organization of Duvall Public Schools, reports a significant difference in student achievement on the Florida State Reading test as teachers increased their days in professional learning (Slabine, 201 2).

PAGE 39

31 Effective Professional Development Led by Teacher Leaders The New Educator Effectiveness Law calls for an increased use of teacher leaders; Colorado State Senator Michael Johnston, the author of the Educator Effectiveness Law in Colorado wrote (201 3): SB 191 recognizes that there are gaps in the teacher career ladder. Besides taking an administrative position, highly effective teachers are often without options if they want to remain in the classroom while assuming leadership roles. SB 191 empowers the Governor's Council to create rewarding career ladders for these educators, ensuring that highly effective teachers will have compensated opportunities to become leaders in their field by recording and disseminating their best practices to educators across the state. The law, in its final version, did not include a plan for highly effective teachers to become leaders, but Colorado can learn from the work of many school districts that are building the skills of teacher leaders and using this expertise to imp rove instruction. Research suggests that principals need help to provide an effective educator in every classroom, and this distributive model of leadership could include expert teachers leaders (Barth 1990; Fink, 2013; Hallinger, 2003; Lambert, 1998). Th e label expert' or highly effective educator' is nebulous in Colorado because most districts are just beginning to evaluate teachers with the new educator effectiveness rubric this school year (SCEE, 2011). Regardless of this complexity, it is likely th at there are expert teachers in all school districts that can be leveraged now to increase the effectiveness of all teachers in all classrooms. Sociocultural theory supports leveraging expert teachers as leaders in

PAGE 40

32 the implementation of the new educator ef fectiveness system. The apprentice model of learning, introduced by Lave and Wegner (1991), which calls for experts to work along side novices as a way for both to improve their craft (Bransford et al., 2000), is a useful model for this work. Additionally, when principals collaborate with expert teachers to design and implement professional learning, teachers in the building tend to have more engagement and trust in the learning activities (York Barr & Duke, 2004). There are benefits to using teacher leade rs, but also, there are consequences when teacher leaders are not utilized; To illustrate this, Curtis (2013) warns that, "When districts fail to use high performing teachers to support their colleague's growth and development, administrators and coaches r emain overburdened, and teachers as well as many students are left lacking individualized support" (p. 1). I ndividually i ncreasing the effectiveness of every educator in Colorado is a heavy lift, and teacher leadership is a possible leverage point to meet this demand. To ensure that every child has an effective teacher in the classroom, principals and expert teacher leaders must collaborate to design professional learning experiences for all teachers. Additionally, to ensure that expert teachers remain in the classroom, principals and districts must provide a career path for expert teachers that utilizes their skill set and provides challenges that keep them engaged in classroom work. Principals and Expert Teachers As Partners in Professional Learning Ev en if a principal has the time and the expertise for instructional leadership, a positive school culture may require that principals lean on expert teachers to create and sustain strong professional learning experiences for all teachers. Principals are ult imately responsible for evaluation, feedback, and post evaluation support (SCEE, 2011),

PAGE 41

33 however, research shows that they must partner with teacher leaders, who are effective educators, to realize the goal of an effective educator in every classroom (York Barr & Duke, 2004). Principals have a wide array of responsibilities that siphon time each day and some principals lack the expertise to provide instructional leadership; Fink (2012) found in a survey of close to 3,000 principals, most self reported being a novice or emerging in instructional leadership in the areas of purpose, engagement, assessment, curriculum, and classroom climate. Hallinger (2003) examined the issue of instructional leadership among principals and found any intention to provide instr uctional leadership, especially in secondary schools, is complicated by the fact that in many cases principals have less expertise than the teachers whom they supervise" (p. 331). An additional factor, a study by Horng, Klasik, and Loeb (2009) explained th at principals may lack the time to focus on instruction. Their year long study of principals in Florida exemplifies this issue and found that high school principals spent 5.8% of their day on instructional tasks, middle school principals spent 8.38% on instructional tasks, elementary principals spent 9.3% of their time on instructional tasks. In light of this research, effective teachers must be used as leverage to increase the learning of all students. Gaps in the Research Taken together, the research reviewed names the characteristics and practices of quality professional learning, shows that quality professional learning has caused student achievement, and calls for an increased use of teacher leaders in professional learning. The research described in th is literature review does not include examples of how schools use teacher learning in evaluation and educator effectiveness systems. This study sought

PAGE 42

34 to understand how school systems move beyond a rating on an evaluation tool and towards post evaluation, collaborative professional learning that increases the effectiveness of educators; Therefore, the question for this research study remained, how is professional development planned and implemented in the new educator effectiveness system? (Borko, 2004; Hargreaves & Full an, 2012 ; MET project, 2012 ). Additionally, Colorado did not include a career ladder for effective teachers in the final version of its law and a second question remains, what role are expert teachers expected to take in districts in the educator effective ness system? This study sought to understand the ways districts plan to use teacher leaders, who understand the complexity of educating students and teachers, in their educator effectiveness systems.

PAGE 43

35 CHAPTER I V METHODOLOGY Research Design The purpose of this study was to understand how schools and districts are considering professional learning as a vehicle to increase the effectiveness of educator s To reach this understanding, research w as conducted in two school districts to discover the plans for post evaluation professional learning In both district s teacher leaders, evaluators, a nd district administrators participate d in a questionnaire and an interview ( Creswell, 2014; Gliner, Morgan, & Leech, 2009) and the combination of these metho ds will seek the answers to these research questions: How is professional development planned and implemented in the new educator effectiveness system? What are the perspectives of district leaders on teacher learning as an aspect of the educator effectiv eness system ? H ow do districts plan to support a teacher's improvement once they have been rated on the teacher evaluation rubric? What role are expert teachers expected to take in districts in the educator effectiveness system? Study Participants To seek answers to my research questions, I use d a purposive sample (Gliner, Morgan, & Leech, 2009) of 1 12 teacher leaders in two school districts who share d their thinking about educator effectiveness and post evaluation professional learning through a quest ionnaire. Next, I interviewed three c entral office administrators in both districts to gain their perspective on teacher learning within the new teacher evaluation system and

PAGE 44

36 to understand their thinking about the data rendered from the teacher leader ques tionnaire. These participants who have different t ypes of jobs (teacher leaders and central office administrators) offer ed diverse perspective s on educator effectiveness and the pr ofessional learning that followed an evaluation rating (Creswell, 2014; Mer riam, 2009). The teacher leaders in the study offer ed insight into the research questions because they understand quality instruction, they have been evaluated, and they have received specific training o n effective professional learning through the teacher leadership academies Central office administrators provide d insight for the research questions because they collaborated with state officials to gain understanding on the legal mandate of the educator effectiveness law, and they have experience in planni ng how a large school system manages teacher accountability and professional learning within educator effectiveness and the new teacher evaluation system. Choosing Expert Teacher Leaders Almost 8,000 teachers are employed in School District A and B and a participant selection process will narrow the sample to 1 12 teachers who provide d insight into the research questions. At the time research was conducted, not all teachers were rated on the state educator effectiveness rubric; therefore I could not cho ose a highly effective teacher based on this measure for my study. Instead, I asked the leaders of the Teacher Leadership A cademies to distribute the questionnaire to teacher leaders and attempted to collect data at least 100 teacher l eaders to respond to the Educator Effectiveness Staff Development Support Questionnaire (Appendix A).

PAGE 45

37 Interviewing District Administrators Interviews with central office administrators were grounded in the data from the results of the teacher leader questionnaire. Once ques tionnaire results were analyzed central office administrator s, in both districts ( n=6 ) including the Director of Teacher Development and Leadership, the Director of the Peer Observers, the Manager of the Teacher Leadership Academy, the Director of Educat or E ffectiveness, the Director of the Teacher Leadership A cademy, and the Director of Professional L earning were interviewed to gain additional perspective on trends from the questionnaire and beliefs in educator effectiveness, post evaluation teacher lear ning, and teacher leadership. This combination of survey, interview, and diverse job type triangulated the data and give broader insight into my research questions (Creswell, 2014; Merriam, 2009) Data was collected in two large, school districts; School D istrict A is co nsidered urban, has 155 schools, and reports the following demographics: ethnicity (15% Black, 5 8% Brown, and 20% White), free and reduced lunch (72%) and 35% of students are ELL. School District B is considered suburban, has 61 schools, and reports the following demographics: ethnicity (1 2 % Black, 18% Brown, and 56% White, 18% Asian), free and reduced lunch ( 27 %) and 8.8 % of students are ELL. District A and District B, were selected because they are in different stages of implementation of the educator effectiveness system and this difference yielded a wider range of perspectives on the subject. In 2010, two years before the educator effectiveness state manda te, District A began development on an educator effectiveness system and created a new evaluation tool. District B is in its first year of implementation and chose to use the state evaluation tool. Additionally, both District A and District B were selecte d

PAGE 46

38 because both place value on teacher leadership as evident by their teacher leadership academies and one of the aims of this study was to collect data about how teacher leaders are being leveraged to increase the effectiveness of all teachers (Creswell, 2 014) Table 2.1 list study participants. Table 2.1 Study Participants District A District B Questionnaire 789 Teacher leaders were contacted 84 Teacher leaders Responded 69 Teacher l eaders were contacted 27 Teacher leaders responded Total Questionnaires from Both Districts = 112 Interviews District A District B Central Office: Director of Teacher Leadership Academy n=1 Central Office: Director of Teacher Leadership Academy n=1 Central Office: Director of Professional Learning n=1 Central Office: Director of Professional Learning n=1 Central Office: Director of Educator Effectiveness n=1 Central Office: Director of Educator Effectiveness n=1 Total Interviews in District A=3 Total Interviews in District B=3 Total Interviews in bot h districts=6 Data Collection Educators faced with the challenges and opportunities of educator effectiveness shared their professional experiences in the data collection process through a questionnaire and interviews. Both quantitative and qualitative data was collected to answer the four research questions and this variety of sources allows for triangulation of data ( Creswell, 2014; Johnson, Onwuegbuzie, & Turner, 2007 ; Merriam, 2009). Results

PAGE 47

39 from the quantitative qu estionnaire of teacher leaders help ed me to be more pointed in my interview questions with the central office administrators (Johnson, Onwuegbuzie, & Turner, 2007). Qualitative interview data was thick, explain ed questionnaire results in from a different perspective and capture d the in dividual experience and perspectives that exist for those who coordinate the Educator Effectiveness, the new teacher evaluation, and professional learning at a district level (Merriam, 2009). Questionnaire A n electronic questionnaire gather ed quantitative data from a sample of teacher leaders to determine beliefs about the educator effectiveness evaluation tool and to collect data about the characteristics of post evaluation professional learning (Gliner, Morgan, & Leech, 2009 ; Creswell, 2014). The close ended questionnaire (Gliner, Morgan, & Leech, 2009) was designed to gather information to answer the research questions Specifically, p articipants were asked about the presence of professional learning practices described in the literature review. Each participant complete d the questionnaire presented in Survey Monkey on their own computers and on their own time. The questionnaire took less than fifteen minutes to complete. The questionnaire had 13 statements pertaining to educator effectiveness and or professional learning. Respondents choose from an ordinal scale with responses including: strongly disagree, disagree, agree, strongly agree, unsure, and N/A (in District B) ( Gliner, Morgan, & Leech, 2009 ). Q uestions were modeled after a questionnaire created for District B and were adapted from a professional learning questionnaire created at the University of Missouri (Missouri, 2006). Three teacher leaders in District B gave feedback on the readability and relevancy of the questionnaire and revisions were made (Creswell, 2014). Results from the questionnaire

PAGE 48

40 inform ed interview questions and in the interviews, I asked participants about trends in the questionnaire data (Johnson, Onwuegbuzie, & Turner, 2007). The question naire is included in Appendix A. Interviews Interviews "capture behavior, feelings, or how people interpret the world" (Merriam, 2009, p.88) and are flexible enough to capture the complexity of experiences that likely exists in the educator effectiveness process. The i nterview questions were semi structured (Merriam, 2009 ) as some questions were asked of all participants based on questionnaire analysis the theoretical framework and the literature review of this study (Merriam, 2009), and some questions ca me organically from conversation. In the interviews, participants were asked to explain their perspective on the problem' or matter' of educator effectiveness and explain the role professional learning played in their personal educator effectiveness fram ework. The interviews were designed with participant s needs in mind. Interviews took place in December and January at a time and location convenient for the participant The participants had time constraints a nd each interview lasted less than one hour. Interviews were recorded on three different audio devices and transcribed as soon as possible after the interview. As I listen ed to the audio, multiple times for accuracy, I transcribe d using methods suggested by Merriam (20 09 ) The in terview protoc ols are included in Appendix B. The data collection plan is illustrated in Table 2.2.

PAGE 49

41 Table 2. 2. Data collection plan Date Questionnaire Interview Participant Follow up November 2013 112 Participants Invite to follow up meeting in spring December 2013 January 2014 Manger of Teacher Leadership Academy, District A; Director of Teacher Leadership Academy, District B Director of Teacher Development and Leadership, District A; Director of Professional Learning, District B Director of Peer Observers, District A; Director of Educator Effectiveness, District B Invite to follow up meeting in spring Data Analysis The data analysis plan for this study was built on a foundation of c ritical theory, sociocultural theory, the f our research questions, and the literature review and this foundation was the lens through which data is analyzed ( Creswell, 2014; Kawulich, 2004 ; Saldana, 2008 \ ). To ensure more study validity, the data analysis plan was steeped the ideas of many res earchers including Creswell (2014), Gliner, Morgan, & Leech ( 2009 ), Johnson, Onwuegbuzie, & Turner, ( 2007 ) Kawulich (2004), Miles and Huberman (1994), Merriam (2009), Punch (2004), Saldana, (2008) and their thinking gave me a

PAGE 50

42 structure to navigate through this topic that I am connected with both personally and professionally. Questionnaire Analysis Gliner, Morgan, & Leech ( 2009 ) suggest "Exploratory studies of a new topic may just describe what people say or feel about that topic"(p. 90) and therefore, in this study, this new topic of educator effectiveness was described in part, through a Descriptive Research Approach. A Descriptive Research Approach includes several variables that are not compared to each other but instead are described through a variety of methods including averages, percentages, and frequency distributions ( Gliner, Morgan, & Leech 2009 ). To illustrate this concept, teachers in this study reported their beliefs th at "The district has a to plan support teachers once a teacher has been rated on the new evaluation tool" and "In post evaluation learning, participants are involved in determining topic and content of the session" but these two items will not be compared to each other. Data analysis began with determining the frequency with which 1 12 teacher leaders reported that teacher learning is part of educator effectiveness and that evaluation is followed by support Once this data was collected, I created a spreads heet that showed the frequency and percentage (Gliner, Morgan, & Leech, 2009 ) with which teachers leaders reported professional learning was planned and implemented with the following characteristics from the literature review: p articipant driven, data driven, clear targets, specific feedback on targets, modeling, and coaching. Because the ordinal data was not likely be normally distributed, for each item, a percentage will be calculated to show the frequency with which a teacher reports an experience strongly disagree, disagree, agree, strongly agree, unsure, and N/A (in District B) In this study, use of practices in literature

PAGE 51

43 review is the desired state and therefore considered the highest standard of practice. During analysis, I kept track of questions that were raised about the data and seek information on these questions in the interviews with district administrators. Figure 3. Questionnaire Analysis Questionnaire H ow will schools and districts support a teacher's improvement once they have been labeled by the teacher evaluation rubric? How is professional development planned and implemented in the new educator effectiveness system? Look for patterns of use and non use practice in lit. review Look for teacher learning as an element of professional learning ; frequency and report % of each response Possible Outcomes Questionnaire 1. Report the frequency of use and non use of practices in Lit erature Review through percentages of agreement 2. Develop i nterview questions based on trends of use and non use of professional learning practices Look for use of teacher leaders ;

PAGE 52

44 Interview Analysis Data Reduction After each interview, I listened to the audio several times and created a transcript (Merriam, 2009; Kawulich, 2004). Once transcripts were created, I will b egan coding which is the first stage of ana ly sis (Merriam, 2009; Kawulich, 2004; Punch, 2004; Creswell, 2014 ). My analysis plan was based on the work of Miles and Huberman (1994) ( as cited by Punch, 2004), which includes three stages data reduction, data display, an d data conclusion. D ata reduction occurs throughout the analysis process when the researcher codes data, links data to other data, and then when th e researcher uses data to create a theory ( Miles & Huberman, 1994). When I first read the word data reduction, it had a negative connotation for me because it sounded like an attempt to break or take away from participant responses. I then thought about th e process of reduction in cooking and how the process transforms raw material into a new, interesting product. In my analysis, I remained open to all possible categories but based on the theoretical framework and literature review of this study, I coded (u se data reduction) the int erview transcripts with these anticipated codes : t eacher learning part of educator effectiveness '(Literature Review) types of professional learning' (Literature Review), use teacher leaders in professional learning a ttentio n to complexity of learning (Theoretical Framework), a ttention to teacher social learning (Theoretical Framework) (Creswell, 2014; Merriam 2009). I did not restrict myself to these codes and 63 codes emerged during analysis. Once data was coded, I began the next part of data reduction, the categorizing process. Data Cate gorizing Once data was coded I began the categorization process. I color coded interview text and cut and pasted excerpts from each of the six interviews

PAGE 53

45 into a table with the 63 categories. I then connected each category to my theoretical framework and research questions. During this process, I drew a graphic display of categories that were connected to my questions and the theoretical framework. I also drew graphics of themes tha t recurred with the six participants (Kawulich, 2004; Saldana, 2004). Once categories were compiled and labeled, I began to think about relationships between categories. Relationships between categories, become the basis of assertions made in the later ste ps of analysis (Creswell, 2014; Merriam, 2009). Data Display. Punch (2004) writes that Miles and Huberman (1994) value creating visual representations of qualitative data throughout the analysis process and they propose, "Data displays organize, compress, and assemble information"(Punch, p. 174). As I was planning the methodology for this study, my thinking became more clear when I created a graphic display for the analysis plan ( Kawulich, 2004 ; Saldana, 2004). Miles and Huberman (1994) (as cited in Punch, 2004) further propose that this graphic display of data changes over time as understanding of the data becomes more focused. To begin my analysis, I created of a data display that has categories with arrows between them and each link or connection is expl ained in the display (See Figure 2.4). During this process, I drew a graphic display of categories that were connected to my questions and the theoretical framework. I also drew graphics of themes that recurred with the six participants (Kawulich, 2004; Sa ldana, 2004).

PAGE 54

46 Drawing and Verifying Con c lusions. The third stage of data analysis is drawing and verifying conclusions (Miles & Huberman, 1994 as cited in Punch, 2004). Conclusions are often drawn early in the process and then solidified and verified throughout data analysis (Punch, 2004). Because the educator effectiveness process is in its' nascent stages of implementation, I hesitated to make conclusions and rather will make assertions' about where the process is at current state. Three categories emerged as possible findings including: 1) Teacher learning is a desired state of the educator effectiveness system in District A and B but is not yet implemented. 2) District A is using teacher leaders to coach, to provide professional learning, to provid e feedback, and to evaluate teachers to increase the likelihood of teacher growth. District B has the desire to use teacher leaders but is not yet strategically using them in such roles. 3 ) Systemic issues block District A and B from using evaluation tools to yield data from teacher observations to inform and plan professional learning. After arriving at these three big findings, I went back through all of the data to verify that I had eno ugh data to support these findings (Miles and Huberman, 1994 as cited in Punch, 2004). I reorganized data by cutting and pasting into tables that provided evidence for these initial findings. Next, I described my initial findings in more than 50 pages of te xt and through this process, I reorganized the categories again into two main topics: 1) the disconnect between the teacher evaluation system and the teacher growth system 2) The selection, training, and utilization of teacher leaders.

PAGE 55

47 Figure 2.4 Interview Analysis Data Display Data Redu ction CODE CODE CODE Category Anticipated Codes and Categories Open to any emerging codes/or categories Teacher learning part of educator effectiveness Types of professional learning Use of Teacher Leaders C ompl exity of learning (Theoretical F ramework) S ocial learning (Theoretical Framework) Drawing adapted from Saldana 2008 Category Look for relationships between categories Data Display Ongoing visual display of data Possible Outcomes Interviews Understandings of and Assertions about H ow professional development planned and implemented in the new educator effectiveness system ; How t eacher learning framed as a dimen sion of educator effectiveness; How schools and districts plan to support a teacher's improvement once they have been labeled b y the teacher evaluation rubric; R oles expert teachers take in schools in the educator effectiveness system.

PAGE 56

48 Issues of Reliability and Validity Issues of validity and reliability have been considered in the research design. The most problematic issue was that sample of expert teachers was not random or representative of all teacher s or teacher leaders in the t wo school districts (Gliner, Morgan, & Leech, 2009 ) Many expert teachers have participated in the teacher leadership academies in years prior to the study and more will participate in future years their voices were not included in this study. Selecting te achers from the leadership academies was the best method available for this study because when data collection begins, teachers were not yet rated on the state educator effectiveness system Beyond the issues associated with sampling, other precautions ha ve been taken to address issues of validity and reliability. The study is framed in both critical theory and socio cultural theory which identify and explain my lens for conducting research ( Kawulich, 2004 ; Merriam, 2009). Methods in this study employ mu ltiple sources of data including interviews ( n = 6 ) and a questionnaire distributed to teacher leaders ( n =1 00 ) across two school districts and three type of jobs (Creswell, 2014; Merriam, 2009). Conclusion Rivers and Sanders (1996) found that the teacher i s the most important factor, in school, for student achievement. Hargreaves and Fullan (2012) call this finding the most abused finding in educational research. However, it is a commonly held belief and is driving substantial reforms in policy and practic e. Quite frankly, it is difficult to disagree that great teachers are powerful. Therefore, this proposed study is driven by the desire to

PAGE 57

49 ensure all students have the opportunity to work with an effective teacher who helps them gain the reading, writing, and speaking skills that are required to access power in the world. To reach this desired state, this study seeks to understand the ways that two school districts are implementing the state mandated Educator Effectiveness Law. To prepare for this study, I reviewed literature of many experts on professional learning, evaluation, and teacher leadership so I might understand critical lessons learned about post evaluation teacher learning in schools beyond District A and District B. Darling Hammond (2013) is one expert who is concerned about teacher learning in evaluation: Virtually everyone agrees that teacher evaluation in the United States needs an over haul. Existing systems rarely help teachers improve or clearly distinguish those who are succeeding fro m those who are struggling. The tools that are used do not always represent the important features of good teaching. It is nearly impossible for principals, especially in large schools, to have sufficient time or content expertise to evaluate all of the te achers they supervise, much less to address the needs of some teachers for intense instructional support. And many principals have not had access to professional development and support they need to become expert instructional leaders and evaluators of tea ching. Thus, evaluation, in its current form often contributes little to either to teacher learning or to accurate, timely information for personnel decisions. (p. I) Through this study, I hope to learn how School District A and School District B are using teacher learning as an aspect of educator effectiveness. As I capture and

PAGE 58

50 analyze the thinking of study participants through the questionnaire and interviews, I hope to compare their thinking to research completed outside of study so that new knowledge is gained about effectively implementing an educator effectiveness system that considers teacher learning as one aspect of the process. Ultimately, by discerning the cu rrent practices of educator effectiveness in these two districts, this study hopes to provide clear data and assertions that help all educators make the best possible decisions about educator effectiveness and positively impact the educational experiences for all students.

PAGE 59

51 CHAPTER IV FINDINGS AND IMPLICAITONS Findings The Educator Effectiveness Law intended to increase student achievement by ensuring that every classroom has an effective teacher. To ensure that an effective teacher is in every Colorado classroom, CDE outlined guidelines to implement a new teacher evaluation system followed by post evaluation support (CDE, Fact Sheet, 2013) District leaders in this study, with valuable knowledge of adult learning theory and practice, grappled with the connection between the new evaluation system and what is known about teacher growth. The findings of this study aim to explain how six district leaders and the teacher leaders they support have attempt ed to navigate through the new state mandate and work toward the complex goal of coupling a teacher evaluation system and a teacher growth system. The primary finding of this study is that in both districts, post evaluation learning did not exist in the i mplementation of the new evaluation system. In other words, teachers were not necessarily learning as a result of evaluation. District leaders in this study had a deep foundation of knowledge and experience with sociocultural theory and adult learning pri nciples. With this foundation, district leaders knew that to ensure an effective educator in all classrooms, their systems needed to move beyond teacher evaluation. My analysis found that a gap between teacher accountability and teacher growth existed in t hese districts for two reasons: A) The lack of communication between evaluators and professional developers and B) The capacity of an evaluator to provide post evaluation professional learning.

PAGE 60

52 The second major finding was that leaders in District A relie d on teacher leaders (TLs) as a major strategy to increase the effectiveness of all teachers and District B aspired to do the same. However, district leaders concluded the teacher leader program could be more intentionally implemented. This reliance on TLs matched the sociocultural beliefs held by participants who considered peer learning an effective method to grow the skills of teachers. Examination of these findings elucidated the struggle to harmoniously combine teacher accountability and teacher growth into a system that ultimately results in more students having access to quality education. Less than half of TLs in this study indicated that their school/district had an articulated plan to support a teacher's post evaluation learning and interestingly, each district leader countered that TLs gave districts too much credit. In District A, 47.06% of teacher leaders surveyed reported that their school/district had an articulated plan for post evaluation support. In District B, 33.34% of TLs surveyed report ed that their school had a plan for post evaluation support. TLs responses did not indicate a high level of agreement with the existence of post evaluation support, yet all six district leaders believed TLs rated the district too high on this question. If the goal of educator effectiveness in these districts was to use a new evaluation tool, the task was complete, but if the goal of educator effectiveness was increasing a teacher's effectiveness with students, the task was ongoing. District leaders knew wh at the qualities of professional learning associated with evaluation looked like and knew their teachers were not getting it. The following describe these findings in detail by first focusing on the depth of knowledge of district leaders, then on the lack of connection between teacher evaluation

PAGE 61

53 and teacher growth, and finally on the use teacher leaders as strategy to increase the effectiveness of all educators. District Leaders Understand Adult Learning District leaders in this study had experience and knowledge in creating more effective educators. Each participant described quality teacher learning using similar language and with characteristics substantiated by literature on quality adult learning incl uding incorporating coaching (Center for Teaching and Learning, 2013; Kraft and Balzer, 2012), utilizing peer modeling (City, 2011; Darling Hammond & McLaughlin, 1995; NBPTS, 2011), using data ( Cross, 2012; Darling Hammond & McLaughlin, 1995; Smith, Johnson &Thompson 2012) giving choice ( Allen, 2007; Cross, 2012 Darling Hammond & McLaughlin, 1995) and giving clear feedback (Cross, 2012; Learning Forward, 2013; MET, 2013). Additionally, each participants spoke with language connected to ideas of the soci ocultural theories that characterize learning as social, ongoing, and complex (Bransford et al, 2000; Tharpe, Estrada, Dalton, & Yamauchi, 2000; Warschauer, 1997). Each of these district leaders understood that having an effective educator in every classro om was a complex task and framed post evaluation teacher learning as a critical aspect of the educator effectiveness system. These district leaders recognized there was no quick fix for educator effectiveness and recognized that teachers improve their practice through collaborative, social learning. Ellen 1 the Director of the Teacher Leadership Academy in District B, posited "We tend to think the presence of a document, i.e. teacher evaluation tool, will someway magically transform what has been apparent on the part of the teacher practitioner or on the part 1 All names in the study are pseudonyms

PAGE 62

54 of an administratornothing could be more incorrect". District leaders in this study believed teacher learning a nd collaboration connected to student needs was paramount for a growth system. Table 1 illustrates how district leaders believed a teacher's effectiveness is increased and shows the connection between their statements and literature on adult learning, soci ocultural theory, and critical theory.

PAGE 63

55 Table 1 Sample language from district leader interviews that indicate connection to literature on adult learning and theoretical frameworks of this study District Leader Excerpt from interview Connection literature on adult learning Connection theoretical frameworks Pruitt, Director of Peer Observers, District A "I think small actionable feedback, so frequent and actionable feedback that comes from someone they deem as reliable" Quality feedback (Learning Forward, 2013; MET, 2013; Cross, 2012) Sociocultural theory; peer collaboration and feedback (Bransford et al, 2000; Tharpe, Estrada, Dalton, & Yamauchi, 2000; Warschauer, 1997) Sociocultural theory; peer collaboration and f eedback followed by time for self reflection to make meaning (Bransford et al, 2000; Tharpe, Estrada, Dalton, & Yamauchi, 2000; Warschauer, 1997) Sociocultural theory; social construction of knowledge (Borko, 2004, Darling Hammond, 2013) Socio cultural framework (Bransford et al, 2000; Tharpe, Estrada, Dalton, & Yamauchi, 2000; Warschauer, 1997) Critical theory; using data to in valid ways drive action (Brookfield, 2005) Sociocultural theory; safe learning environments ( Bonk ,1998 ) John, Manager of Teacher Leadership Academy, District A "[Offerings] that are more community oriented and engaging with a body of peers with opportunities for consistent reflection and reapplication and adjustment over time" Participant driven professional learning communities (Darling Hammond & McLaughlin, 1995; Cross, 2012; Allen, 2007) Terri, Director of Teacher Development and Leadership, District A "How is it that we can develop a system that gives teachers feedback about where they have strengths, where they have growth areas, and then providing opportunities for them to continue to grow?" Clear feedback on targets (Learning Forward, 2013; MET 2013; Cross, 2012) Lorie, Director of Educator Effectiveness, District B "Having that ongoing support to implement whatever the learning is." Ongoing support with peer coaching; Cornett & Knight (2008) Ellen, Director of Teacher Leadership Academy, District B "What data are you using to inform not only your initial investment in this work, but your monitoring and your outcomes?" Data driven professional learning ( Smith, Johnson, &Thompson 2012; Darling Hammond & McLaughlin, 1995; Cross, 2 012 ) Jane, Director of Professional Learning, District B A professional learning community looking at student work, for example, when there's more people participating the threat, the feelings of fear aren't as prevalent, because everybody is in it together Professional learning community ( Hewitt & Weckstein, 2012)

PAGE 64

56 Teacher Learning not Connected to Evaluation at the District Level All six district leaders reported that no specific plan for teacher learning connected to the evaluation tool existed. Specifically, there was no clear connection between data yielded from the evaluation tools and post evaluation professional learning offe red to teachers. Without an explicit plan for teacher learning tied to the tools, participants concluded the new evaluation system was not different from the old system. Lorie, the Director of Educator Effectiveness in District B, quipped "Frankly [this is ] a practice that we've done in District B, it's really not that different." The districts already had an evaluation tool and these leaders believed teacher learning existed beyond the use of an evaluation tool. Lack of communication between evaluators an d professional developers. In both districts professional developers and evaluators did not have an effective communication system and most often evaluation data and professional learning knowledge existed in separate silos. District leaders reported that principals, the evaluators, were not always aware of professional learning opportunities available to support teacher learning on a specific indicator. Pruitt, The Director of Peer Observers District A, illustrates this communication gap: "[I don't know] i f there's been any sort of support for building leaders onwhat .are you offering your teachers after their evaluation to figure out what is productive I think the systems are not connected." Terri, The Director of Teacher Leadership and Development fr om District A, explained that principals may not be aware of professional learning in their own building: it depends on what the school leader has in place and what they might know is even

PAGE 65

57 available at their school. There's a need to have more insight in to how they can tap into and understand what structures are in place for that teacher. Jane, the Director of Professional Learning in District B, reported her district also suffered a lack of communication between evaluators and professional developers. Jane explained I can't guarantee with all certainty that everything at every school is being implemented with fidelity and that it will make a difference to the students." Participants in the study reported that a teacher could not count on communication between their evaluator and those who planned professional learning. Another impediment to communication, study participants reported that professional developers had no way to create data driven professional learning connected to teacher scores on the ev aluation tools. Pruitt explained that the teacher's association closely protected evaluation data, and consequently professional learning was planed without a link to educator effectiveness One of Pruitt's goals was to move educator effectiveness from an accountability system to a growth system; Her objective was to create a system where Teacher Effectiveness Coaches (TECs) use evaluation data collected by Peer Observers (POs) to focus teacher coaching cycles. This objective did not come to fruition becaus e evaluation data could not be utilized by anyone but evaluators. In a more favorable example, participants explained that schools in District A reported a collective score on teacher evaluations prompting each school to choose two focus areas for teacher learning. This is a promising first step towards a greater connection between the evaluation system and teacher learning. Evaluator capacity to give quality feedback. District leaders understood that feedback on the indicators on the evaluation tools woul d help teachers increase their

PAGE 66

58 effectiveness. Even more, district leaders thought the indicators might serve as a common language of instruction for the district. Unfortunately, they reported that many administrators lacked the time, the pedagogical experi ence, and practice to give good feedback. This is consistent with literature on instructional leadership skills of principals ( Fink, 2012; Hallinger, 2003; Horng, Klasik, and Loeb, 2009). Terri, District A, and Lorie, District B, explained that evaluators might have the ability to give feedback, but they did not have the time to follow up in a timely manner with the multitude of teachers on their caseload. Ellen, District B, explained that many teachers had difficulty understanding evaluator feedback under the old tool and that the new, more complex tool may make it even more difficult to receive actionable feedback. Lorie, District B, posited that the whole new system hinged on an evaluato r's ability to give actionable feedback. She recounted multiple stories of reviewing evaluations with principals "and I can tell youalmost with 99% [accuracy] you'd read their evaluations and they were glowing." Lorie further explained that although the n ew state tool was specifically designed to elevate feedback, improvement would not occur unless evaluators improved their skill at giving the type of feedback that guides a teacher towards increased effectiveness. In this study district leaders surmised th at a principal's capacity to give feedback to teachers was limited, and leaning on teacher leaders to help provide feedback was a promising practice in the quest to have an effective teacher in every classroom. Teachers Lead Growth of Peers In its origin al draft, the Educator Effectiveness Law called for the construction of a teacher career lattice that would give teachers leadership opportunities while simultaneously keeping them in the classroom; this career lattice did not come to fruition.

PAGE 67

59 This lapse in implementation might be a critical flaw in the new educator effectiveness system and to illustrate this point, Curtis (2013) warns that, "When districts fail to use high performing teachers to support their colleagues growth and development, administrat ors and coaches remain overburdened, and teachers as well as many students are left lacking individualized support" (p. 1). In the spirit of sociocultural theory, participants in this study believed that teacher leadership is an essential strategy for coll aborative, thoughtful, teacher learning, as well as a leverage point to successfully implement the new educator effectiveness system. The literature on teacher leadership supports the validity of the district leaders' beliefs about these new roles (Curtis, 2013; Leithwood & Jantzi, 2006; Moreau, Dumouchel, & Sallafranque St Louis, 2012; York Barr & Duke, 2004) and even asserts that teachers often have a higher level of collaboration and growth when an administrator is not in the room (Hargraves & Fullan, 20 12; Peck et al, 2009 ). TLs hold various roles in District A and District B, but just having TLs does not increase the effectiveness of all teachers. In interviews, leaders in both districts self reflected and reported that they could be more strategic in s electing, training, and using TLs. Roles of teacher leaders in District A. District A was at the forefront of use of TLs. Their advantage was due to access to grant money, early implementation of an educator effectiveness evaluation tool, and the sponsor ship of the District Superintendent. During the time of the study, the Superintendent expressed his confidence in the strategy of TLs in an internal district communication that read No other highly skilled, knowledge based profession has our traditional o rganizational structure, with one leader responsible for coaching and supervising 30 to 40 professionals. Our profession will

PAGE 68

60 greatly benefit from smaller teams with greater collaboration and peer learning" (Superintendent District A, 2014). TLs in Distric t A held a variety of roles that aspired to increase the effectiveness of all teachers. These roles included Peer Observers (POs), Teacher Effectiveness Coaches (TECs, not related to technology), both of which were funded and trained by the central office, and school based TLs. Such use of TLs indicated a district commitment to sociocultural approach of peer learning and collaboration. Peer Observers (POs) evaluated and gave feedback to teachers on their caseload twice a year using their district's evaluat ion tool. This year, POs evaluated 2,382 teachers (about 1/3 of the teaching force); these evaluations were ancillary to principal evaluations. Peer observers entered scores into a centralized database but did not collaborate or calibrate scores with princ ipals. The PO observation model varied each year. This year it included teachers new to the district, teachers who were below a certain score cut point last year, and those seeking TL positions. PO observations added another data point in the body of evide nce utilized for a teachers' evaluation. Teacher Effectiveness Coaches (TECs), another central office TL role, also aspired to increase the effectiveness of all teachers. Each of the 50 TECs worked in two schools and assisted the leadership teams and the teachers in achieving the goals on the school Unified Improvement Plan (UIP). The Mission of the TEC Team is to increase the achievement of all students by building capacity for educators' growth, teachers as leaders, and sustainable systems. TECs work in 82 of the 153 schools in the district to provide ongoing coaching that is proven to increase teacher effectiveness in implementing specific skills ( Cornett & Knight, 2008).

PAGE 69

61 Of the TLs in District A who participated in the questionnaire, 41.67% reported th at their post evaluation professional learning included an observation from a peer or expert teacher. This indicated that POs and TECs provided services to less than half of the TLs in the district. To close this gap in services District A began to employ TLs in buildings. TLs in these roles lead small teams of teachers at the school level and support them in their learning and evaluation needs. John, the Manager of the Teacher Leadership Academy, described the role as "end to end support and evaluation" an d explained that teacher leaders coached, observed, gave feedback, co planned lessons, and facilitated team planning. Interestingly, some TLs in District A served as both coach and evaluator, and this combination is at the heart of quandary of connecting t eacher learning and teacher growth. The school based TL structure was in its nascent stages at the time of this study, and district leaders were aware of the need to define and shape the specific role of school based TLs in order to make better use of this resource. Roles of teacher leaders in District B. TLs in District B did not have roles explicitly tied to the educator effectiveness evaluation system; however, all three District B leaders in this study believed TLs could be utilized to increase teacher effectiveness. Jane, District B, believed that her district needed to offer more roles to teacher leaders because expert teachers know instruction best. Ellen, District B, explained "Unfortunately I think that's where District B probably could do a bett er job of following through in that [Using TLs to promote educator effectiveness] realm." Specifically, Ellen believed that teacher leaders might be used to run professional learning communities (PLCs) and to model specific skills such as close readings an d math critiques. Lorie, District B, touted the effectiveness of the First Year Teacher

PAGE 70

62 Mentor Program that supported new teachers and also offered leadership positions to expert teachers. Lorie believed more money would be required to increase the probabi lity of using TLs in the role of instructional coach. Going forward, District B might utilize their 69 Teacher Leaders in specific roles to increase post evaluation support for teachers and might consider some TL strategies being implemented in District A. Table 2 provides a summary of teacher leader roles, selection, and training in both districts.

PAGE 71

63 Table 2 Roles of Teacher Leaders Role Description Selection Training Teacher Effectiveness Coach (TEC), District A 52 TECs in District Coach works in two schools, make agreements with principals meet UIP; Facilitates PLCs; planned and facilitated professional learning; coaches teachers in non evaluative setting; builds capacity in building teacher leaders; collaborated with district level instructio nal superintendents who supervise principals; Does not have access to data from evaluation tool unless offered by teacher Successful teacher minimum 5 years, successful coaching experiences, rigorous interview process Centrally trained in coaching, faci litation, standards, assessment, equity, ELL, content, and other district initiatives. Attend two week institute in summer, minimum of four hours a week. Trained in national coaching and facilitation models. One on one coaching visits from program directo r Centrally trained, 100 hours on observation and feedback at beginning of each school year, and additional throughout the year. One on one coaching visits from program director Training varied; two day summer institute on standards and leadership skills; attend once monthly cohort facilitated by Teacher Effectiveness Coaches on standards and leadership; some attend additional training on scoring on evaluation tool Centrally trained in coaching, facilitation, standards, assessment, equity, ELL, co ntent, and other district initiatives a minimum of four hours a week. Trained in national coaching models. One on one coaching visits from program director Training varied; attend 100 hours of training in teacher leadership academy over two school years; topics included data literacy, equity, coaching, facilitation, assessment Peer Observer (PO), District A 43 POs in District Teachers on special assignment; Go into classroom watch an entire lesson, record evidence of teacher and student behaviors at tha t time, align the evidence to new evaluation tool, assign scores based on where that evidence aligns; conduct a 30 minute feedback conversation after both evaluations H ired based on content areas and sometimes based on their own score on the new evaluation tool (this is available for some applicants and not others), rigorous interview process School based teacher leaders, District A 1,200 in District Part time teacher and part time teacher leader; role varies widely to meet school need; might coach teachers, lead PLCs, plan with teachers, facilitate PD, some in pilot schools evaluated teachers on certain parts of new tool; sometimes school leaders utilize them, sometimes they do not No consistent selection process, some voted in and some cho sen by principals; some have to show a minimum score on evaluation tool First Year Teacher Mentor, District B 7 in District Teachers on special assignment; Assigned a caseload of 15 25 first year teachers, helped new to profession teachers with planning, assessment, instruction, classroom management, and any other needs; non evaluative position. Considered most master teacher by principal; rigorous interview process School based TLs District B No exact number; 69 completed the TL academy Role varied widely to meet school need; might coach teachers, lead PLCs, plan with teachers, facilitate PD; sometimes school leaders utilized them, sometimes they do not No consistent selection process, some voted in and some chosen by principals

PAGE 72

64 Recruiting teacher leaders. School based TLs are a valuable resource (Barth 1990; Fink, 2013;Hallinger, 2003;Lambert, 1998) and ye t there is not a strategic system for selecting teacher leaders in either district. Ellen, District B, recruited participants by speaking at principal and leadership meetings and through word of mouth. John, District A, pointed out "The identification and selection, right now [of TLs], it's all over the place. Some schools vote people in, some schools, its only the principal who is selecting people, some schools have this very thoughtful process." It is possible that because school based TLs were not strate gically selected, they were also not strategically used because principals did not learn about TLs through an interview process. John hoped the selection process could be differentiated based on the exact role of the teacher leader. Leaders recruiting scho ol based TLs could learn from the PO and TEC selection processes. For example, POs are selected through a rigorous interview process in which they must provide evidence of content area expertise and have earned high scores on personal evaluations. TECs, al so managed by the central office, must provide evidence of experience in successfully coaching teachers and leading professional learning communities in their interviews. If a district invests in teacher leaders, they must also invest in a careful selectio n process. Without a quality TL selection and recruitment process, TLs may not add value to the educator effectiveness system. Ensure strategic training of teacher leaders. Like principals and teachers, TLs require training and support to be effective. The roles of POs, TECs, and First Year Teacher Mentors had been established for more than three years, and these TLs participated in ongoing, quality professional learning provided by the central office. POs had about 100 hours of professional learning on observation and feedback at the

PAGE 73

65 beginning of each school year and continued to practice their skill in professional learning sessions throughout the school year. Both First Year Teacher Mentors and TECs participated in a minimum of four hours per week (ab out 200 hours per school year) of professional learning around standards, coaching, facilitation, planning, and assessment. The number of hours of professional learning does not always equate to quality learning; however professional learning offered to th ese three TL roles used practices represented in the literature on quality adult learning. This included incorporating coaching (Center for Teaching and Learning, 2013; Kraft & Balzer, 2012), utilizing peer modeling (City, 2011; Darling Hammond & McLaughlin 1995; NBPTS, 2011), using data ( Cross, 2012;Darling Hammond & McLaughlin, 1995; Smith, Johnson, &Thompson 2012) giving choice ( Allen, 2007;Cross, 2012; Darling Hammond & McLaughlin,1995) and giving clear feedback (Cross, 2012; Learning Forward, 2013; MET, 2013). Professional learning for TL roles was facilitated in large and small group settings and followed by coaching sessions facilitated by program directors. TLs in these three roles engaged in more training than principals had around standards, assessment, planning, coaching, giving feedback, and facilitating. Their training practices can be used as a model for TLs in all schools. School based TLs, in both districts, received fewer hours of training, but the sum of the description of their learning experience was congruent with practices of quality adult learning. TLs in District A were trained in small cohort groups that meet two full days before school started and once a month ther e after, about 32 hours a school year. TECs facilitated each of the cohorts of approximately 10 teacher leaders. TLs in District B received about 60 hours of training per school year around standards, data literacy,

PAGE 74

66 equity, and coaching. Ellen and other co ntent experts led the work of these cohorts. None of the TL training was explicitly connected to data yielded from the educator effectiveness evaluation rubric. TLs in both districts participated in some group professional learning but did not receive feed back on their specific performance in their roles. Understanding that principals do not always have the capacity to provide learning support to teachers, it is likely they lack capacity to support TLs. Therefore, a systems for training needs to be thoughtf ully considered and implemented to ensure TLs are equipped to provide the best learning opportunities for teachers. Essentially, this research suggests that because each of these six district leaders had deep knowledge of how to grow effective teachers, i t was difficult to implement the educator effectiveness system because they felt it emphasized teacher accountability rather than teacher growth. The paramount challenge faced by district leaders in the study was to actualize a connection between teacher e valuation and teacher growth. A connection they knew was necessary to fully implement the Educator Effectiveness Law. Implications Drawing from the results of this study it appears that to realize the ideal of an effective educator in every classroom, s chool districts might employ two strategies: 1) Thoughtfully connect the often disjointed systems of teacher accountability and teacher growth; 2) Carefully select, train, and utilize expert teacher leaders as an additional support for all educators. Contr ary to the spirit of this law, the truth is, in two large Colorado school districts, professional learning is not planned or implemented as a part of the Educator Effectiveness system.

PAGE 75

67 A first step towards connecting accountability and growth is underway in District A where the new evaluation tool is used as a common language of instruction. All district leaders in this study believed the teacher performance indicators on their evaluation tools were valid and worthy of pursuing, and the indicators on the r ubrics were used as the common language of good instruction in the districts. District A hoped this common language, spelled out in the indicators of the evaluation tool, becomes the foundation of all school and district professional learning opportunities including: professional learning community (PLCs) work, coaching sessions, feedback conversations, and in service days. Terri, District A, articulated her belief in using the tool as a common language by making sure that our teachers have an understandin g and a common definition of what it means to be effective. Ideally, district leaders would not accept grants, programs, or curriculum unless it matched to the common language of the district. Additionally the interview processes for all district educator s might be based on evidence of understanding and use of notions held in the indicators, which is the practice in some teacher leader selection in District A. Using the evaluation tool as a common language of teaching and learning could be a strategy to mo ve educator effectiveness beyond accountability and all six participants showed some inclination towards this practice. To further connect these systems of accountability and growth, schools might use the data yielded from the new evaluation tool to plan professional learning and actualize this common language of instruction. Professional developers should always use data to plan formal course offerings tied to specific indicators on the evaluation tool (Cross, 2012; Learning Forward, 2013; MET, 2013 ) If this were the case, evaluators could strategically offer learning choices to teachers connected to indicators on the rubric. Jane,

PAGE 76

68 District B, explained her desire that "the rubric will drive professional learning that we will need to have in our dis trict, at least that's my hope." Additionally, professional developers could use change in evaluation scores as one data point to track the effectiveness of professional learning offerings. One final benefit of such a system, when school and district leade rs use data to inform decisions, adjust professional development, and individualize learning for teachers, they are modeling good practice for teachers and their work with students. Using data from the evaluation tool to plan for teacher growth is compl ex. Because these systems have long been separated, evaluation has never really been viewed as a learning experience and actually, at times, has been seen as a threat. Sociocultural theory tells us that it is difficult for learning to take place under such conditions. This history of mistrust between evaluators and teachers has likely led to the prohibition of sharing evaluation data. Even if data sharing is for the benefit of planning professional learning, it is still forbidden by district policy and by e mployment law. A current example of this discord between evaluation and growth is illustrated this school year in which the new evaluation system coincides with new standards and assessment implementation; this situation requires a teacher to experiment an d innovate to improve their craft while simultaneously grappling with taking risks in a higher stakes evaluation system. Quality professionals demand accountability in their work and teachers are no exception, but the accountability offered through this ne w evaluation system has higher stakes than in the past. This culture may make teachers even less inclined to share their evaluation data. Any plan to use of evaluation data for professional learning will have to be thoughtfully constructed by multiple stak eholders. Both teacher accountability and teacher growth will

PAGE 77

69 be enhanced by strategic use evaluation data used for planning individualized professional learning. The second strategy to connect the accountability system to a growth system is to employ and train expert teacher leaders who can help principals provide feedback and provide quality professional learning. Much research asserts that principals do not have the time and in some cases, the expertise to provide quality professional learning to increa se the effectiveness of educators (Barth 1990; Fink, 2013; Hallinger, 2003; Lambert, 1998; York Barr, J. & Duke, K., 2004 ). An uncritical approach to educator effectiveness would accept that a principal could evaluate a teacher one time, hold a feedback m eeting, and expect educator growth. In accordance with the ideas of sociocultural learning theories, TLs, who are experts in their craft and content, could be used to model and train other teachers. This is promising practice is not yet fully actualized i n either districts because TLs have been named but not empowered with specific roles and quality training. Simply naming TLs would be an uncritical approach to educator effectiveness. More suitably, TLs must be carefully chosen, trained, and utilized in schools; However district leaders within this study wrestled with these complexities. District leaders admitted that the teacher leader selection process was less than strategic; for example, some TLs were nominated by principals, some were voted in, and some self selected into the position. Not all teachers have the skill to lead another in instruction, and therefore teacher leaders must be carefully screened, interviewed, and selected based on multiple data points including articulation of sociocultural learning beliefs, personal scores on the district evaluation tool, coaching disposition, and the ability to create strong relationships

PAGE 78

70 with adults in the school. To improve the selection process, teacher leaders might be chosen based on school needs; for example, one school may need a math expert or one school may need an expert in giving feedback. Schools would interview with these specific roles in mind. Once selected, TLs would be a valuable resource for teachers, but only if they have ongoing training in coaching, giving feedback, data literacy, planning, assessment, and instruction. In District A, Teacher Effectiveness Coaches provided much of the ongoing training for school based TLs. Finally teacher leaders need to be utilized effectively in their s chools, and this is perhaps one of the biggest challenges faced by district and teacher leaders. Like any other valuable resource, principals need training on how to effectively use teacher leaders and teacher leaders need to know how to make their skills fit the needs of the school. In District A, some TLs are being used as both evaluators and coaches and this dichotomy is at the heart of the complexity of combining teacher accountability with teacher growth. Keeping this complexity in mind, the work Glic kman (2002) would suggest proceeding with caution if using a TL as both a coach and an evaluator. Data in this study suggests that to make teacher leadership effective, principals need to give up some of the absolute control of their buildings and be willi ng to distribute the leadership to expert teachers. Perhaps, if TLs are better selected and trained, principals may be more willing to share leadership and this second strategy of teacher leadership could be realized. Connecting the accountability system and the growth system is complex but a worthy goal to pursue; A goal that will require thoughtful planning, a focus on adult learning for all educators, and the intentional and consistent use of the common language offered by the evaluation tools. Most cr itically, the integration of evaluation and growth

PAGE 79

71 requires a commitment to building strong relationships of trust between teachers and evaluators. It also requires time for teachers and evaluators to co create a belief system about this new purpose and ou tcomes of evaluation these critical steps cannot be circumvented or rushed. Sociocultural theory helps educators understand that the State and subsequently a district, cannot honestly call an evaluation tool a growth tool unless teachers, the learners in t his case, agree it is so. As the Educator Effectiveness system endures, further research is required to determine if a connected system of teacher accountability and teacher growth causes more effective educators or if educators are better served if the s ystems remain unconnected. Ultimately, students would benefit from a thoughtful district theory of the integration of accountability and growth as they experience both firsthand each day in the classroom. Perhaps if the adults in a school system can figure out how to fruitfully couple these two complex concepts, educators can better design a student's classroom experience and ensure that accountability through assessment actually causes student growth.

PAGE 80

72 REFERENCES Allen, D. (2006). The push to excellence: teachers fo cus on professional learning to lift student achievement. Journal of Staff Development 27 (1), 56 60. Auerbach, E. R. (1993). Reexamining English only in the ESL classroom. TESOL Quarterly 27, 9 32. doi: 10.2307/3586949 Barth, R. ( 1990) Improving schools from within. San Francisco, CA: Jossey Bass. Berg, J. H., Miller, L. R., & Souvanna, P. (2011). Boston shifts learning into high gear. Certificate program accelerates student learning by building teacher capacity. Journal of Staff D evelopment, 32 (3), 33 36. Borko, H. (2004). Professional development and teacher learning: Mapping the terrain. Educational Researcher, 33 (8), 3 15. Bransford, J., Brown, A., & Cocking, R. (2000). How people learn: Brain, mind, experience, and school (Expanded ed.). Washington, DC: National Academy Press. Brookfield, S. (2005). The power of critical theory: Liberating adult learning and teaching San Francisco, CA: Jossey Bass. Center for Teaching and Learning (2013). Teaching resources. University of Minnesota. Retrieved from http://www1.umn.edu/ohr/teachlearn/resources/peer/guidelines/ Chetty, R., Friedman, J. N., & Rockoff, J. E. (December, 2011). The long term impact of t eacher: Teacher value added and student outcomes in adulthood (Working Paper 17699). Cambridge, MA: National Bureau of Economic Research. Retrieved from http://www.nber.org/papers/w17699

PAGE 81

73 City, E. (2011). L earning from instructional rounds. Educational Leadership, 69 (2), 36 41. Clemans, A., Berry, A., & Loughran, J. (2012). Public anticipation yet private realization: The effects of using cases as an approach to developing teacher leaders. Australian Journal of Education 56 (3), 287 302. College Board. (2013). College board. Retrieved from www.collegeboard.org Colorado Department of Education. (May 2013). Educator effectiveness: A Colorado priority (CDE Fact Sheet). Retrieved from http://www.cde.state.co.us/sites/default/files/documents/educatoreffectiveness/do wnloads/factsheets/statemodel.pdf Cornett, J., & Knight J. (2009). Coaching approaches and perspectives Thousand Oaks, CA: Corwin Press. Creswell, J. W. (2003 ). Research design: Qualitative, quantitative, and mixed method approaches (2nd ed.). Thousand Oaks, CA: Sage Publications. Cross, J. (2012). Lasting legacy for achievement: Colorado coaches boost teacher effectiveness with a rubric for professional learning. Journal of Staff Development, 33 (5), 36 44. Curtis, R. (2013). Finding a new way: Leveraging teacher leadership to meet unprecedented demands. Retrieved from http://www.aspeninstitute.org/publications/finding new way leveraging teacher leadership meet unprecedented demands#sthash.r3eRYde3.dpuf Darling Hammond, L. (2013). Getting teacher evaluation right: What really matters for effectiveness and improvement New York, NY: Teachers College Press.

PAGE 82

74 Darling Hammond, L., & McLaughlin, M. W. (1995). Po licies that support professional development in an era of reform. Phi Delta Kappan, 76 (8), 597 604. DCPS. (2011, October 24.). Washington DC Public Schools. Survey: DCPS stakeholder sentiment improving in several categories. Retrieved from http://dc.gov/DCPS/ Denning, S. (2011, May). The single best idea for reforming K 12 education. Forbes Magazine. Retrieved from www.forbes.com/sites/stevedenni ng/2011/09/01/the single best idea for reforming k 12 education/ Engdahl, T. (2010, March). Teacher effectivenes s council starts 18 month run. Education News Colorado. Retrieved from http://www.ednewscolorado.org/2010/03/11/3679 teacher effectiveness co uncil starts 18 month run Fertig, B. (2013, July 23). Annual school survey shows parents are more satisfied than teachers. School Book. Retrieved from http://www.wnyc.org/story/302090 annual schools survey shows parents more satisfied than teachers/ Fink, S. (October, 2012 ). School and district leaders as instructional experts: What we are learning. Center for Educational Leadership, University of Washington. Retrieved from http://depts.washington.edu/uwcel/5d/level_of_expertise_article_revised_oct2012 .pdf Goe, L., Bell, C., & Little, O. (2008). Approaches to evaluating teacher effectiveness: A research synthesis. Washington, DC: National Comprehensive Center for Teacher Quality.

PAGE 83

75 Goldin Dubois. (1999). Increasing participation in American politics (Master's t hesis University of Colorado at Denver ) Hallinger, P. (2003). Leading e ducational c hange: R eflections on the practice of instructional and transformational leadership. Cambridge Journal o f Education 33 (3), 329 351. Hanushek, E. A., & Lindseth A. A. (2009). Schoolhouses, courthouses, and statehouses: S olving the funding achievement puzzle in America's public schools Princeton NJ : Princeton University Press. Hargreaves, A., & Fullan, M. (2012). Professional capital: T ransforming teaching in e very school New York: Teachers College Press. Hargreaves, A ., & Shirley, D. (2009). The persistence of presentism. Teachers College Record 111 (11), 2505 2534. Hartley, D. (2009). Education policy, distributed leadership and socio cultural theory. Educational Review, 61 (2), 139 150. Heifetz, R. A., & Laurie, D. L. (1999, January February). The work of leadership. Harvard Business Review, 75 (1), 124 134. Hewitt, K. K. & Weckstein, D. K. (2012). Programs in practice: Differentiated instruction: B egin with teachers! Kappa Delta Pi Record, 48 ( 1 ) 35 46. Horng, E., Klasik, D., & Loeb, S. (2010). Principal's time use and school effectiveness. American Journal of Education 116 (4), 491 523. Jefferson, T. (1789, January 8). [Letter to Richard Price]. Co py at Monticello.

PAGE 84

76 Johnson, L. (1964 January 8 ). President Lyndon B. Johnson's a nnual m essage to the Congress on the State of the Union. Retrieved from http://www.lbjlib.utexas.edu/johnson/archives.hom/speeches.hom/640108.asp Johnson, R., Onwuegbuzie, A. J., & Turner, L. (2007) Toward a definition of mixed methods research. Journal of Mixed Methods Research 1 (2), 112 133. Johnson, S. (n.d). Why teachers must have an effective evaluation system. American Federation of Teachers. Retrieved from www.danielso ngroup.org/.../ Why%20Teachers%20Must%20Have%20an ... Johnston, M. (2013). SB 10 191: Great teachers & leaders. Retrieved from http://www.mikejohnston.org/issues/sb 10 191/ Kawulich, B. (2004). D ata analysis techniques in qualitative research. Retrieved at www.eeraonline.org/journal/files/ 2004 /JRE_ 2004 _08_ Kawulich pdf Knight, J. (2007). Instructional coaching: A partnership approach to improving instruction Thousand Oaks, CA: NSDC. Kozulin, A., Gindis, B., Ageyev, V. & Miller, S. (2003). Vygotsky's educational theory in cultural context. Cambridge, United Kingdom: Press Syndicate of the University of Cambridge. !"#$%&'() & *'+,#-#"&'.)'/0123 &'45"678 9)' !"#$%&'()*+,-./,$*#$-.+'.,0*12#,$'",(+-3* ,&'4,(.,* %(*'(4'&'45-3'6,4*+,-./,$*.%-./'()7 :";<;7%;='#%'%>;'0123' 4?@@'45"678' AB7$;";7C;) ' !D"6#E6=;<&'F) & A";;G;"<&'+)&' *' H7%B76BI&':)'/011J9)'K;#C>;"'L;>#M6BI"'#7='<%I=;7%' BI%CBG;'B7'%;#C>;"'%"#67678'#7='5"B$;<<6B7#,' =;M;,B5G;7%'%;#C>678 )' 8,-./,$* 1 45.-+'%(9*:; /29 &' 20 O 03)

PAGE 85

77 Labato v. State of Colorado. (2013, April 16). Colorado Department of Law. John W. Suthers, Attorney General. Case can be retrieved at http://www.coloradoattorneygeneral.gov/departments/state_services/ education/lob ato Ladson Billings, G. (2009). The dreamkeepers. San Francisco, CA: Jossey Bass. Ladson Billings, G. (1995) Toward a theory of culturally relevant pedagogy. American Educational Research Journal 32 (3) 465 491. Landmark Cases. (2013). Key excerpts from the majority opinion, Brown I (1954). Retrieved from http://www.streetlaw.org/en/Page/519/Key_Excerpts_from_the_Majority_ Lambert, L. (1998). How to build leadership capacity. Education Week, 55 (7), 17 19. Learning Forward (2013). Standards for professional learning. Retrieved from http://learningforward.org/standards for professional l \ /learning#.UaId24KhBdQ Learning Forward (2013). Definitions for professional learning Retrieved from http://learningforward.org/standards for professional l learning#.UaId24KhBdQ Leithwood, K., & Jantzi, D. (2006). Transformational schoo l leadership for large scale reform: Effects on students, teachers, and their classroom practices. School Effectiveness and School Improvement 17 (2), 201 227. Leclerc, M., Moreau, A., Dumouchel, C., & Sallafranque St. Louis, F. (2012 December 10 ). Factors that promote progression in schools functioning as professional learning community. International Journal of Education Policy& Leadership, 7 (7), 1 14.

PAGE 86

78 Lloyd, K., Tienda, M., & Zajacova, A. (2001). Trends in educational achievement of minority student since Brown vs. Board of Education. Retrieved at theop.princeton.edu/reports/misc/trends_in_ed.pdf Markley, T. (2004 ) Defining the effective teacher: Current arguments in education. Retrieved from www.usca.edu/essays/vol112004/markey.pd Marshall, K (2005, June). Time to rethink teacher evaluation. Phi Delta Kappan, 86 (10), 730. Marzano, R. J., & Frontier, T. (2011). Effective supervision supporting the art and science of teaching Alexandria, V A: ASCD. Merriam, S. B. (2009). Qualitative research: A guide to design and implementation San Francisco: Jossey Bass. The MET Project. (2012). Gathering feedback for teaching, combining high quality observations with student surveys and achievement gains. Retrieved from http://www.metproject.org/downloads/MET_Gathering_Feedback_Practioner_Bri ef.pdf The Met Project. (2010). Initial findings from the m easures of effective teaching project. Retrieved from http://www.metproject.org/downloads/Preliminary_Finding Policy_Brief.pdf Meyer, D., Madden, D., & McGrath, D. J. (2004). English language learner students in U.S. public schools 1994 and 2000. Washington, DC : National Center for Education Statistics, U.S. Dept. of Education, Institute of Education Sciences. Mezirow, J. (1981 September ). A critical theory of adult learning and education Adult E ducation Q uarterly 32, 3 24.

PAGE 87

79 Missouri Department o f Elementary a nd Secondary Education. (2006, Revised). Missouri professional development guidelines for student success. Retrieved from http://www.sikeston.k12.mo.us/profdev/270484.pdf Mongiello, P., Brady, D., Johnson, G., & Berg, J. H. Cao, L.; Gorodetsky, V. Liu, J. Weiss, G ., & Yu, P. S. (2009). Strength training: I nstitutes pump up tea chers' roles as instructional leaders. Journal of Staff Development 30 (4), 20 24. Munoz, M., Prather, J., & Stronge, J. (2011). Exploring teacher effectiveness using hierarchical linear models: Student and classroom level predictors and cross year stabil ity in elementary school reading. Planning and Changing, 42 (3/4), 241 273. National Research Council and National Academy of Education. (2010). Getting value out of value added: Report of a w orkshop Washington, DC: Center for Education, Division of Behavi oral and Social Sciences and Education. National Board for Professional Teaching Standards. (2011). Getting it right : A comprehensive guide to developing and sustaining teacher evaluation and support system. Arlington, VA : National Board for Professional Teaching Standards National Center for Educational Statistics. (2013). [Website for National Center for Educational Statistics]. Retrieved from nces.ed.gov/ National Commission on Excellence in Education (1983). Nation at risk ( Report to the Nation and S ecretary of Education United States Department of Education ) Author.

PAGE 88

80 National Governors Association Center for Best Practices & Council of Chief State School Officers. (2010). Common Core State Standards. Washington, DC: Authors. The New Teacher Project. (2010 October 5 ). Teacher evaluation 2.0. Retrieved from http://tntp.org/ideas and innovations/view/teacher evaluation 2.0 The New Teacher Project [TNTP] (2012 July 30 ). The irreplaceables, understanding the real retention crisis in Americas' urban schools. Retrieved from http://tntp.org/ideas and innovations/view/the irreplaceables understanding the real retention crisis Owens, C. (2008). Leading without leaving the classroom. National Staff Development Council, 29 (3) 57 60. PBS (2013). Frontline: The testing industry's big four. Retrieved from http://www.pbs.org/wgbh/pages/frontline/shows/schools/testing/companies.html Peck, C. A., Gallucci, C., Sloan, T., & Lippincott, A. (2009). Organizational learning and program renewal in teacher education: A socio cultural theory of learning, innovation and change. Educational Research Review 4 (1), 16 25. Punch, K. (2009). Introduction to research methods in educatio n. London: Sage. Regional Education Laboratory [REL]. (2007). Reviewing the evidence on how teacher professional development affects student achievement. Educational Researcher, 33 (8), 3 15. Saldana, J. (2009). The c oding manual for qualitative researcher s Thousand Oaks, CA: Sage Publications.

PAGE 89

81 Sanders, W., & Rivers, J. (1996). Cumulative and residual effects of teachers on future student academic achievement (Research progress report). In University of Tennessee Value Added Assessment Center, Knoxville, TN. Retrieved March 28, 2001, from http://mdk12.org/practices/ensure/tva/ tva_2.html Slabine, M. A. (2012). 841 miles of square commitm ent. District wide plans makes professional learning a priority. Jour nal of Staff Development 33 22 27. Slavin, R. E., & Cheung, A. (2005). A s ynthesis o f research o n language o f reading instruction f or English Language Learners. Review of Educational Research 75 (2), 247 284. Smith, F. (1998). The book of learning and for getting New York: Teachers College Press. Smith, R., Johnson, M., &Thompson, K. (2012). Data, our gps. Education Leadership 69 (5), 56 59 State Council for Educator Effectiveness. (2011, April 13). Report and recommendations. Denver, CO: State Board of Education. Steele, J., Hamilton, L., & Stecher, B. (2010, December 1). Incorporating student performance measures into teacher evaluation systems Santa Monica, CA: Rand Corporation. Stein, S. J. (2004). The culture of education policy New York: Teachers College Press. Taylor E., & Tyler J. (2011). The effect of evaluation on performance: Evidence from longitudinal student achievement data of mid career teachers. Cambridge, MA: National Bureau of Economic Research.

PAGE 90

82 Tharp, R. G., Estrada, P., Dalton, S., & Yamauchi, L. (2000). Teaching transformed: A chieving excellence, fairness, inclusion, and harmony Boulder, CO : Westview Press. Thibodeau, G. (2011, June). A content literacy collaborative study group: High school teachers take charge of their professional learning Journal of Adolescent & Adult Literacy, 52 (1), 54 64. doi :10.1598/JAA.52.1.6 Thompson, L. (n.d.). Principles of data driven instruction. Retrieved from epd mh.com/leadership/pdfs/Principles_of_ Data_Driven_Instruction. pdf Torres, Z. (June, 2013). Disparities in D ouglas C ounty schools teacher evaluations draw fire. Denver Post ztorres@denverpost.com Vygotsky, L. S (1978). Mind in society: T he development of higher psychological processes Cambridge MA : Harvard University Press. Willam, D. (2012). Feedback: P art of a system. Ed ucational Leader ship 70 (1), 30 34. Warschauer, M. (1997). A sociocultural approach to literacy and its significance fo r In K 12. In J. Murphy & R. Sanders (Eds.), Nexus: The convergence of research & teaching through new information technologies (pp. 88 97). Durham NC : University of North Carolina. Wong, K. K., & Rothman, R. (2009). Clio at the table: U sing history to inform and improve education policy New York: Peter Lang. York Barr, J. & Duke, K. (2004). What do we know about teacher leadership? Findings from two decades of scholarship. Review of Educational Research Fall, 74, 255 316.

PAGE 91

83 APPENDIX A TEACHER LEADER SURVEY

PAGE 92

84

PAGE 93

85 APPENDIX B INTERVEIW PROTOCOLS Semi Structured Interview Protocol #1 Protocol for Director of Teacher Leadership Academy You are the director of the Teacher Leadership Academy in your school and in the district. I am going to ask you some questions about your role as a leader in your district, your b eliefs about professional development, your role in professional development, and your experience with the new teacher evaluation system. We will start with some general questions about your experience in schools 1. Tell me about your work as an educat or Probes: How long have you been in this role? Why did you seek this role? Where have you worked? What is your teaching experience? What are your responsibilities in this role? What are some of your beliefs about student learning? Teacher lear ning? Principal learning? Colorado and your school district are implementing a new educator effectiveness system to meet the requirements mandated by the educator effectiveness law. For this study, I am focusing on the teacher side of the system and th e next few questions are about educator's effectiveness, teacher learning, and the new evaluation system. 2. Explain the issue of teacher effectiveness from your perspective Probing questions: Why is educator effectiveness an initiative in your district? What role does the new evaluation system play in the educator effectiveness initiative? What actions should schools take to increase teacher effectiveness? What do you believe makes teachers more effective for students? How is teacher learning part of your thinking about increasing teacher effectiveness? How are educators in schools involved in the district decision making process around educator effectiveness? What sort of professional le arning can be designed to support teachers once they have been labeled? 3. Explain the issue of teacher effectiveness from your school district s point of view

PAGE 94

86 Probing questions: What are the reasons your school district is implementing a new educator effectiveness system? What actions is your district going to take to improve educator effectiveness? How is professional learning connected to educator effectiveness in your district? What action steps does your district/school need to take to meet thes e goals? 4. In your experience, what causes teachers to increase their effectiveness for all students? Now, I would like to move to your work in the Teacher Leadership Academy and your work with teacher leaders. 5. Tell me about the teacher leadership academy Probes: What value do you place on teacher leaders? How and why is the academy in existence? What are the goals of the academy? How do you recruit and retain teachers in the academy? How do teacher leaders help realize the vision for the district? What sort of training have you received around using teacher leaders? How do you cultivate relationships with teacher leaders? How do you remain connected with teachers who have left the academy? How do you assess the value the academy has had on teachers, students, and principals? How do you collaborate with principals about utilization of teacher leaders? How do you collabo rate with other district leaders and departments about the utilization of teacher leaders? Expert teachers, who are also building leaders, may play a role in helping all teachers to become more effective. 6. What roles and responsibilities do teach er leaders hold in schools? Probes: Are their roles and responsibilities you wish you could add for teacher leaders? Could remove? In your experience, how are teacher leaders being utilized to improve student and teacher learning in their buildings? I am wondering about your beliefs and practice in professional learning and how they connect to your work with teacher leaders and the learning experiences for the academy. 7. In your experience, what are some characteristics of quality professional learning? Probes: What is the best professional learning you have experienced? The worst? Does professional learning the academy embody these named characteristics?

PAGE 95

87 What do you believe about how adults learn? What causes teachers to change practice? How do you connect your beliefs about PD to your work in the teacher academy? 8. How do you plan professional learning in the teacher academy? Probes: Who do you collaborate with to plan for the teacher academy? What are the ways you use s tudent data to plan for professional learning? What are the ways you use district goals to plan for professional learning? How do you know if professional learning is working? How do you solicit and use feedback from teachers about professional learn ing? 9. Do you believe the new evaluation system will improve teacher effectiveness? Probes: How does the feedback on the rubric lead to teacher change? What sort of in building and district professional learning are in place to support teachers once they have been labeled? What role do you envision teacher leaders having to improve teacher effectiveness in their buildings? 10. What are the challenges of the educator effectiveness initiative? What are the strengths of the educator effectiveness initiative? 11. I want to talk to get your impressions about some data I collected from teacher leaders in your district. Conclusion: Thank you so much for your time and your thinking today. I will share the results of my study with you in the spring.

PAGE 96

88 Semi Structured Interview Protocol #2 Protocol for Director of Professional Learning /Director of Teacher Development You are the director of professional learning in the district. I am going to ask you some questions about your role as a leader in your district, your beliefs about professional development and teacher leaders, your role in professional development, and yo ur experience of the new teacher evaluation system. We will start with some general questions about your experience as an educator 1. Tell me about your work as an educator Probes: How long have you been in this role? Why did y ou seek this role? Wh ere have you worked? What is your teaching experience? What are your responsibilities in this role? What are some of your beliefs about student learnin g? Teacher learning? Principal learning? Colorado and your school district are implementing a new ed ucator effectiveness system to meet the requirements mandated by the educator effectiveness law. For this study, I am focusing on the teacher side of the system and the next few questions are about educator's effectiveness, teacher learning, and the new ev aluation system. 2. Explain the issue of teacher effectiveness from your perspective Probing questions: Why is educator effectiveness an initiative in your district? What role does the new evaluation system play in the educator effectiveness initiative? What actions should schools take to increase teacher effectiveness? What do you believe makes teachers more effective for students? How is teacher learning part of your thinking about increasing teacher effectiveness? How are educators in schools involved in the district decision making process around educator effectiveness? What sort of professional learning can be designed to supp o rt teachers once they have been labeled? 3. Explain the issue of teacher effectiveness from your school districts point of view Probing questions: What are the reasons your school district is implementing a new educator effectiveness system? What role does the new evaluation system play in the educator effectiveness initiative? What actions is your district going to take to improve educator effectiveness?

PAGE 97

89 How is professional learning connected to educator effectiveness in your district? What action steps does your district/school need to take to meet these goals? 4. In your experience, what causes teachers to increase their effectiveness for all students? Now I would like to focus on professional learning from a district point of view and how you connect to the work that occurs with teachers in schools. 5. Tell me about the professional learning structures in your district Probes: What are the goals of professional learning in the district? How does professional learning look in a site based district? How does professional learning help realize the vision for the district? How do you manage enhance teacher learning from the district level? How do you collaborate with principals about professional learning? How do you collaborate wit h other district lea ders and departments about the professional learning? How is professional learning connected to educator effectiveness? Expert teachers, who are also building leaders, may play a role in helping all teachers to become more effective. 6. What roles and responsibilities do teacher leaders hold in schools? Probes: What value do you place on teacher leaders? How are teacher leaders being used both at the district and building levels to increase educator effectiveness? The State Council on Educator Effectiveness advises that teachers need ongoing feedback and support needed to improve performance. I would like to m ove towards the support teachers get once they are labeled on the rubric. 7. In your experience, what are some characteristics of quality professional learning? Probes: What is the best professional learning you have experienced? The worst? Does professional learning in your district embody these named characteristics? What do you believe about how adults learn? What causes teachers to change practice? How do you connect your beliefs about PD to your work in the office of professional learni ng? 8. Do you believe the new evaluation system will improve teacher effectiveness? Probes:

PAGE 98

90 How does the feedback on the rubric lead to teacher change? What sort of building and district professional learning is in place to support teachers once they have been labeled? What role do you envision teacher leaders having to improve teacher effectiveness in their buildings? 9. What are the challenges of the educator effectiveness initiative? What are the strengths of the educator eff ectiveness initiative? 10 I want to talk to get your impressions about some data I collected from teacher leaders in your district. Conclusion: Thank you so much for your time and your thinking today. I will share the results of my study with you i n the spring.

PAGE 99

91 Semi Structured Interview Protocol #3 Protocol for Director of Educator Effectiveness / Peer Observer You are the director of Educator Effectiveness in the district. I am going to ask you some questions about your role as a leader in your district, your beliefs about professional development and teacher leaders, your role in professional development, and your experience of the new teacher evaluation system. We will start with some general questions about your experience as an educator 1. Tell me about your work as an educator Possible probing questions: How long have you been in this role? Why did y ou seek this role? Where have you worked? What is your teaching experience? What are your responsibilities in this role? What are some of your beliefs about student learnin g? Teacher learning? Principal learning? 2. Explain the issue of teacher effectiveness from your perspective Possible probing questions: Why is educator effectiveness an initiative in your district? W hat role does the new evaluation system play in the educator effectiveness initiative? What actions should schools take to increase teacher effectiveness? What do you believe makes teachers more effective for students? How is teacher learning part of yo ur thinking about increasing teacher effectiveness? How are educators in schools involved in the district decision making process around educator effectiveness? What sort of professional learning can be designed to support teachers once they have been la beled? 3. Explain the issue of teacher effectiveness from your school districts point of view Possible probing questions: What are the reasons your school district is implementing a new educator effectiveness system? What actions is your district going to take to improve educator effectiveness? How is professional learning connected to educator effectiveness in your district? What action steps does your district/school need to take to meet these goals? 4. In your experience, what causes teachers to increase their effectiveness for all students? 5. Tell me about the Educator Effectiveness structures in your district Possible probing questions:

PAGE 100

92 What are the goals of Educator Effectiveness in the district? How does Educator Effectiveness help realize the vision for the district? How do you manage Educator Effectiveness from the district level? How do you collaborate with principals about Ed ucator Effectiveness? How do you collaborate with other district lea ders and departments about the Educator Effectiveness? How is professional learning connected to Educator Effectiveness? The State Council on Educator Effectiveness advises that teach ers need ongoing feedback and support needed to improve performance. I would like to move towards the support teachers get once they are labeled on the rubric. 6. In your experience, what are some characteristics of quality professional learning? Possible probing questions: What is the best professional learning you have experienced? The worst? Does professional learning in your district embody these named characteristics? What do you believe about how adults learn? What causes teachers to cha nge practice? How do you connect your beliefs about PD to your w ork in the office of Educator Effectiveness? As you think increasing teacher effectiveness, I wonder about how the district might utilize expert teachers, who are also building leaders, to p rovide support post evaluation 7. What roles and responsibilities do teacher leaders hold in schools that help to increase educator effectiveness? Possible probing questions: What value do you place on teacher leaders in connection to Educator Effectiveness? How are teacher leaders being used both at the district and building levels to increase Educator Effectiveness? 8. How do you plan for professional learning connected to Educator Effectiveness in the district? Possible probing question s: How do you plan for professional learning connected to Educator Effectiveness? What role does student data hold in Educator Effectiveness? How do you know if the Educator Effectiveness structures are working? How do you solicit and use feedback fro m teachers about Educator Effectiveness? How do you solicit and use feedback from principals about Educator Effectiveness? For this study, I am focusing on the teacher side of the system and the next few questions are about how teacher evaluation affec ts an educator's effectiveness. You may refer to the evaluation rubric to answer these questions. 9. Do you believe the new evaluation system will improve teacher effectiveness?

PAGE 101

93 Probes: How does the feedback on the rubric lead to teacher change? What sort of building and district professional learning is in place to support teachers once they have been labeled? What role do you envision teacher leaders having to improve teacher effe ctiveness in their buildings? 10. What are the challenges of the educator effectiveness initiative? What are the strengths of the educator effectiveness initiative? 11. I want to talk to get your impressions about some data I collected from teacher lea ders in your district. Conclusion: Thank you so much for your time and your thinking today. I will share the results of my study with you in the spring.

PAGE 102

94 APPENDIX B Semi Structured Interview Protocol # 1 Protocol for Director of Teacher Leadership Academy You are the director of the Teacher Leadership Academy in your school and in the district. I am going to ask you some questions about your role as a leader in your district, your beliefs about professional development, your role in prof essional development, and your experience of the new teacher evaluation system. We will start with some general questions about your experience in schools 1. Tell me about your work as an educator Probes: How long have you been in this role? Why did you seek this role? Where have you worked? What is your teaching experience? What are your responsibilities in this role? What are some of your beliefs about student learning? Teacher learning? Principal learning? Colorado and your school di strict are implementing a new educator effectiveness system to meet the requirements mandated by the educator effectiveness law. For this study, I am focusing on the teacher side of the system and the next few questions are about educator's effectiveness, teacher learning, and the new evaluation system. 2. Explain the issue of teacher effectiveness from your perspective Probing questions: Why is educator effectiveness an initiative in your district? What role does the new evaluation system play in the educator effectiveness initiative? What actions should schools take to increase teacher effectiveness? What do you believe makes teachers more effective for students? How is teacher learning part of y our thinking about increasing teacher effectiveness? How are educators in schools involved in the district decision making process around educator effectiveness? What sort of professional learning can be designed to support teachers once they have been l abeled? 3. Explain the issue of teacher effectiveness from your school districts point of view

PAGE 103

95 Probing questions: What are the reasons your school district is implementing a new educator effectiveness system? What actions is your district going to take to improve educator effectiveness? How is professional learning connected to educator effectiveness in your district? What action steps does your district/school need to take to meet these goals? 4. In your experience, what causes teachers to increase their effectiveness for all students? Now, I would like to move to your work in the Teacher Leadership Academy and your work with teacher leaders. 5. Tell me about the teacher leadership academy Probes: What value do you place on teacher leaders? How and why is the academy in existence? What are the goals of the academy? How do you recruit and retain teachers in the academy? How do teacher leaders help realize the vision for the district? What sort of training have you received around using teacher leaders? How do you cultivate relationships with teacher leaders? How do you remain connected with teachers who have left the academy? How do you assess the value the academy has had on teachers, students, and principals? How do you colla borate with principals about utilization of teacher leaders? How do you collaborate with other district leaders and departments about the utilization of teacher leaders? Expert teachers, who are also building leaders, may play a role in helping all te achers to become more effective. 6. What roles and responsibilities do teacher leaders hold in schools? Probes: Are their roles and responsibilities you wish you could add for teacher leaders? Could remove? In your experience, how are teacher leaders being utilized to improve student and teacher learning in their buildings? I am wondering about your beliefs and practice in professional learning and how they connect to your work with teacher leaders and the learning experiences for the academy

PAGE 104

96 7. In your experience, what are some characteristics of quality professional learning? Probes: What is the best professional learning you have experienced? The worst? Does professional learning the academy embody these named characteristics? What do you believe about how adults learn? What causes teachers to change practice? How do you connect your beliefs about PD to your work in the teacher academy? 8. How do you plan professional learning in the teacher academy? Probes: Who do you collaborate with to plan for the teacher academy? What are the ways you use student data to plan for professional learning? What are the ways you use district goals to plan for pr ofessional learning? How do you know if professional learning is working? How do you solicit and use feedback from teachers about professional learning? 9 Do you believe the new evaluation system will improve teacher effectiveness? Probes: How does the feedback on the rubric lead to teacher change? What sort of in building and district professional le arning are in place to support teachers once they have been labeled? What role do you envision teacher leaders having to improve teache r effectiveness in their buildings? 10 What are the challenges of the educator effectiveness initiative? What are the strengths of the educator effectiveness initiative? Conclusion: Thank you so much for your time and your thinking today. I will s hare the results of my study with you in the spring.

PAGE 105

97 Semi Structured Interview Protocol #2 Protocol for Director of Professional Learning You are the director of professional learning in the district. I am going to ask you some questions about your role as a leader in your district, your beliefs about professional development and teacher leaders, your role in professional development, and yo ur experience of the new teacher evaluation system. We will start with some general questions about your experience as an educator 1. Tell me about your work as an educator Probes: How long have you been in this role? Why did you seek this role? Wh ere have you worked? What is your teaching experience? What are your responsibilities in this role? What are some of your beliefs about student learning? Teacher learning? Principal learning? Colorado and your school district are implementing a new educator effectiveness system to meet the requirements mandated by the educator effectiveness law. For this study, I am focusing on the teacher side of the system and the next few questions are about educator's effectiveness, teacher learning, and the new evaluation system. 2. Explain the issue of teacher effectiveness from your perspective Probing questions: Why is educator effectiveness an initiative in your district? What role does the new evaluation system play in the educator effectiveness initi ative? What actions should schools take to increase teacher effectiveness? What do you believe makes teachers more effective for students? How is teacher learning part of your thinking about increasing teacher effectiveness? How are educators in schools involved in the district decision making process around educator effectiveness? What sort of professional learning can be designed to support teachers once they have been labeled? 3. Explain the issue of teacher effectivene ss from your school districts point of view Probing questions:

PAGE 106

98 What are the reasons your school district is implementing a new educator effectiveness system? What role does the new evaluation system play in the educator effectiveness initiative? What ac tions is your district going to take to improve educator effectiveness? How is professional learning connected to educator effectiveness in your district? What action steps does your district/school need to take to meet these goals? 4. In your experience, what causes teachers to increase their effectiveness for all students? Now I would like to focus on professional learning from a district point of view and how you connect to the work that occurs with teachers in schools. 5. Tell me about the professional learning structures in your district Probes: What are the goals of professional learning in the district? How does professional learning look in a site based district? How does professional learning help realize the visi on for the district? How do you manage enhance teacher learning from the district level? How do you collaborate with principals about professional learning? How do you collaborate with other district leaders and departments about the professional learning? How is professional learning connected to educator effectiveness? Expert teachers, who are also building leaders, may play a role in helping all teachers to become more effective. 6. What roles and responsibilities do teach er leaders hold in schools? Probes: What value do you place on teacher leaders? How are teacher leaders being used both at the district and building levels to increase educator effectiveness? The State Council on Educator Effectiveness advises tha t teachers need ongoing feedback and support needed to improve performance. I would like to move towards the support teachers get once they are labeled on the rubric. 7. In your experience, what are some characteristics of quality professional learning?

PAGE 107

99 Probes: What is the best professional learning you have experienced? The worst? Does professional learning in your district embody these named characteristics? What do you believe about how adults learn? What causes teachers to change practice? How do you connect your beliefs about PD to your work in the office of professional learning? 8 Do you believe the new evaluation system will improve teacher effectiveness? Probes: How does the feedback on the rubric lead to teacher change? What sort of building and district professional learning is in place to support teachers once they have been labeled? What role do you envision teacher leaders having to improve teacher effectiveness in their buildings? 9 What are the challenges o f the educator effectiveness initiative? What are the strengths of the educator effectiveness initiative? Conclusion: Thank you so much for your time and your thinking today I will share the results of my study with you in the spring.

PAGE 108

100 Semi Stru ctured Interview Protocol # 3 Protocol for Director of Educator Effectiveness You are the director of Educator Effectiveness in the district. I am going to ask you some questions about your role as a leader in your district, your beliefs about professiona l development and teacher leaders, your role in professional development, and your experience of the new teacher evaluation system. We will start with some general questions about your experience as an educator 1. Tell me about your work as an educator Possible probing questions : How long have you been in this role? Why did you seek this role? Where have you worked? What is your teaching experience? What are your responsibilities in this role? What are some of your beliefs about student learnin g? Teacher learning? Principal learning? 2. Explain the issue of teacher effectiveness from your perspective Possible probing questions : Why is educator effectiveness an initiative in your district? What role does the new evaluation system play in the educator effectiveness initiative? What actions should schools take to increase teacher effectiveness? What do you believe makes teachers more effective for students? How is teacher learning part of y our thinking about increasing teacher effectiveness? How are educators in schools involved in the district decision making process around educator effectiveness? What sort of professional learning can be designed to support teachers once they have been labeled? 3. Explain the issue of teacher effectiveness from your school districts point of view Possible probing questions : What are the reasons your school district is implementing a new educator effectiveness system? What actions is your district go ing to take to improve educator effectiveness? How is professional learning connected to educator effectiveness in your district? What action steps does your district/school need to take to meet these goals? 4. In your experience, what causes teachers t o increase their effectiveness for all students?

PAGE 109

101 5. Tell me about the Educator Effectiveness structures in your district Possible probing questions : What are the goals of Educator Effectiveness in the district? How does Educator Effectiveness help realize the vision for the district? How do you manage Educator Effectiveness from the district level? How do you collaborate with principals about Educator Effectiveness? How do you collaborate with other district leaders and departments about the Educator Effectiveness? How is professional learning connected to Educator Effectiveness? The State Council on Educator Effectiveness advises that teachers need ongoing feedback and support needed to improve perform ance. I would like to move towards the support teachers get once they are labeled on the rubric. 6. In your experience, what are some characteristics of quality professional learning? Possible probing questions : What is the best professional lea rning you have experienced? The worst? Does professional learning in your district embody these named characteristics? What do you believe about how adults learn? What causes teachers to change practice? How do you connect your beliefs about PD to yo ur work in the office of Educator Effectiveness? As you think increasing teacher effectiveness, I wonder about how the district might utilize expert teachers, who are also building leaders, to provide support post evaluation 7. What roles and respons ibilities do teacher leaders hold in schools that help to increase educator effectiveness? Possible probing questions : What value do you place on teacher leaders in connection to Educator Effectiveness? How are teacher leaders being used both at the district and building levels to increase Educator Effectiveness? 8. How do you plan for professional learning connected to Educator Effectiveness in the district? Possible probing questions : How do you plan for professional learning connected to Educator Effectiveness?

PAGE 110

102 What role does student data hold in Educator Effectiveness? How do you know if the Educator Effectiveness structures are working? How do you solicit and use feedback from teachers about Educator Effectiveness? How do you soli cit and use feedback from principals about Educator Effectiveness? For this study, I am focusing on the teacher side of the system and the next few questions are about how teacher evaluation affects an educator's effectiveness. You may refer to the evalu ation rubric to answer these questions. 9 Do you believe the new evaluation system will improve teacher effectiveness? Probes: How does the feedback on the rubric lead to teacher change? What sort of building and district professional lea rning is in place to support teachers once they have been labeled? What role do you envision teacher leaders having to improve teacher effectiveness in their buildings? 10 What are the challenges of the educator effectiveness initiative? What are th e strengths of the educator effectiveness initiative? Conclusion: Thank you so much for your time and your thinking today. I will share the results of my study with you in the spring.