Citation

## Material Information

Title:
Perceptions of faculty development practices and structures that influence teaching at high performance colleges and universities
Creator:
Bates, Barbara A
Place of Publication:
Denver, CO
Publisher:
Publication Date:
Language:
English
Physical Description:
xiii, 232 leaves : ; 28 cm.

## Subjects

Subjects / Keywords:
College teachers -- In-service training -- United States ( lcsh )
Universities and colleges -- Faculty -- United States ( lcsh )
Career development -- United States ( lcsh )
Career development ( fast )
College teachers -- In-service training ( fast )
Universities and colleges -- Faculty ( fast )
United States ( fast )
Genre:
bibliography ( marcgt )
theses ( marcgt )
non-fiction ( marcgt )

## Notes

Thesis:
Bibliography:
Includes bibliographical references (leaves 225-232).
General Note:
School of Education and Human Development
Statement of Responsibility:
by Barbara A. Bates.

## Record Information

Source Institution:
Holding Location:
|Auraria Library
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
707477358 ( OCLC )
ocn707477358

Full Text

PAGE 1

PERCEPTIONS OF FACULTY DEVELOPMENT PRACTICES AND STRUCTURES THAT INFLUENCE TEACHING AT HIGH PERFORMANCE COLLEGES AND UNIVERSITIES by Barbara A. Bates BS, University of Wisconsin, Madison, 1969 MA, University of Colorado Denver, 2000 A thesis submitted to the University of Colorado Denver in partial fulfillment of the requirements for the degree of Doctor of Philosophy Education Leadership and Innovation 2010

PAGE 2

PAGE 3

This thesis for the Doctor of Philosophy degree by Barbara A. Bates has been approved by Ellen Stevens Laura Border Date

PAGE 4

Perceptions of Faculty Development Practices and Structures that Influence Teaching at High Performance Colleges and Universities Thesis directed by Associate Professor Ellen Stevens, Program Advisor ABSTRACT Perceptions of faculty development practices and structures were compared between 13 high performing schools identified in the DEEP study (Kuh, Kinzie, Schuh, Whitt, 2005) to clarify the relationship between faculty development variables and effective teaching. A phenomenological design was employed to triangulate quantitative and qualitative data including faculty development director surveys, interviews, administrator surveys, and faculty development program websites. The data provided a framework to compare similarities and differences among schools and the related faculty development literature. Analysis of the director interviews revealed the practices and structures of their programs and activities. The results included descriptions of themselves, their programs, their institution, and their perceptions of how the faculty development program impacted effective teaching. A different set of faculty development directors from the same DEEP cohort, participated in a survey that asked them to rate their use of specific faculty development practices and structures. Administrators of the participating interviewees completed a survey that asked for the degree of administrative support

PAGE 5

for faculty development, their perceptions of the effectiveness of the faculty development program on their campus, their attitudes about effective teaching, and their perceptions of the factors that helped promote effective teaching on their campus. Faculty development director reports from surveys and interviews were compared and contrasted within this cohort and to best practices literature. Results indicated that elements of the faculty development variables included activities and programs, accountability practices, goals, institutional communication, reporting structure, the location of the faculty development office, faculty involvement, and relationships between the faculty development program and other institutional units. The faculty development variables that emerged from these data perceived to be most relevant to success included: extensive use of accountability measures; strong involvement of faculty in the planning and deployment of faculty development programs and activities; strong and mutually supportive relationships among faculty, faculty developers, and administration; establishment of priority faculty development goals for student success and improvement of campus climate for effective teaching. While the initial theoretical framework focused on how faculty developers affected faculty and facilitated their effective approaches to teaching, results of this dissertation point to a more complex interdependent set of relationships that

PAGE 6

together impacted effective teaching approaches. Faculty developers and faculty worked together in symbiotic relationships to construct a climate of respect for teaching and teaching excellence that resulted in more effective teaching and leaming opportunities at these high performing schools. These high performing schools serve as a model of how the symbiotic relationship of administration, faculty and faculty developers influenced and were influenced by each other. Working together they created an institutional climate that welcomed and supported efforts toward change and toward effective educational practices at high performing schools.

PAGE 7

DEDICATION This dissertation is dedicated to my husband, Steve Bates, without whose constant support and loving encouragement, I could not have completed this life goal.

PAGE 8

ACKNOWLEDGEMENT I am so grateful for the unwavering support, guidance, and encouragement of my advisor, Ellen Stevens. She helped me find the way through many difficulties, disappointments, and setbacks in the arduous dissertation process. Her expertise in faculty development and in the dissertation landscape was invaluable in the completion of this intellectual and personal journey. I am thankful to my lab partners without whose perceptive insights, I would have floundered. I am especially grateful for Michael Wray's support and expert guidance through the maze of details required for successful completion of this dissertation. His insights and careful attention to detail provided a framework for me to bring this research to the finish line. I want to acknowledge all the excellent teachers whose dedication to opening doors for others inspired me, in midlife, to make a career change and join their ranks. I am thrilled and delighted to be a part of this outstanding profession whose work makes a difference in lives of students, every day. And too, I am so grateful to the faculty development directors in this dissertation who so willingly gave of their time and expertise to share their stories and perceptions. They have inspired me to be a beacon for faculty trying to be the best that they can be to change the world within their institutions and beyond.

PAGE 9

TABLE OF CONTENTS TABLES ................................................................................................ Xlll CHAPTER I. INTRODUCTION .................................................................................... 1 Purpose of the Study ................................................................................ I General Problem and Background ........................................................... 3 Changing Environment in Higher Education ..................................... 2 The Changing Environment's Impact on Faculty and Faculty Developers .......................................................................................... 4 Factors Influencing Student Success in College ................................ 5 Theoretical Framework ............................................................................ 6 Approaches to Teaching ..................................................................... 6 Faculty Development Practices that Impact Effective Approaches To Teaching ....................................................................................... 8 Relationship between Organization and Faculty Development.. ..... 10 Influence of Faculty approach to Teaching on Student Success in College ............................................................................................. 11 Relationship Between Organizational Factors and Student Success In College ......................................................................................... l4 Research Questions ................................................................................ 15 Implications ............................................................................................ 16 Methodology .......................................................................................... 16 Structure ofthe Dissertation ................................................................... 18 2. REVIEW OF THE LITERATURE .......................................................... 20 Introduction ............................................................................................ 20 Historical Context and Early Trends in Faculty Development .............. 20 Scope of this Research and the Literature Review ................................. 26 Key concepts and Tenns ........................................................................ 27 Underlying Factors Impacting the Variables in this Study .................... 29 Changing Environment in Higher Education ................................... 29 Impact of Changing Environment on Faculty and Faculty Development .................................................................................... 32 Changing Role of Faculty Development in Higher Education ........ 33 Changing Expectations Affect Student Success in College ............. 34 IX

PAGE 10

Research Variables and Their Interactions ............................................ 36 Best Practice in Faculty Development: Summaries and Recommendations ............................................................................ 36 Planning ........................................................................................ 3 7 Evidence-Based Measurement and Feedback .............................. 38 Development Activities' Criteria for Success ............................. .41 Support and Recognition for Faculty ............................................ 43 Relationship between Organization and Faculty Development.. ..... 45 Climate and Culture ...................................................................... 47 Collaboration ................................................................................ 48 Evidence-Based Practice .............................................................. 49 Appropriate Funding for Faculty Development ........................... 50 Focus on Improvement ................................................................. 51 Continuous Assessment, Feedback, Analysis, and Adjustment ... 52 Approaches to Teaching that Affect Student Learning Outcomes ... 53 Teacher Conceptions of Learning ................................................. 53 Teacher Characteristics and Beliefs ............................................. 54 Student Perceptions of Exemplary Teachers ................................ 55 Teaching Behaviors Connected to Student Persistence and Cognitive Growth ......................................................................... 56 Relationship between Organizational Context and Student Success In College ..................................................................................... 59 3. METHODS .............................................................................................. 66 Introduction ............................................................................................ 66 Theoretical Framework .......................................................................... 66 Design ................................................................................................. 68 Participants ............................................................................................. 74 Procedures .............................................................................................. 7 8 Data Collection ................................................................................. 80 Measurement Tools-Interview Protocol ........................................ 81 Data Analysis ................................................................................... 83 Methods Summary ................................................................................. 85 4. RESULTS ................................................................................................ 87 Overview ................................................................................................ 87 Summary of Method .............................................................................. 88 Program Descriptions ............................................................................. 90 Faculty Development Directors' Online Survey Results ....................... 92 X

PAGE 11

Faculty Development Program Goals ........................................... 93 Practices Used in Faculty Development Programs ....................... 98 Characteristics of Faculty Development Activities .................... I 00 Assessment ................................................................................. I 02 Director Perceptions of Support from Administration ............... I 06 Online Survey Responses-Collaboration ................................. l 07 Administrators' Online Survey Responses .................................... 110 Faculty Development Directors' Interview Responses .................. 116 Goals ........................................................................................... Il7 Goals Connected to Student Performance ............................. Il7 Goals Aligned with Curriculum and Institutional Mission .... 119 F acuity Involvement and Ownership .......................................... I23 Engaging Activities .................................................................... I26 Measuring the Effectiveness of Faculty Development Activities and Progran1s .............................................................................. 132 Administration Support .............................................................. 13 7 Access to Administration ............................................................ 140 Cross-Campus Connections to Support Mission and Goals ....... 141 Perceived Impact of Faculty Development Program on Approaches to Teaching ............................................................. 146 Community Building and Developing a Culture of Teaching .... 147 Facilitating Effective Pedagogy .................................................. 151 Motivating Faculty ..................................................................... 153 Summary of Findings from Online and Interview Data ................. 154 5. DISCUSSION ........................................................................................ 161 Summary and Interpretation ................................................................. 161 Questions I & 2 .............................................................................. 163 Structures .................................................................................... 164 Practices ...................................................................................... 166 Goals ........................................................................................... 166 Activities ..................................................................................... 170 Accountability Practices ............................................................. 173 Institutional Communication and Relationships ......................... 178 Question 3 ...................................................................................... 186 Question 4 ...................................................................................... 190 Implications of Findings ...................................................................... 195 Limitations of the Study ....................................................................... 199 Implications for Further Research ........................................................ 202 Conclusion ............................................................................................ 202 Xl

PAGE 12

APPENDIX A. INTERVIEW PROTOCOL FOR FACULTY DEVELOPMENT DIRECTORS ........................................................................................ 205 B. SURVEY QUESTIONS FACULTY DEVELOPMENT DIRECTORS ........................................................................................ 208 C. SURVEY QUESTIONS ADMINISTRATOR ..................................... 222 REFERENCES ............................................................................................... 225 Xll

PAGE 13

LIST OF TABLES Table 3.1 Data collection strategies ............................................................................. 69 3.2 Faculty development and institutional demographics phone interview ....... 72 3.3 Faculty development demographics from online survey ............................. 73 4.1 Faculty development program demographics (phone interview participants) ......... 85 4.2 Importance of program goals to the faculty development program ......................... 88 4.3 Factors influencing achievement of the faculty development program goals ......... 90 4.4 Obstacle hindrance to achievement of faculty development goals .......................... 92 4.5 Importance of these practices to the success of the faculty development program .93 4.6 Importance of characteristics that contribute to success of faculty development Activities ............................................................................................................. 95 4. 7 Faculty developer use of assessment strategies to measure effectiveness of their programs ............................................................................................................. 97 4.8 Commonly measured outcomes to determine effectiveness of faculty development activities ............................................................................................................. 99 4. 9 Examples of administrative support for faculty development programs ............... 1 01 4.10 Importance of collaborative practices to the success of faculty development Program ........................................................................................................... 1 03 4.11 Importance of collaboration to success of faculty development activities 104 4.12 School administrators' perceptions of teaching and faculty development ............ 1 06 4.13 Use of required hiring practices related to teaching ................................... 107 4.14 Contribution of factors to teaching effectiveness on their campus ............. 109 Xlll

PAGE 14

CHAPTER 1 INTRODUCTION Purpose of the Study As post-secondary institutions step up their efforts to meet society's increasing expectations for student success, it becomes critical to clarify the antecedents to successful student outcomes. While teachers are central to student performance, it remains unclear how they gain the necessary pedagogical expertise. One factor in improving teaching expertise is faculty development. The purpose of this study is to 1) examine faculty development practices and structures at institutions with higher-than-predicted student outcomes, 2) compare their practices and structures to those recommended in "best practice" literature, and 3) explore the relationship between specific faculty development practices and structures and faculty approaches to teaching that contribute to student success at participating institutions. Examples of faculty development practices to be examined include activities and programs offered, accountability practices, goals, and institutional communication. Examples of faculty development structures include the program reporting structure, the location of the faculty development office, faculty involvement in the design and delivery of programs, and the relationship between the faculty development program and other institutional units.

PAGE 15

General Problem and Background There is a body of research connecting specific faculty development practices to positive changes in faculty beliefs and successful teaching practices (Gibbs, 2003; Rust, 1999). But much of it does not use evidence sufficiently strong to be conclusive (Menges, 1980; Weimer & Lenze, 1989). There is also a body of research connecting improved student learning, satisfaction, and persistence in higher education with specific approaches to teaching, including attitudes, beliefs, and pedagogical practices (Hativa & Goodyear, 2002; Kember & Gow, 1994; Pascarella & Terrenzini, 2005). However, there is little if any research that examines the relationship between specific faculty development practices and structures and approaches to teaching that lead to successful student outcomes. If faculty developers are to have a viable role in successful higher education programs, it would be helpful for them to know, specifically, what practices are likely to have an impact on approaches to teaching that contribute to student success. This study attempts to examine and clarify those connections through an investigation of faculty development practices at colleges with demonstrated higher-than-predicted student success. Changing Environment in Higher Education For the past thirty years, higher education has experienced calls for reform to better meet the needs of a changing environment; however, the scope and speed of change today has increased the urgency of calls for institutional reform 2

PAGE 16

(American Association of Colleges & Universities [AAC&U], 2002). Too many students are graduating without the requisite skills and knowledge to become successful employees, leaders, or citizens in today's environment (AAC&U). There are also too many students dropping out or under-performing in their studies costing themselves and the institution thousands of dollars; institutions commonly report five-year graduation rates as low as 40% (ACT, 2006; Gardiner, 1994). These problems are indicators of an educational system that isn't equipped to meet the demands of society's mandate ofnear-universal post-secondary education for its citizens. Current pressures on higher education include a growing and more diverse student body; increased demand for accountability of student outcomes; more stringent accreditation standards; new enrollment patterns; impacts of technology; workplaces that demand a highly educated citizenry; and a rapidly evolving understanding of the teaching and learning process (AAC&U, 2002; Diamond & Adam, 2002; Gardiner, 1994). The changing environment in higher education has stimulated subsequent changes in every aspect of institutions, among them, the purpose of education, the teaching learning paradigm, teacher roles and expectations, and faculty development programs (Lick, 2002). Lick quotes a special report by the American Association of University Professors [AAUP] (1999): The world of higher education is in the midst of accelerating and sometimes turbulent change. New modes of communication are profoundly affecting the work of faculty members: they are reshaping the process of teaching and 3

PAGE 17

learning, redefining the role and authority of faculty members in organizing and overseeing the curriculum, and altering the bases for evaluating student and faculty performance. (p. 31) The purpose of higher education has changed from providing instruction to producing learning; the consumers of higher education have changed from the successful elite to the large majority of high school graduates; the process of education has changed from primarily lecture to whatever means works best (Barr & Tagg, 1995). The panel of national experts in post secondary education who drafted the "Greater Expectations" report (AAC&U, 2002) concluded, "It is society's responsibility to ensure all students powerful learning that prepares them not only for a job, but for career advancement and a fulfilling life as well" (p. 3 ). To that end they recommend a shift from old educational principles to ones that include a new set of faculty beliefs, attitudes, practices, and expectations (AAC&U, 2002). These significant and pervasive changes facing the academy are, however, colliding with an academic culture that has evolved over generations to provide stability and protect the status quo; faculty, as the front line providers of post-secondary education, are in the middle of this turbulence (Lick, 2002). The Changing Environment's Impact on F acuity and F acuity Developers The shift from a teaching to a learning paradigm also affects teaching goals and strategies and thus changes not only faculty roles and practices but also those of students, college administrators, and faculty developers. Instead of merely being prepared to "teach students," today's faculty and staff have the added challenge of 4

PAGE 18

structuring a learning environment to facilitate student learning for the majority of students and assessing that learning with credible tools (Gardiner, 1994). This change demands a much more sophisticated understanding ofthe teaching and learning process from faculty and administrators than was previously required. While there exists a strong body of evidence about the new learning paradigm and how to function successfully within it (Kuh, Kinzie, Schuh, & Whitt, 2005; Lick, 2002; Tagg, 2005), faculty may have little awareness of or access to resources to help them understand and respond to these new demands. Faculty developers, therefore, can be a key bridge to this new paradigm for faculty who need to adapt to the changing environment and make the shift. The next section examines research in faculty development to help us discern areas of consensus and elements of quality in faculty and professional development programs. Factors IJ?fluencing Student Success in College In order to understand the important contribution of faculty development to the successful performance of higher education institutions, specifically student learning and persistence, it is necessary to discuss the complex factors and interactions affecting student success in higher education. Although student prior academic achievement is a strong predictor of their success in college, the emphasis of this study is on factors that the institution can intluence that contribute to student success, such as faculty approaches to teaching. 5

PAGE 19

While organizational factors make a significant contribution to student success, this research study focuses on variables related to faculty development. The factors discussed here include the changing environment in higher education and its effect on teachers and faculty development practices; organizational impacts on faculty development programs and student success; how faculty approaches to teaching contribute to student success; and faculty development impacts on approaches to teaching as well as on organizational context. Theoretical framework The previous discussion examined the underlying factors that have an important influence on student success in college, such as the changing environment in higher education and its effect on faculty. However, the following, interdependent factors have a more immediate impact on student success: organizational context, which affects both faculty development and faculty approaches to teaching; faculty approaches to teaching; and faculty development, which is affected by organizational context and has an impact on approaches to teaching. The next section will introduce these interdependent variables that are the focus of my research as well as the interaction among them. Relationship between Faculty Development and Approaches to Teaching Since the mid-twentieth century, faculty development has grown in response to increasing societal and institutional demands for more effective teaching and education practices. Most of the literature describes the goal of faculty 6

PAGE 20

development as trying to improve teaching through a variety of means aimed at enhancing teachers' motivation to adopt teaching approaches that lead to improved student outcomes (Bergquist & Phillips, 1979; Gardiner, 1994; Gibbs, 2003; Menges, 1980; Weimer & Lenze, 1989). But it remains unclear how that relationship improves teaching in ways that matter to student success. Best practice in faculty development is, ostensibly, that which produces the changes in teaching approaches that lead to student success. However, difficulties in measuring those outcomes have resulted in recommendations for best practice that are not substantiated by strong empirical evidence. The early literature described a focus on changing inputs in order to improve teaching: admit brighter students, hire new faculty from the best graduate schools, decrease the student-faculty ratio, and buy new instructional hardware (Bergquist & Phillips, 1979). However, the authors concluded that these efforts alone did not have an impact on teaching practice. Attempting to elucidate best practices in faculty development, other researchers focused on providing learning opportunities for faculty to improve their pedagogical skills. Development activities such as workshops, seminars, consultations, use of resource materials, peer assistance, grants, and leaves were evaluated for their impact on teaching effectiveness (Levinson-Rose & Menges, 1981; Weimer & Lenze, 1989). These research efforts concluded that most of the studies that were analyzed merely looked at participant satisfaction as indicators of 7

PAGE 21

success rather than actual changes in teaching behavior or student learning, which would be clearer indicators of successful faculty development programs. Faculty Development Practices that Impact Effective Approaches to Teaching Studies that were sufficiently evidence-based to be valid were mostly of faculty development programs for teaching assistants and new faculty orientations (Levinson-Rose, & Menges, 1981 ). Weimer and Lenze ( 1989) found no evidence for the effectiveness of grants, leaves, or short workshops on instructional improvement. There was support, however, for the effectiveness of faculty development consultation especially in conjunction with lower-than-expected student ratings. There also appeared to be reasonable support for longer workshops particularly when time for practice and immediate feedback were included to help guide faculty skill acquisition (Weimer & Lenze). In general, most of the more current faculty development literature continues to cite faculty satisfaction as the primary criteria used to measure program effectiveness (Helyer & Boschmann, 1993; King & Lawler, 2003; Murray, 2000; Sorcinelli, 2001 ). In an attempt to find more substantiated measures of faculty development effectiveness, two recent studies went beyond satisfaction measures and faculty developers' perceptions. Rust ( 1999) measured the degree of intended and actual change in faculty participating in workshops but this research still relied on self-reports without corroborating evidence. Gibbs (2003), however, 8

PAGE 22

used multiple measures of success including faculty and student change determined by self-reports, outside observers, and performance indicators. Although recommendations for effective faculty development practice from the literature are based on somewhat weak evidence, a number of researchers cite a common set of practices that lead to successful faculty development programs (Gibbs, 2003; Hellyer & Boschmann, 1993; King & Lawler, 2003; Levinsosn-Rose & Menges, 1981; Rust, 1999; Sorcinelli, 2001 ). Agreement of findings in studies from various decades and research contexts provides a stronger rationale for '"best practice" than any of the studies alone can give. Those faculty development practices found to make a positive impact on effective approaches to teaching include: faculty ownership and community building with support and rewards for the change process; outcome measurements from a variety of sources integrated into the faculty development program; development activities modeled on participant engagement and learner focus which address the full range of faculty roles; and the alignment of faculty development goals with institutional mission (Gibbs, 2003; Hellyer & Boschmann, 1993; King & Lawler, 2003; Levinsosn-Rose & Menges, 1981; Rust, 1999; Sorcinelli, 2001 ). This research study I) compares and contrasts the recommendations for best practice, discussed above, with best practices as perceived by directors of faculty development programs at successful higher education institutions, and 2) explores the relationship of the perceived best practices at DEEP (Documenting Effective 9

PAGE 23

Educational Practice, Kuh et al., 2005) schools to teaching approaches that contribute to student success. Relationship Between Organization and Faculty Development While the discussion thus far has focused on evaluating the effectiveness of specific faculty development practices, there is a body of literature that examines the impact of organizational elements on faculty development (Allen, 2003; Astin, 1980; Cowan, George, & PinheiroTorres, 2004; Diamond, Gardiner, & Wheeler, 2002; Eckel, Hill, Green, & Mallon, 1999; Gardiner, 1994, 2005; Kuh et al., 2005; Saroyan & Amundsen, 2004; Sorcinelli, 2001 ). It is to this research that I now tum for a more complete understanding of factors affecting faculty development practices and structures. F acuity developers are acutely aware of the relationship between organizational context and the success of their programs (Frantz, Beebe, Horvath, Canales, & Swee, 2005). For example, a campus climate emphasis on research, rather than teaching, can have a significant impact on faculty receptivity to faculty development programs that focus on instructional improvement. Likewise, a reward structure that values research over teaching can undermine the efforts of the faculty development program to improve teaching. Therefore, any discussion of effective faculty development is incomplete without an understanding of how organizational elements affect and impact faculty development. 10

PAGE 24

In addition to factors of campus climate and culture, research has found that other factors can help or hinder faculty development efforts. Sufficient and appropriate funding, cross-campus collaborative planning, the use of evidence based practice to guide planning and change initiatives, and a school-wide focus on improvement can make a positive contribution to a successful faculty development program. If these elements are missing, even well conceived programs may fail (Diamond et al., 2002; Gardiner, 1994, 2005; Kuh et al., 2005; Saroyan & Amundsen, 2004; Sorcinelli, 2001 ). Therefore, this research study included questions about these factors salient to effective faculty development. The Influence of Faculty Approach to Teaching on Student Success in College Another of the complex factors affecting student outcomes in higher education is the impact the faculty member's approach to teaching has on student learning and persistence. It may seem obvious that what a teacher does in the classroom and expects of the students directly affects what students do and how well they learn. However, it has only been in the last fifteen years that empirical evidence has corroborated this common-sense assumption. One area of research connects teachers' conceptions of learning and teaching with student learning outcomes. Kember and Gow ( 1994) contrast a "facilitation of learning" approach with a "transmission of knowledge" approach. The former assumes that students need to construct the meaning of what is learned and need supports from faculty and context to do so. The latter assumes teachers 11

PAGE 25

can tell the information and students, by listening and taking notes, will learn it. Teachers who value a "facilitation of learning" approach to teaching correlate positively with students' use of more successful studying practices such as deep and achievement approaches to learning. Deep learning is characterized by student's genuine need to engage the learning task appropriately and meaningfully and to understand the deeper, comprehensive principles and applications (Biggs, 1999). Conversely, teachers who value a "learning transmission" view of teaching are correlated positively with students' use of less successful studying practices that result in surface learning. A surface learning approach is characterized by students' intention to get the task out of the way with minimum effort while appearing to meet the requirements (Biggs, 1999). While a surface approach to learning may describe much of today's higher education, the deep approach is the goal sought by most educators and participants in colleges and universities. A second area of research investigates approaches to teaching that reduce classroom effectiveness as measured by observation, student feedback, and student learning (Hativa, 2002). Approaches to teaching associated with poor teaching include an ineffective communication style, non-linear thinking, and insecure behavior. Beliefs and misconceptions about teaching and students may also be barriers to effective teaching; these include negative ideas about the instructor's 12

PAGE 26

responsibility for student learning, the instructor's ability to adapt instruction, and the ability of students to appreciate instruction (2002). A third body of research investigates the connection between approaches to teaching, student perceptions of exemplary teachers, and higher student achievement (Feldman, 1998). Ranking high on lists of teaching approaches associated with higher student ratings and higher student achievement are teacher clarity, preparation, and organization; the ability to motivate students to do their best; and the stimulation of interest in the course. A final body of evidence regarding the impact of approaches to teaching on improved student learning analyzed 2600 studies to find common variables that affect student learning and growth (Pascarella & Terrenzini, 2005). Among other findings, teacher clarity, preparation, organization, helpfulness, as well as concern for and rapport with students had significant and positive correlations with student mastery of content. Active, cooperative, and constructivist pedagogical approaches also were found to improve student learning. In addition, many of the same approaches to teaching were positively correlated with student persistence and intent to continue their education. The good news for faculty developers is that the majority of approaches to teaching most highly correlated with improved student success are learnable. The factors affecting student success in college are no longer a mystery; however, the challenge remains for faculty developers to communicate this information in a way 13

PAGE 27

that faculty are motivated to learn and which subsequently facilitates their adoption of more effective teaching approaches. Relationship between organizationalfactors and student success in co11ege In order to distinguish the contribution of faculty development to student success from those of organizational factors, this section discusses the relationship between organization and student success. One of the most detailed and valid measures of the impact of specific organizational practices on student learning and retention is the work of Kuh et al.(2005) as part of the project, Determining Effective Educational Practices (DEEP). They examined the evidence from over 700 institutions using the National Survey of Student Engagement (NSSE), an instrument that measures student learning and persistence during the freshman and senior years to determine which colleges and universities provided a value-added educational experience for their students (Edgerton, 2005). That is, they determined which schools had higher scores on the NSSE survey and higher graduation rates than would be predicted from their students' educational background and demographic data. Of the schools who met these criteria, 20 were selected for more detailed analysis of their institutional practices. These became the DEEP schools. These effective institutions were then analyzed through surveys and multiple, in-depth interviews, to discern common practices contributing to their success. The organizational factors contributing to student success include: a 14

PAGE 28

"living" mission, a pervasive focus on student learning; the use of space and location for educational enrichment; clear policies, procedures, and systems for student success; an improvement-oriented ethos; and a campus-wide, shared responsibility for student success. In spite of the detailed analysis of these institutions, it remained unclear how their faculty development programs contributed to the successful student outcomes. Did those faculty development programs contribute to more effective approaches to teaching? Did they influence teachers' attitudes by helping them cope with the multiple demands of the environment? My research sought to clarify why teachers at these institutions were so effective and what role faculty development played in contributing to their effectiveness. Research Questions While there is good evidence for 1) what students need to be doing and thinking in order to succeed in college (Edgerton, 2005), 2) how approaches to teaching facilitate or impede student success (Hativa, 2002; Kember & Gow, 1994; Kuh eta!., 2004; Pascarella & Terrenzini, 2005), and 3) how organizational context creates the conditions that matter to student success (Kuh et al., 2005), there exists less robust evidence for how faculty development leads to more effective teaching approaches (Frantz et al., 2005; Gibbs, 2003; Hellyer & Boschmann, 1993; King & Lawler, 2003; Menges, 1980; Roche, 2001 ; Weimer & Lenze, 1989), and scant literature investigating which elements of a faculty development program are 15

PAGE 29

effective in changing approaches to teaching in ways that contribute to improved student learning and persistence. My research addressed this challenging question. Specifically, in my study, I examined the following questions: I. What are the common structures and practices among faculty development programs at institutions that produce better than predicted student outcomes (DEEP schools)? 2. How congruent are the faculty development structures and practices of DEEP programs with best practice as described in the literature? 3. What are the elements of faculty development practice at DEEP schools that differ from best practice as described in the literature? 4. What is the perceived influence of faculty development best practice at DEEP schools on faculty approaches to teaching that contribute to student success? Implications The findings of this research have implications not only for faculty developers but also for those in higher education attempting to meet the demands of the environment within the constraints oftoday's funding and societal expectations. Faculty developers are ready and willing to add their expertise to the organizational drive for excellence; however, unless we have more complete information about what is effective and why, efforts may be squandered, or worse, counterproductive. This research provides clear guidance about where and how to invest scarce 16

PAGE 30

resources in order to help achieve higher education's goal of improved student learning and persistence. Methodology This was an exploratory study designed to develop a clearer understanding of the specific factors in faculty development programs at successful higher education institutions that influenced approaches to teaching which contribute to student success. I conducted phone interviews with a sample of 13 directors of faculty development programs at the 20 DEEP schools to gain detailed information about the practices and structural components of their programs. These interviews were recorded, transcribed, coded, and analyzed using qualitative procedures and NVivo qualitative software. An online survey of 12 faculty development directors from the DEEP schools was conducted to answer demographic questions about their budget, staff, and percent of director time devoted to faculty development as well as programmatic quantitative data such as ranking of best practices and goals described in previous faculty development literature. These 12 directors responded anonymously so the degree of overlap between participants in the online survey and phone interview is unknown. DEEP participants have demonstrated by their scores and through the DEEP study process that they share a set of successful institutional practices that contribute to their outstanding levels of student success (Kuh et al., 2005). By selecting the DEEP schools for study, I chose to limit this research to the 17

PAGE 31

contribution of faculty development to faculty approaches to teaching that lead to student success. The DEEP study demonstrated the organizational contexts that contribute to student success; therefore, I did not plan to investigate those variables but planned to use them as a context for my examination of institutional factors that affect faculty development. However, as data were analyzed, a subset of organizational context emerged as a critical factor in the results described at the DEEP schools. This subset included relationships among faculty developers, faculty, and administrators and will be analyzed and interpreted in Chapter 5. In order to gain additional perspectives on the faculty development programs, reduce self-report bias, and increase confidence in the research findings, I gathered corroborating evidence from academic and departmental administrators at four institutions through a survey posted online. Structure of the Dissertation Following this introductory chapter, a review of the literature is presented in Chapter 2. The areas of research that are relevant to this study include: the impact of faculty development on approaches to teaching; organizational impacts on faculty development, teaching approaches, and on student success in college; and the effect of faculty approaches to teaching on student success in college. Chapter 3 describes the methodology used in this study including the process of data collection and analysis as well as a description of the participants and their process of selection. Chapter 4 discusses the results of data collection and 18

PAGE 32

analysis, while Chapter 5 discusses the meaning of the results and implications and limitations of the study. 19

PAGE 33

CHAPTER2 REVIEW OF THE LITERATURE Introduction The goal of this literature review is to clarify effective faculty development practices and structures through an examination of historical context, underlying influences, and more immediate factors that contribute to best practice in faculty development. This chapter first examines the historical context, the scope of the review, and clarification of terms. Then, underlying influences are discussed, such as: the changing environment in higher education and its impact on faculty and faculty development, the changing roles of faculty development, and various mediators of student success in college. Next, this paper examines the immediate factors affecting effective faculty development and the interactions among them; these include organizational context, approaches to teaching, faculty development practices and structures, and student learning outcomes in higher education. Finally, this chapter summarizes the literature findings and discusses their meaning in the context of this study. Historical Context and Early Trends in Faculty Development Prior to the 1960s when baby boomers suddenly expanded the student enrollments in post-secondary institutions, faculty development was conceptualized as research productivity (Eble & McKeachie, 1985). When this upstart generation began loud complaints about the quality of teaching and learning on many 20

PAGE 34

campuses across the United States, universities and colleges responded with research into effective teaching, which resulted in the expansion of faculty professional development to include a focus on instructional improvement. The rise of instructional improvement lead to efforts to define, measure, and teach effective teaching practices. The Carnegie foundation funded an initiative, in 1969, by the American Association of University Professors (AAUP) and the American Association of Colleges and Universities (AACU) to improve college teaching; this brought student ratings into widespread use and revealed the need for increased attention to instructional development ( 1985). Thus began the search for good teaching and ways to achieve it, among them a growing interest in successful faculty development ( 1985). Several studies, in the late 1970s and 1980s surveyed directors of faculty development programs and faculty at institutions with such programs to clarify faculty attitudes about teaching and their faculty development experiences and also to try to determine the goals, strategies and characteristics of successful programs (Blackburn, Pellino, Boberg, & O'Connell, 1980; Eble & McKeachie, 1985; Gaff & Morstain, 1978). Blackburn et al. (1980) investigated faculty attitudes about teaching and faculty development at 24 institutions encompassing the range of Carnegie classifications. They found, surprisingly, that all faculty highly valued their teaching role but didn't feel they required help with their teaching because 90% of 21

PAGE 35

participants perceived themselves to be above average or superior teachers. The participating faculty rated knowledge of their discipline and subject matter as the most important component of good teaching and 73-83% indicated lecture was their principle teaching method. While these faculty perceived faculty development as helpful to their teaching, they believed its contribution lay in keeping them abreast of their discipline, not in the pedagogical instruction. It is a concern of this author that 25 years later, after thousands of faculty development programs, countless hours of training and intervention, and a significant investment of institutional funds, there remains a large number of faculty with similar beliefs about their teaching competence, perceptions of good teaching strategies, and attitude toward faculty development (Dale, 1999; Kuh & Nelson-Laird, 2004; Hativa, 2002; Wray et al., 2005). Research discussed later in this paper demonstrates the implications of these commonly held faculty attitudes for successful student learning (Hativa & Goodyear, 2002; Kember & Gow, 1994 ). Other researchers (Eble & McKeachie, 1985) surveyed a variety of post secondary institutions that were participating in the three-year Bush foundation study on effective teaching. Their findings detailed lists of effective practices, such as comprehensive, collaborative planning and assessment; however, they also revealed the difficulty in measuring effectiveness. The goal of faculty development was to improve education, but as they noted, "faculty development, instructional 22

PAGE 36

development, and curricular change are intended to improve education, but measuring educational outcomes is difficult, time-consuming, and expensive" (p.l78). One of the earlier studies to evaluate outcomes of faculty development programs (Gaff & Morstain, 1978) surveyed 479 participants and 442 non participants to discern the impact of faculty development on faculty and the institution. The faculty who participated in faculty development reported positive impacts on their teaching practice such as increased use of groups and increased student participation in discussions. Both participants and non-participants noted positive institutional outcomes such as improved climate for teaching and increased administrative support for teaching excellence While these three groups of researchers surveyed faculty and faculty developers directly in an effort to discern effective faculty development practice, other researchers attempted to define effective practices through a review of literature research documenting successful faculty development outcomes (LevinsonRose & Menges, 1981; Weimer & Lenze, 1989). Recognizing the difficulty of determining and measuring effective faculty development practice, Levinson-Rose and Menges (1981) ranked the 71 studies in their review according to the degree of validity they demonstrated resulting in high, medium and low confidence findings. In an attempt to avoid a Type II error, that is citing no significance when there really is, they also examined power in studies with 23

PAGE 37

insignificant results and small sample sizes. Their attention to the statistical process provided a significant measure of confidence in their findings. Of the 71 studies examined, there were few with sufficient rigor to clearly demonstrate effective faculty development practices; however, Levinson-Rose and Menges ( 1981) reached several conclusions about the state of faculty development in American colleges and universities. While workshops were found to be the most commonly used intervention to improve teaching, most of the workshops were designed for congeniality, were undemanding, and over-emphasized rapport at the expense of changed approaches to teaching. The programs they reviewed were well received by participants but lacked sufficient impact to change teaching practice, which is one of the stronger criteria for successful intervention. Some of the recommendations for best practice in faculty development derived from this research included: having clear training objectives, creating opportunities for relevant practice, providing feedback on practice efforts coupled with suggestions for improvement, then more practice. They also recommended programs include an opportunity for delayed practice after the training in order to create lasting changes. These recommendations are still relevant today and are echoed in research from the past 25 years, which will be discussed later in this chapter. Nearly a decade after the Levinson-Rose and Menges ( 1981) review, Weimer and Lenze ( 1989) used that model to review the literature published since 24

PAGE 38

1981 and corroborated many of the earlier findings. They ( 1989) looked for the same five categories as criteria for valid assessment of research findings as the 1981 study: teacher attitude from self-report, teacher knowledge from pre and post tests or observation, teacher skill from observation, student attitude from self report, and student learning from tests or observation. Like Levinson-Rose and Menges ( 1981 ), these researchers determined the strongest evidence for effective faculty development practice to be impact on students and the weakest to be self report; yet, like the earlier study, they found the large majority of articles analyzed in their review used self-report because of the relative ease of assessing these data compared to student outcomes. The results of Weimer and Lenze' s (1989) investigation demonstrated unequivocal support for only a few interventions: workshops longer than four hours duration and consultations about student ratings if the session involved discussion of teaching behaviors. They concluded that research evidence supporting the effectiveness of other instructional interventions, such as shorter workshops, other types of consultation, grants, leaves, and peer mentoring was for the most part "feeble and inconclusive, at best"(p.55). Echoing LevinsonRose and Menges ( 1981 ), Weimer and Lenze called for not only more research of effective faculty development practices, but also more rigorous empirical design of such research. 25

PAGE 39

While there have been some improvements in research about effective faculty development since 1989, many of the same shortcomings are evident in the recent literature and will be discussed in this chapter. The lack of clarity about faculty development contributions to improved student outcomes led to the framework for this research study, discussed next. Scope of This Research and the Literature Review Like many other studies, this research study sought to clarify effective faculty development practices and structures. However, this investigation attempted to strengthen the connection between faculty development practices and approaches to teaching that contribute to successful student learning outcomes by selecting participants only from institutions found to demonstrate higher-than-predicted student success (Kuh eta!., 2005). Kuh et a!. (2005) found that the 20 institutions investigated in this study produce better student learning outcomes and persistence, as measured by graduation rates and amount of time and effort their students devote to effective educational activities, than the other 700 participating institutions matched for Carnegie classification, selectivity, and size. We can assume these institutions are adding value to students' educational experience and hence are models of effective educational practice because of their exceptional student gain scores. Since faculty were primarily responsible for designing and facilitating the effective educational experiences that led to these students' success, by inference, 26

PAGE 40

teachers at these schools must have engaged in effective practice. In fact, the DEEP study found teachers at participating institutions shared a set of effective educational practices. The question arises, "How did these faculty become so effective?" We know that faculty development can have an impact on effective approaches to teaching (Gibbs, 2003; Hativa & Goodyear, 2002; Rust, 1999) and that those approaches to teaching in tum affect student learning outcomes (Hativa & Goodyear, 2002; Kember & Gow, 1994). It is unclear, however, the degree to which faculty development at these schools contributed to approaches to teaching that influence student success. Perhaps other factors, such as hiring practices or faculty reward systems were more salient to student success at the DEEP schools. In order to clarify the role of faculty development in creating the conditions that matter to student success in college, this research study focused on the relationship among the variables described above at successful institutions. In order to sharpen the focus on the role of faculty development, I attempted to distinguish its influence from that of another powerful factor in student success, organizational context (Diamond et al., 2002). Key Concepts and Terms In this research study, the term faculty development equates with staff development or professional development which are terms used in the United Kingdom and Australia. It includes practices and structures in a post-secondary institution that are intended to assist faculty maximize the quality of their 27

PAGE 41

contribution to the institution through any of their roles, but particularly teaching. For example, activities commonly practiced by faculty development programs include programs and workshops to enhance teaching strategies, consultation on teaching for individual faculty, integration of technology and teaching, providing a resource materials library, offering new faculty orientation, and providing faculty development grants. Effective faculty development describes a practice or a program that has been evaluated as successful in meeting its goals and objectives. Those goals usually include increasing the alignment of faculty approaches to teaching with those known to foster student success, the ultimate goal being improved student outcomes and persistence. While there is substantial agreement on the appropriate tools needed to measure successful faculty development programs (Gibbs, 2003; Levinson-Rose & Menges, 1981; Weimer & Lenze, 1989) there is scant literature citing the actual use of those tools. Measuring changes in faculty approaches to teaching as well as changes in student learning outcomes has been recommended since the 1970s. However, the most widely used tool, described by the literature, to measure success has been faculty satisfaction with the program, which most researchers, except for Rust ( 1999), agree is a weak measure of success and insufficient to address the underlying purpose of faculty development programs-improved student outcomes. 28

PAGE 42

Accurate and valid measurement of effective faculty development is one of the themes of this paper and will be discussed in more detail in this chapter. Although there are hundreds of articles investigating the variables and issues discussed in this research, only a sample of these have been included in this review of the literature. Research I considered most important was that which was congruent with the topic of this research study (relationship between faculty development and approaches to teaching that contribute to student success), provided useful insights, had strong empirical underpinnings, and was frequently cited by subsequent literature. Other important criteria for inclusion were studies that corroborated findings or highlighted differences of other major research as well as more recent work or older but classic studies. Underlying Factors Impacting the Variables in this Study While faculty development, organizational context, and approaches to teaching are the main variables in this study, there are several contextual influences that interact with these variables and thus require a brief discussion. These factors include the changing environment in higher education and its impact on faculty and faculty development, the changing role of faculty development, and how these changes affect student success. Changing Environment in Higher Education The changing environment in higher education has increased the focus on what students are learning, whether teachers are creating successful learning 29

PAGE 43

environments, and institutional contributions to student success. The following factors describe changes that impact faculty roles and perfonnance in the institution and challenge faculty developers to facilitate those changes in teaching approaches that contribute to student success. Changing student demographics in colleges and universities is one of the key factors in a higher education environment today that differs dramatically from that of the past. Students today are more diverse in race, culture, age, physical ability, gender identity, and socioeconomic status than ever before. They are also less prepared for the rigors ofhigher education, yet they are more demanding consumers of our "product"-education (Gardiner, 1994; Mooney, 2002). Another contributor to higher education's changing environment is the technological revolution that is shaping the values and communication among students, faculty, administrators, infonnation, and society. Access to infonnation has altered the authority of knowledge as well as the roles of professors and students (Association of American Colleges and Universities [AAC&U], 2002; Guskin & Marcy, 2002; Lick, 2002). Demands for accountability are a third factor in the changing higher education environment. It is no longer acceptable for professors to merely cover the topic and ignore the results of their teaching. Accreditation and government agencies as well as employers, parents, and students expect a new level of 30

PAGE 44

accountability and responsibility for the quality of education at American colleges and universities (AAC&U, 2002; Diamond et al., 2002). Linked to accountability is the challenge of decreased public funding for higher education. Today's climate of scarce resources and increasing financial pressure on institutions and programs increases the necessity for faculty development programs to demonstrate their value and prove that their cost is justified by their worth. Budget constraints have also forced some institutions to rely on less costly part time instructors who may not have the time for or interest in instructional improvement. A fourth factor in the changing environment is the paradigm shift occurring in post-secondary education, from a focus on teaching to a focus on learning (Gardiner, 1994, 2004; Tagg, 2005). While this shift may seem subtle or insignificant, it has had an enormous impact on every aspect of higher education including the primary organizing principle, the purpose of higher education. The former purpose of college was to cull those with lesser ability from the ranks of college students and prepare the remaining elite for their roles as leaders in our society. The purpose oftoday's colleges is to educate as many of our young people as possible in order to prepare them for their roles as workers, citizens, and leaders in the information age ( Barr & Tagg, 1995). 31

PAGE 45

Impact of Changing Environment on Faculty and Faculty Development Since learning is the goal of higher education organizations, and faculty are primarily responsible for ensuring student learning, then faculty are at the center of many of the changes in higher education. However, most faculty have not been prepared to deal with these changes and many remain ignorant of the nature of the sea change occurring in higher education as well as their role in it (Gardiner, 1994). Faculty developers are stepping into this turbulence and trying to help faculty understand and adjust to their new roles. The following two examples demonstrate how faculty development programs and practices mediate between the demands of the changing environment and faculty needs and issues. First, faculty development includes programs and strategies aimed at helping faculty understand and cope with the changing environmental demands and those of their institutions (Frantz et al., 2005; Hellyer & Boschmann, 1993; King & Lawler, 2003; Roche, 200 1 ). This strategy helps faculty deal with or avoid burnout or insecurity, two elements that increase faculty resistance to change and decrease their willingness to take risks (Allen, 2003; Austin, Brocato, & Roher, 1997). In this way, faculty developers aim to remove some of the barriers to effective instructional practice. Secondly, faculty development programs help faculty understand and learn effective teaching principles and practices. These learning-centered practices successfully meet many of the challenges of the changing environment. For 32

PAGE 46

example, by learning how to use small group interaction to facilitate learning with a variety of diverse students, faculty are able to improve the quality of student learning as well as student satisfaction with the institution (Pascarella & Terenzini, 2005). Both ofthese changes lead to increased student success and stronger graduates that meet workplace and societal demands (Kuh et al., 2004). Changing Role ofF acuity Development in Higher Education In spite of some success in helping faculty adapt to the changing environment, there remain significant challenges for faculty developers; one of these challenges is the changing role of faculty development within the institution. While faculty are struggling to adapt to the changing environment, faculty development programs are struggling to define their role in the institution as well as to find effective ways to help faculty make the necessary changes and learn the requisite skills and conceptual frameworks demanded in today's higher education environment (Sorcinelli, Austin, Eddy, & Beach, 2006). The previous role of faculty developer was to provide the necessary pedagogical skill training for untrained college educators. While faculty still need to learn effective teaching practices, there is a paradigm shift in the goals of faculty development, from merely training teachers, to the more difficult task of facilitating their comprehensive understanding of the learning process and helping them devise ways to facilitate and measure learning in their students (Tagg, 2005). In addition, developers are challenged to help faculty adapt to their changing roles and 33

PAGE 47

expectations in 21st_ century higher education (Tiberius, 1995). In fact, some researchers go further and suggest that faculty developers could play a pivotal role in facilitating institutional change, not just adapting to it (Roche, 200 l; Zahorski, 1993). It would be helpful for faculty developers trying to negotiate this paradigm shift to have a clear understanding about what constitutes effective practice in faculty development. However, just as it has been a struggle to define excellent teaching, it has remained difficult to define excellence in faculty development (Chism, 2004; Gibbs, 2003). Without clear information about effective practice in faculty development, developers aren't able to maximize their contribution, neither to faculty nor to the institution, both struggling to meet the new challenges of a changing environment. This research study attempts to clarify faculty development practices that contribute to teaching approaches that improve student success. A more detailed discussion of the research on effective faculty development practices follows later in this chapter. Changing Expectations Affect Student Success in College Having discussed the nature of the changing environment in higher education and its impact on faculty and faculty developers, it is helpful now to examine how these changes impact students. Most observers and participants in higher education agree that merely graduating is no longer the sole measure of success in college. Parents, employers, legislators, college administrators, as well 34

PAGE 48

as students themselves, are calling for a quality educational experience which includes deep, long-lasting, transferable learning as well as mastery of effective problem solving and higher order thinking skills (AAC&U, 2002). Success in college therefore, is measured by how well each student has mastered these processes that are integral to a quality education. While graduation, alone, is an insufficient measure of success in college, it remains one of the necessary indicators; the best predictors of student graduation are academic preparation and motivation (Kuh eta!., 2005). Many oftoday's students, however, are not sufficiently prepared for college academic rigors and many struggle with motivation to persist. So colleges and universities seek to influence students in ways that contribute not only to their graduation, but also to their success in acquiring those habits of mind now associated with quality education. Research on college student development has demonstrated that student engagement, that is, the time and energy students devote to educationally purposeful activities, is the single best predictor of their quality of learning and personal development (Kuh et al.). Student engagement, therefore, is one of the most important drivers of student success in college; it has input and output factors. The output is the amount of time and effort students put into their studies and other relevant activities. The input is the way colleges and universities use their resources and organize 35

PAGE 49

meaningful learning opportunities to motivate students to devote the appropriate amount of time and energy for success (Kuh et al., 2005). Just how colleges and universities create conditions that matter to student success and how teachers influence those processes and practices that lead to student success are research variables that will be discussed in more detail in a subsequent section of this chapter. Before examining those topics, however, the next section will review research into best practice in faculty development programs in order to establish a connection between faculty development and approaches to teaching that facilitate student success. Research Variables and Their Interactions The goal of this research study is to clarify the relationship between faculty development practices and structures and teaching approaches that contribute to successful student outcomes. To that end, this section examines three main variables and their interactions: best practices in faculty development, organizational context of faculty development, and faculty approaches to teaching that affect student outcomes. Best Practice in Faculty Development: Summaries and Recommendations Literature from the past 15 years that addresses perceived best practices includes descriptive studies (Austin et al., 1997; Frantz et al., 2005; Hellyer & Boschmann, 1993; King & Lawler, 2003; Murray, 2000; Saroyan & Amundson, 2004; Sorcinelli, et al., 2006; Whelburg, 2005) as well as more evidence-based 36

PAGE 50

research (Eble & McKeachie, 1985; Eckel et al., 1999; Holmgren, 2005; Kuh et al., 2005; Levinson-Rose & Menges, 1985; Rust, 1999). While some studies are more rigorous than others, there is substantial agreement about structures and practices that contribute to effective faculty development programs and activities. Synthesized from the best practice literature are the following five recommendations for effective practice: planning alignment, faculty involvement in planning, evidence-based measurement, ongoing activities that model a focus on learners and learner engagement, and collaborative support and rewards for the change process with a focus on improvement. Planning Researchers agreed that two aspects of planning, alignment and faculty participation, were necessary for effective faculty development. Several researchers (Cowan, George, & Pinheiro-Torres, 2004; Kuh et al., 2005; Murray, 2000) indicated the importance not only of aligning faculty development goals, objectives, methods and assessment in the program and in activities, but also of aligning faculty development goals with the institutional mission. The strength of that synergy is advocated by Cowan et al. (2004) and demonstrated by the success of schools using synergy in the DEEP study (Kuh et al., 2005). A second aspect of planning, faculty involvement, has strong support in the literature. Faculty development programs demonstrating the greatest success were characterized by a strong collaboration between faculty and administration in the 37

PAGE 51

planning and operation phases of the development programs. Where such collaboration was lacking, programs offered a disconnected smorgasbord of activities without a widespread or lasting impact on the institution ((Eble & McKeachie, 1985; King & Lawler, 2003; Murray, 2000; Sorcinelli, 200 l; Sorcinelli et al., 2006). Faculty development programs need alignment and participation in their planning to be effective. Once they know where they're headed, they then need to know if they are on the right track and whether their programs are having the intended impact on teachers, students, and the institution. Evidence-based measurement is the second practice, described here, which contributes to effective faculty development. Evidence-Based Measurement and Feedback Researchers strongly agreed that in order to guide effective practice, there was a need for better measurement of all the elements of a faculty development program or activity including content, process, and outcomes, both of teachers and students (Gibbs, 2003; Levinson-Rose & Menges, 1981; Weimer & Lenze, 1989). But studies that investigated faculty development programs found few that sufficiently utilized these indicators of effective practice. While most researchers recommended evidence-based measurement, Frantz et al. (2005) found that the majority of programs continued to rely on faculty satisfaction and participation as measures of success. Of the I 09 faculty 38

PAGE 52

development directors they surveyed, only 21 used any form of accountability for their programs and activities. Furthermore, the primary means of measurement used by those 21 directors was satisfaction and participation rates. While one study (Rust, 1999) reported a correlation between program satisfaction and teacher change, there is, as yet, no corroboration from research that satisfaction is related to teacher changes or increased student outcomes. Though scant, some literature described the use of appropriate assessment (Gibbs, 2003; Rust, 1999; Saroyan & Amundson, 2004). A comprehensive faculty development plan and program described by Saroyan and Amundson (2004) included teacher attitude from self-report, teacher knowledge from tests or observation, teacher skill from observation, student attitudes from self-report, student learning from tests and observation, and institutional benefit or change. Although this work was merely descriptive, it supported the need for rigorous measurement of practice outcomes. In other work, Rust (1999) found a significant percentage (70%) of the 245 workshop participants made changes in their teaching practice following participation in a expert-led workshop. Two key findings of this study have implications for faculty developers. One was that participants intent-to-change statements made immediately after the workshop correlated (r =.31) with actual changes made in the subsequent four months. Secondly, there was a moderate (r =.52) correlation between overall workshop rating and the actual changes reported 39

PAGE 53

by participants. Therefore, intent-to-change statements might be useful criteria to measure the success of faculty development interventions and overall satisfaction might be a more valid indicator of success than previously assumed by many researchers. Although the findings from Rust (1999) provided a significant contrast to all other research regarding the use of satisfaction measures as criteria for effective faculty development practice, it is important to note the context of this research. This study was done in the United Kingdom where there is a strong national emphasis on instructional development and outcomes measurement. Ongoing faculty development programs are mandatory for all post-secondary faculty and expectations for effective teaching practice are high. Under this strong pressure for change, from national agencies and their own institutions, faculty may be more inclined to report changes in order to meet these expectations. In addition, the results are self-reported changes without corroboration. While these factors suggest caution in interpreting the results, the strength of the findings merits follow up in American research. Another researcher from the United Kingdom (Gibbs, 2003) investigated training workshops for faculty in British colleges and universities and found that few of the studies had sufficient data to justify the use of workshops as tools to improve teaching. However, he cited several studies that created or adapted measurement tools that would yield the appropriate evidence. 40

PAGE 54

Gibbs (2003) recommended the use of student and teacher questionnaires to measure salient variables and outcomes in order to have valid evidence of successful programs and activities. The indicators Gibbs endorsed included: classroom teaching practice as perceived by students; change in the way students approach their studies (deep versus superficial); the impact of training on the teacher's approach to instruction, from teacher focus to student focus; and the impact of training on teacher's reflection, repertoire, and self-efficacy. Pre) iminary results from research of 24 institutions using these tools before and after training indicated a positive impact on teaching behaviors after three months of training. Development activities' Criteriafor Success The best Faculty Development activities model a focus on the learner and learning outcomes, encourage participant engagement, address a full range of faculty roles and provide follow up. The concept "walk your talk" applies to this third recommendation for effective practice. We know from adult education and learning literature that active participation, learning in small groups with immediate feedback, engagement in the process, choice in types and content of learning opportunities, and strategies to enhance motivation have a positive effect on faculty attitudes, beliefs, and willingness to try new teaching behaviors (Bess, 1997; Brookfield, 1995; Hativa & Goodyear, 2002; Kember, 1996; Sorcinelli eta!., 2006). 41

PAGE 55

Several studies demonstrated the validity of learning theories underpinning faculty development approaches. In the study by Rust ( 1999) described above, three of the six reasons for participants' self-reported successful changes in their teaching practice involved participant engagement in the workshop: interactive process, group work, and meeting others. Other researchers (King & Lawler, 2003) surveyed 249 faculty developers about perceived components of effective practice; these participants rated active learning at the top of their best practice list. Another aspect of this recommendation for best practice concerning activities is that interventions need to be ongoing and include follow-up rather than just one-shot workshops. We know that faculty change is a learning process and nothing as complex as teaching can be mastered in a single three-hour workshop (Eble & McKeachie, 1985). The last component of this cluster of recommendations for best practice is the need for faculty developers to address multiple faculty roles. While the changing environment in higher education has necessitated expanded faculty roles, including grant writing, research, teaching, community service, and service to the institution, faculty only recognize two roles: research and teaching. Researchers questioned whether faculty were aware of the other roles and expectations and whether faculty developers might be able to assist faculty in understanding and developing their other roles (Austin et al., 1997). 42

PAGE 56

This work also discovered that faculty got mixed messages about the relative value of their roles. Institutional leaders talked about the various roles but rarely did they explicitly describe what faculty were expected to do and how their efforts would be recognized or rewarded. Although most institutions say they value teaching, at some, the primary reward structure is for research and publication. While that would be consistent with the goals of a research institution, an increasing emphasis on research is found at comprehensive and liberal arts institutions as well. Faculty are expected to pursue grants and research, excellence in instruction, community service, and advising. Yet, without clear connections between expectations and compensation for their various roles, it isn't surprising that 70% of participants commented on the pressures of the workload within limited time --an indication of role stress (Austin et al., 1997). Faculty stress and its impact on their approach to teaching become an additional element faculty developers need to consider; faculty may need help balancing and prioritizing their multiple roles. As faculty developers try to deal with or help diminish faculty stress they might use community and collegial support, both of which are components of the last recommendation for best practice from the literature that follows. Support and Recognition.for Faculty The last recommendation for effective faculty development includes a variety of human resource elements that are essential to successful organizations (Bolman & 43

PAGE 57

Deal, 2003). Best practice encourages faculty developers to build support and recognition for the faculty change process into faculty development activities and programs and to focus on talent development, not remediation. Eckel et al. ( 1999) report on a longitudinal study of 26 campuses in which successful change leaders had an agenda that focused on improvement without blame and a climate of goodwill; these were crucial factors in successful change initiatives. It makes sense that if faculty developers were asking faculty to take a risk and change something about their teaching, it would be helpful to create a collegial climate of support and provide appropriate rewards and recognition for their efforts. Holmgren (2005) found in a needs survey on his campus that faculty desired but didn't find a community that supported their teaching so his team created a year long teaching partners program. It included many collegial supports for faculty: a weeklong summer workshop; exchange of class visits; regular events for participants; and weekly lunches for participants and alumni to discuss teaching and leaming in an open format. While participants in the program demonstrated substantial differences in their teaching compared to those who didn't participate, it is unclear whether those who chose to participate were better teachers before the workshop. Although some researchers (Levinson-Rose & Menges, 1980) have criticized the selection bias of voluntary participants, others have provided statistical analysis demonstrating no 44

PAGE 58

significant differences between voluntary participants and non-participants (Gaff & Morstain, 1978). It is important to note that the teaching differences demonstrated by participants in Holmgren's program (2005) included many practices found to be determinants of improved student learning: greater likelihood to incorporate a mix of active learning strategies into their course; increased use of collaborative learning strategies, class discussion, student-designed projects and assignments; and use of classroom assessment tools such as minute papers. In summary, although researchers are still seeking best practice in faculty development, the early focus on specific activities that contribute to effective practice has grown to include the contextual framework and process in which these activities occur. A synthesis of research literature demonstrated agreement on four categories of effective faculty development structure and practice, described above, that contribute to successful faculty development. While these practices are strong indicators of effective faculty development, another factor, organizational context, can impede or facilitate these faculty development structures and practices. The next section of this chapter examines how organizational context affects faculty development. Relationship benveen Organization and Faculty Development The research reviewed thus far discussed the purposes, goals, practices, strategies, and assessment techniques of faculty development programs in an 45

PAGE 59

attempt to discern elements of effective programs. However, faculty developers are definitely aware of the importance of organizational context to the success of their programs and any discussion of effective faculty development is incomplete without an understanding of how organizational context impacts faculty development. There is a body of literature that focused on the institutional factors, specifically organizational structures and processes that affected the effectiveness of all other programs, including faculty development (Allen, 2003; Astin, 1980; Cowan et al., 2004; Diamond eta!., 2002; Eckel eta!., 1999; Gardiner, 1994, 2005; Kuh et al., 2005; Saroyan & Amundsen, 2004; Sorcinelli, 2001 ). The following analyses will investigate the relationship between faculty development programs and the organization. The six main themes synthesized from this literature describe the interaction between faculty development and organizational context and include: campus climate and culture: collaboration among all levels of administration, staff, and faculty; the use of evidence-based practice; adequate funding of professional development; a school-wide focus on improvement; and clear, collaborative plans using a continuous process of assessment, feedback, analysis, and adjustment. A more detailed discussion of each of these themes follows. 46

PAGE 60

Climate and Culture Several researchers mentioned the impact of climate and culture on faculty development. According to Lick (2002), when organizational culture is a barrier to change, one must modify the change effort or change the assumptions beliefs or behaviors of the culture; otherwise, the change initiative will likely fail. This type of situation is demonstrated in institutional cultures where the focus on research trumps the importance of teaching. Although there are institutions where teaching is highly valued, the reward structure of many higher education organizations as well as the preparation of the faculty often mediate for a greater emphasis on research than teaching (Lick, 2002). Even colleges, for example, who advertise themselves as teaching institutions, in practice, typically require faculty to teach more classes than colleagues in other types of institutions, and do not necessarily reward or value teaching (Murray, 2000). This cultural orientation then becomes a barrier for effective faculty development programs if it isn't addressed through administrative support for teaching. Another way culture can help or hinder organizational change efforts is by establishing an envelope or range of possible behaviors within which an organization usually functions (Ramalcy, 2000). Models of institutional behavior include: collegial, where power is shared; bureaucratic, where rational decision making occurs, usually top-down; political, where different constituencies vie for power and resources; and anarchical, where each component of the organization is 47

PAGE 61

an island unto itself (Ramaley). Collegial decision-making was more likely to lead to successful faculty development programs (Roche, 2001; Zahorski, 1993). An aspect of culture that was disturbed by transformative change was the deep layer of culture, which emphasized a sense of identity, belonging, and citizenship in a community of like-minded people. Faculty developers grappling with changing teacher's beliefs and attitudes to enhance their teaching practice may indeed run into the anger and cynicism that arises from disturbance of deep cultural issues described here (Ramaley, 2000). While most of the literature emphasizes how organizations affect faculty development, Holmgrem (1994) investigated the reverse; he believed that a carefully designed faculty development program could facilitate a shift in campus climate around the issue of teaching. While we can't force faculty to change or adopt effective approaches to teaching and advising, change in campus culture can grow out of activities in which faculty willingly engage and which provide the intrinsic rewards necessary to sustain interest and commitment. Interestingly, Holmgren ( 1994) found that the majority of faculty in his study desired but didn't find a community that supported their teaching and he encouraged faculty developers to provide for this unmet need. Collaboration Another important concept in organizational change that influences the roles and goals of faculty development is collaboration among faculty, faculty 48

PAGE 62

developers, departments, administrators, and staff. Several researchers found collaborative work to be the best, and perhaps only, way to implement lasting and successful organizational change (Astin, 1980; Sorcinelli, 200 I; Sorcinelli et al., 2006). Sorcinelli included collaboration across campus departments and administrative levels as a requisite for successful faculty development programs and declared the new millennium as the "Age of the Network", for faculty development (p. 5). Other researchers believed that faculty were at the heart of higher education success, or lack thereof, and that collaborative change strategies were the only true way to get lasting change (Allen, 2003). Hierarchical, top-down initiatives have not been found to be effective in the long run (Saroyan & Amundson, 2004). Evidence-Based Practice A recurring theme in much of the faculty development literature is the use of valid evidence to determine the effectiveness of programs and processes (Gibbs, 2003; LevinsonRose & Menges, 1981; Rust, 1999; Weimer & Lenze, 1989). In addition to its importance for defining quality in faculty development, several researchers have emphasized its crucial role in successful organizational change. Research, technology, and best practice must be the foundation of all institutional change including learning, student development, teaching, curriculum design and instruction, assessment, and organizational change (Diamond et al., 2002). Often, many administrators and faculty are unaware of this research and fall 49

PAGE 63

back on anecdotal and historical precedent; therefore, leaders at all levels, striving for quality, need to be committed to systematic and continuous data collection and use to inform policy and decisions (Astin, 1980; Diamond et al., 2002). Adding to the emphasis on evidence-based practice, Ramaley (2002) found that a discipline of reflection, culture of evidence, and insistence that everyone support their perspectives with real information, not just opinions, were the cornerstones of effective organizational change. In fact, evidence is the basis for everything that is taught in higher education. It is paradoxical that institutions' primary work rests on valid and reliable evidence yet the way in which this process occurs is commonly based on history, custom, and myth, not evidence. Appropriate Funding for Faculty Development A key element in the success and quality of faculty development programs is their level of funding as well as the commitment of institutional resources, financial and otherwise, to support teaching, learning, and the work of faculty (Diamond et al., 2002). Gardiner (1994) pointed out that today's businesses are well aware of the necessity for personnel development. While average businesses spend 2% of their payroll on development and training in order to remain competitive in the global marketplace, successful businesses spend up to 5 or 6% of their payroll on development and training. Gardiner ( 1994) noted that education falls short of those standards and believed that "a significant improvement in the 50

PAGE 64

quality of higher education will require a major investment in the development of the institution's human resources" (p.l34). For institutions whose primary mission is learning, all aspects of the organization need to reflect and support that mission including how resources are allocated, how decisions are made, and in the reward and recognition system for individuals, programs, and support units, such as faculty development (Diamond et al., 2002). Academic leaders seeking successful organizational change should integrate fiscal and academic planning as well as recognize the need for ongoing professional development for themselves, their faculty and staff (200 1 ). It is unfortunate that public financial support for higher education has been decreasing at a time when there is need for greater investment to meet the demands of a changing environment as previously discussed. Focus on Improvement In addition to organizational factors such as climate, collaboration, evidenced-based research, and adequate funding, researchers have demonstrated that a school-wide focus on improvement has a powerful influence on faculty attitudes about teaching and the success of faculty development efforts (Astin, 1980; Eckel et al., 1999; Kuh et al., 2005). As early as 1980, Astin found that a continuous process of critical self-examination that focused on the institution's contribution to student intellectual and personal development were necessary precursors of high quality academic institutions. Subsequent literature identified a 51

PAGE 65

climate of goodwill and improvement without blame as important contributions to successful change initiatives (Cowan eta!., 2004). In a recent qualitative investigation into the successful practices at institutions found to have greater-than-predicted student success, Kuh et a!. (2005) discovered that all twenty of the highly successful institutions they studied demonstrated an improvement-oriented ethos throughout all aspects and at all levels of their institution. In fact, an improvement orientation became one of their key recommendations to colleges and universities seeking to improve institutional performance. Continuous Assessment, Feedback, Analysis, and Adjustment The last organizational factor discussed here that impacts successful faculty development is the use of clear, collaborative planning that includes the continuous use of assessment, feedback, analysis, and adjustment. Whether the focus is on individual faculty, a faculty development program, or the institution, change is not a one-time initiative but rather, a constant, ongoing process of improvement. Diamond eta!. (2002) noted that research, technology, and best practice must be the foundation of all institutional change including: learning, student development, teaching, curriculum design and instruction, assessment, and organizational change. Leaders at all levels need to be committed to systematic and continuous data collection used to inform policy and decisions. 52

PAGE 66

Cowan et al. (2004) found that change efforts throughout the institution reinforced each other. When faculty development, curriculum, and organizational change efforts were aligned and all focused on learning outcomes, their synergy enhanced change efforts. This multiplier effect is one of the key findings from a study of successful institutions (Kuh et al., 2005). The colleges and universities in this study weren't doing anything unusual but they were engaged in improvement efforts at all levels and areas of the institution and it was this alignment of efforts toward successful student learning that made a crucial difference (Kuh et al.). The next section of this chapter deals with one of those variables that affect student success in college faculty approaches to teaching. Approaches to Teaching that Affect Student Learning Outcomes In this section, four areas of research will be examined that connect student learning and persistence with what teachers think and do in and out of the classroom. These elements include: teachers' conceptions of learning, teacher characteristics and beliefs, student perceptions of exemplary teachers, and teaching behavior connected to student persistence and cognitive growth. Teacher Conceptions ofLearning One area of research related teachers' conceptions of learning and teaching to student learning approaches, specifically a surface approach, deep approach, or an achievement approach (Kember & Gow, 1994). Teachers in this study had various conceptions of teaching and learning including teaching as "learning 53

PAGE 67

transmission" and teaching as "facilitation of learning". The forn1er teaching approach, learning transmission, was correlated with students who increasingly used a surface approach to learning, which was characterized by extrinsic motivation, minimization ofwork, and use of memorization to "learn" what might be on the exam (Kember & Gow, 1994). Conversely, an approach to teaching that was conceptualized as "facilitation of learning" was correlated with students who increasingly used deep and achievement approaches to learning. A deep approach was characterized by intrinsic interest in the content and finding meaning in the content; an achievement approach was characterized by enthusiasm, a high will to succeed, and highly organized study skills. Both of these approaches led to successful, transferable learning (Kember & Gow, 1994). Therefore, a facilitation-of-learning approach to teaching led to more successful student learning. Teacher Characteristics and Beliefs A second area of research about approaches to teaching that affect student learning investigated teacher characteristics and beliefs that impeded teaching effectiveness as measured by observation, student feedback, and student learning (Hativa, 2002). This researcher found the teaching behaviors most often associated with poor ratings from students included speech and language problems, associative thinking rather than linear thinking, and insecure behavior. 54

PAGE 68

In addition, the following negative teacher beliefs were related to a decreased motivation to teach well: a transmission-of-knowledge view of teaching, the student as solely responsible for learning, a perception of the ultimate need in the classroom as content coverage, the misconception that adapting instruction was watering down material and decreased the level of teaching, and a perception that the students were unfit and unequipped to appreciate instruction (2002). This work echoed that of Stevens ( 1988) who found that those faculty who embraced change the least, placed sole responsibility for learning on the students, and blamed them when things didn't go right. These negative attitudes, then, were found to be a barrier to effective teaching and successful student learning. Student Perceptions of Exemplwy Teachers A third area of research that connects approaches to teaching and student success investigates the relationship between student perceptions of exemplary teachers and higher student achievement. Feldman ( 1998) examined 26 studies relating student perceptions of exemplary teaching to student learning outcomes. The highest correlations were between student achievement and the following approaches to teaching: teacher preparation and course organization (.57), clarity and understandableness (.56), course objectives met or clearly pursued (.49), teacher stimulated interest in course (.38), and motivates students to do their best (.38). 55

PAGE 69

These factors were not only related to student achievement but were also ones that students rated most highly for effective teaching. Rated less important were teacher knowledge, enthusiasm, and availability. Students in this study rated difficulty of the course as not important to teaching effectiveness; this finding is in contrast to a common myth, that faculty can improve their ratings by making their courses eas1er. Teaching Behaviors Connected to Student Persistence and Cognitive Growth The last area of research about approaches to teaching that affect student learning outcomes was similar to Feldman's study (1998) but examined student cognitive growth and persistence in addition to academic achievement as they related to teacher variables (Pascarella & Terrenzini, 2005). Pascarella and Terenzini surveyed 2600 research articles about the impact of college on students and added the results to their previous study of similar work from 1970-1990 (Pascarella & Terrenzini, 1991 ). They found that of the nine factors impacting student persistence, seven were determined by the classroom teacher: use of cooperative learning and active learning; instructor organization, skill, clarity, and preparation; classroom discussions and activities requiring higher order thinking skills; absence of classroom racial discrimination or prejudice; and faculty availability. Pascarella and Terrenzini (2005) determined that the following pedagogical approaches improved student learning by the noted percentages of a standard 56

PAGE 70

deviation: cooperative learning (.51), small group learning (.51), mastery learning (.41-.68), supplemental instruction (.39), computer-assisted instruction (.31 ), constructivist-oriented approaches ( .14-.40), and active learning ( .25). In addition, students' cognitive growth was positively influenced by a close relationship and frequent interaction with faculty. Therefore, caring teachers who were clear, organized, and prepared and who used active learning strategies, groups, and constructivist pedagogy to facilitate deep learning in their students not only were rated as exemplary teachers but in fact produced significantly better learning outcomes than peers who were not so skilled or inclined. Data from the NSSE study demonstrated that effective faculty shared common practices that led to greater student learning and persistence such as designing learning environments with more educationally purposeful activities and formats and behaving in ways that fostered student engagement in learning. Examples of these effective teaching practices included: 1. Assign more papers with drafts (20-page preferable) 2. Assign more reading 3. Assign collaborative projects 4. Hold students accountable for performance on assignments and projects 5. Grade and give timely feedback 6. Create challenging learning opportunities 7. Significant use of active learning strategies and methods 57

PAGE 71

The DEEP study highlighted additional details about faculty approaches to teaching that contribute to student success. Compared to teachers at non-DEEP schools teachers at DEEP institutions demonstrated more of the following: I. Focus on the quality of their work and its impact on students and institutional performance. 2. Are conscious of the need to understand students and engage them actively in the classroom. 3. Recognize they need to be more creative to keep students interested 4. Are directly involved in driving, shaping, and implementing curricular innovations. 5. Are willing to give up a little of their own area to benefit the whole. 6. Are willing to think broadly about the nature of education. 7. Spend more time with students focused on educationally purposeful activities. 8. Demonstrate an intensity of focus on student growth and development. 9. Accommodate multiple student learning styles and provide opportunities to apply their learning. I 0. Provide the challenge and support needed to get students to perform at high levels. II. Demonstrate high quality and frequency of interactions with students concerning their academic and social well-being. Teachers at the schools in the DEEP study were found to use more of the best teaching practices more frequently than teachers from schools with less successful student outcomes (Kuh et al., 2005). However, we do not know how the DEEP faculty have acquired their effective teaching approaches. What contribution did faculty development at these schools make to teaching excellence? My research clarifies the contribution of faculty development structures and practices to the effective teaching approaches found at DEEP schools. 58

PAGE 72

While these approaches to teaching have a profound effect on student learning and persistence, there are additional aspects of colleges and universities that affect student success. The next section of this chapter will examine these organizational influences on students. Relationship between Organizational Context and Student Success in College While there are many studies that connect specific organizational structures and practices with various measures of student success in college (Astin, 1980; Cowan et al., 2004; Diamond et al., 2002; Gardiner, 1994; Pascarella & Terrenzini, 2005; Saroyan & Amundson, 2004), the most comprehensive study has come from the DEEP project described by Kuh et al. (2005). This study was based on data from the National Survey of Student Engagement (NSSE) that examined the degree to which students engaged in the activities and practices that research has determined result in increased student success (2005). The NSSE research determined the following five clusters of effective educational activities, that is, activities in which students engage and which thereby contribute to their college success: level of academic challenge, participation in active and collaborative learning, student interactions with faculty members, enriching educational experiences, and supportive campus environment. DEEP schools were selected based on their scores on the NSSE survey, a tool that measures perceptions of student engagement in various college experiences. This instrument is used as a proxy for student success and by 59

PAGE 73

implication quality of the institutional experience. All DEEP schools had participated in the National Survey of Student Engagement (NSSE) and had scored better than predicted across some or all of this survey's five benchmarks of effective educational practice: level of academic challenge, active and collaborative learning, student interaction with faculty members, enriching educational experiences, and supportive campus environment. All twenty also enjoy a graduation rate that is higher than predicted after taking into account relevant student and institutional characteristics. Because this research study shares many assumptions with the DEEP study, it was helpful to examine the validity of the assumptions made by DEEP and the NSSE scores from which DEEP derived its standards. The Project DEEP team used student engagement as a proxy for quality because engagement had been shown to be the best predictor of student success, after controlling for past academic performance and preparation (Kinzie & Kuh, 2004 ). Student success in the NSSE study was defined as engagement in educationally purposeful experiences which were categorized by their Benchmarks of Effective Educational Practice. Some literature argues that the self-report nature of any college student survey makes it an invalid measure because college students are unable to accurately report about their experiences (Porter, S. R., 2009). Other researchers counter this assertion. According to research from Pike & Kuh (2005) the validity of self-report data has been extensively studied (Baird, 1976; Berdie, 1971; Pace, 60

PAGE 74

1985; Pike, 1995; Pohlmann & Beggs, 1974). Findings ofthese studies indicated that self report data is likely to be valid if the following conditions are met. l. The information requested is known to the respondents; 2. The questions are phrased clearly and unambiguously; 3. The questions refer to recent activities; 4. The respondents think the questions merit a serious and thoughtful response; 5. Answering the questions does not threaten, embarrass, or violate the privacy of the respondent or encourage the respondent to respond in socially desirable ways (Kuh et al., 2001, p. 9). The NSSE survey meets four of the five validity criteria. The only criteria that NSSE can't control for is the student belief that the questions merit a serious and thoughtful response. The survey is not asking about nebulous concepts that require substantial interpretation but rather specific activities and experiences in which students have engaged over the past year. These questions include, among others: the number of hours each week they spend preparing for class; the number of papers they have written more than 5 pages long, more than 20 pages long; how often they discussed grades, assignments, career plans or other ideas with faculty outside of class; how often they work together with other students for class projects in class and out of class; how often they talk with students who differ from them racially or politically; and how well the campus environment provides support needed to succeed academically and thrive socially. Another concern about NSSE data is that the internal validity is questionable when self reported gains are taken to be a learning outcome of 61

PAGE 75

effective educational measures described in the NSSE benchmarks. Because there is no pre-college measure of a student's receptiveness to educational experiences, the changes that NSSE measures in college students might be explained by differential openness to the college experiences rather than the added value of college as the instrument's authors claims. Research by Pascarella, Siefert, and Blaich (2010) addressed those concerns in a longitudinal study that measured outcomes directly in a pre-test-post-test design and compared them to responses on the NSSE instrument. They found that all but one of the Benchmarks had a significant partial correlation (mean value of all correlations was .34) with at least one of the seven liberal arts outcomes derived from a model of college outcomes developed by King, Kendall Brown, Lindsay, and VanHecke (2007). The level of analysis was the institution. While the small sample lead to lower statistical power and limited the analysis to partial correlations, the longitudinal nature of the study enable control of pre-college scores on each of the liberal arts outcomes, thus providing a more valid estimate of the value added by the college experience. The findings of this study, then, supported NSSE's claim that students' educational experiences are good proxy measures for growth in important educational outcomes detailed in the NSSE Benchmarks. Their gauge of academic quality, then, can be viewed as a reasonable way to ascertain relative quality of DEEP institutions selected through NSSE outcomes. 62

PAGE 76

A final criticism of NSSE that will be examined is that the response rates of NSSE affect the trustworthiness of the results and thus could affect selection of DEEP schools. A personal response from Dr. Jillian Kinzie, one of the lead DEEP authors, indicated that while she could not reveal details from specific schools, "The data quality at all these schools was a factor; we did not include schools if data quality was poor.'' The limitations ofthe NSSE data, then, affect aspects of the DEEP data and as such need to be kept in mind when interpreting data in this research study. These limitations are reviewed in chapter 5. Trying to document effective educational practice, the DEEP project examined freshman and senior data, relative to the five clusters of effective practice, from over 1,000 colleges and universities who had participated in the NSSE survey. These data were then analyzed to determine what changes in students' learning, thinking, and performance had occurred during their four-year college experience. In other words, what kind of gains did these students make and to what degree. Using a regression analysis, these researchers looked for schools with greater numbers of students who scored higher on the NSSE survey than predicted by their educational preparation or demographic data. That means, the performance of students from these institutions surpassed that of similar students at institutions within the same Carnegie classification and size. The most successful institutions in the DEEP study not only had more students who gained more during their college 63

PAGE 77

experience, but these schools also achieved graduation rates significantly higher than predicted from student preparation data on a regression line. The most successful institutions, then, produced exceptional results by adding value to their students' educational experience not by just recruiting top scholars. Subsequent qualitative analysis of the top performing institutions by teams of investigators uncovered six features, shared by all twenty selected schools, that made a difference crucial to the success of students: a "living" mission and a "lived" educational philosophy, an unshakeable focus on student learning, local environments adapted for educational enrichment, clearly marked pathways to student success, an improvement-oriented ethos, and a shared responsibility for educational quality and student success (Kuh et al., 2005). While these key conditions that impact student success were examined and discussed in detail, very little was noted about the impact of faculty development at these schools. Faculty development was only specifically mentioned during the discussion of four out of the twenty selected schools and nothing unusual was documented about those four development programs. Much of the discussion about the focus on student learning in the DEEP study echoed the same approaches to teaching detailed in others' research on effective teaching, which were discussed earlier in this chapter. The question remains, however, what are the connections between faculty development at these successful schools and approaches to teaching that contribute 64

PAGE 78

to their students' success? What are the elements of their faculty development programs that are effective in changing teaching approaches in ways that we know lead to successful student outcomes? In summary, there is good evidence for what students need to be doing and thinking in order to succeed in college, how approaches to teaching facilitate or impede student success, how organizational context creates the conditions that matter to student success, and some evidence for how faculty development leads to faculty adoption of more effective teaching approaches. However, there is scant literature investigating which elements of a faculty development program are effective in changing teaching approaches in ways that improve student outcomes. This research study focuses on those details in an effort to illuminate the relationship between faculty development practices and structures and effective teaching approaches. 65

PAGE 79

CHAPTER3 METHODS Introduction The purpose of this study was to explore the relationship between specific faculty development practices and structures and teaching approaches that contribute to student success at institutions with higher-than-predicted student outcomes. As described in chapters one and two, there has been little research in higher education describing the specific factors in faculty development that influence this relationship. Theoretical framework The description of the theoretical framework for this research hypothesized three university-controlled factors that impact student success in college: organizational context, faculty approaches to teaching, and faculty development as it affects teaching approaches (Kuh et al., 2005; LevinsonRose & Menges, 1981; Pascarella & Terrenzini, 2005; Wehlburg, 2005). The initial purpose ofthis research study was to examine faculty development and its impact on effective teaching approaches. These schools have demonstrated that they share effective organizational structures and practices that contribute to their high-than-average levels of student success; therefore, these variables were not initially a focus of this research study (Kuh et al.). However, the relationship among administration, faculty development programs, and faculty within their organizational structure, 66

PAGE 80

emerged as a salient factor in the relationship between faculty development and teaching effectiveness. Therefore it became one of the foci for the research study and will be discussed in Chapters 4 and 5. A focus of this research study was the congruence of faculty development best practices and those faculty development practices and structures found at the DEEP schools. These best practices have been connected to positive changes in faculty approaches to teaching, as described in chapters one and two (Gibbs, 2005; Helyer & Boschmann, 1993; King & Lawler, 2003; Murray, 2000; Rust, 2001; Sorcinelli, 2001 ). Effective faculty developers: I. Have clear goals connected to student performance and congruent with institutional mission and have detailed plans to reach those goals. 2. Involve faculty in setting goals, designing plans and assessment tools, and implementing activities. 3. Design and implement ongoing, engaging development activities with appropriate practice, follow up, and support for faculty. 4. Use appropriate tools to measure the effectiveness of their activities and programs. 5. Make appropriate changes as a result of the assessment feedback. 6. Develop connections throughout the institution to help support their mission and goals. These best practices formed the organizational structure for the data findings in Chapter 4 and data analysis in Chapter 5. 67

PAGE 81

Design Four research questions were proposed in chapter one to explore how to create the connection between faculty development practices and structures and teaching approaches that lead to higher-than-expected student outcomes at high performance colleges and universities. The following research questions drove the design of the research. These questions led to a qualitative design with two quantitative surveys lending additional perspectives that helped triangulate these data and increase confidence in the findings. 1. What are the common structures and practices among faculty development programs at institutions that produce better than predicted student outcomes (DEEP schools)? 2. In what ways are faculty development structures and practices at DEEP schools congruent with best practice as described in the literature? 3. What are the elements of faculty development structure and practice at DEEP schools that differ from best practice as described in the literature? 4. What is the perceived influence of faculty development best practice at DEEP schools on faculty approaches to teaching? In order to answer the first question, the nature of each faculty development program needed to be ascertained through the faculty development directors' online survey and the faculty development directors' phone interviews. Areas of interest included program structures, practices, philosophy, and priorities. The online survey was completed through open-ended questions that asked directors to rate in importance their practices, list their program goals, and list obstacles that were 68

PAGE 82

barriers to achieving those goals. Phone interviews with directors, using a semi structured interview protocol of questions and probes, provided additional information needed for question 1 [see director interview questions, Appendix A]. Comparing director responses in the online survey and in the phone interviews clarified the similarities and differences among the DEEP schools' faculty development programs and between DEEP schools and those reported in previous literature To answer questions two and three, this research study compared and contrasted faculty development programs at DEEP schools with recommended best practice. Comparing director responses in the online survey and in the phone interviews with best practice highlighted areas of congruence and divergence with the list of best faculty development practices noted above and analyzed in chapters four and five. Question four, "What is the perceived influence of faculty development best practice at DEEP schools on faculty approaches to teaching?" was designed to examine whether programmatic and philosophical choices made by the faculty development directors influenced the better-than-average performance of their faculty, as reported in the Kuh study (2005). Evidence gleaned from the literature de::;.cribed faculty development best practices that are perceived as effective in helping faculty make positive changes in their teaching approaches. These effective practices include faculty ownership, evidence-based measurement, activities modeled on participant engagement, and collegial, collaborative 69

PAGE 83

community experiences. To clarify the relationship between faculty development and effective teaching approaches, interviews with the directors of faculty development programs and surveys of the directors and administrators were examined and analyzed. This analysis sought to clarify the rationale for faculty developers' practices, structures, philosophies, and priorities and to determine their possible influences on teaching approaches. Results were analyzed and discussed in Chapters 4 and 5. This was an exploratory study that used a qualitative phenomenological approach in conjunction with quantitative data to develop a clearer understanding of the specific factors in faculty development programs that positively influence faculty approaches to teaching that contribute to student success at high performance post-secondary institutions, referred to in this study as DEEP schools. Phenomenology is a qualitative method that provides a systematic research paradigm with which to explore individual or group experiences (Miles & Huberman, 1994). It is used to help us uncover an individual's understandings and perceptions about a phenomenon, in this case faculty development and its relationship to successful teaching approaches. This approach allows researchers to capture participants' experiences, attitudes, and beliefs about faculty development, their programs, faculty, administrators, and institutions that could help lead to new insights about these factors and their relationships. Therefore, a phenomenological 70

PAGE 84

approach is best suited for this research study that explores relationships to successful teaching approaches from several data sets. According to Miles and Huberman ( 1994) the technique of triangulation is used in qualitative research to confirm the findings and increase the trustworthiness of these data. There are different kinds of triangulation, such as triangulation by data source, by data type, by method, and by researcher. This research used triangulation of sources and methods to improve conceptual clarity and purpose (Cresswell, 2008; Knafle & Breitmeyer, 1989). The source triangulation included individuals with different roles, such as faculty development directors and administrators, as well as individuals in the same role, faculty development director, at different institutions and different institutional types. Another source used was information collected from the online website of each faculty development program from the DEEP schools. Use of different methods is another form of triangulation. Data collection strategies used in this research study listed in Table 3.1 included qualitative analysis of interview responses, quantitative analysis of survey responses, means comparisons from survey data, as well as response rate and frequency comparisons of responses from interview participants. A third part of the triangulation process in this research was to enhance the trustworthiness of the responses by soliciting feedback from participants. They were asked to review the qualitative findings and 71

PAGE 85

confirm their responses, add to, or change information that did not accurately represent their views and situations (Krefting, 1991 ). In order to gain additional perspectives on the faculty development programs, reduce self-report bias, and increase confidence in the research findings, administrators at each institution were asked about organizational influences on faculty development, such as campus climate, program support, and appropriate funding. They were also asked about their relationship to the faculty development program and their perception of the impact of that program on effective teaching practices found at their institutions. Those results were analyzed and reported in Chapters 4 and 5. This research study used four data points to answer each of the first three questions concerning common faculty development structures and practices at DEEP schools and congruence of faculty development practices and structures at DEEP schools with best practice. Those four data points included the directors' survey open-ended responses, interview responses which were qualitative measures and two quantitative measures: directors' survey rankings of practices and interview response frequencies. Three data points were used to answer the last question the impact of faculty development practices and structures on faculty approaches to teaching. One data point, director's interview responses, was qualitative and the other two were quantitative: administrator's survey response frequencies and directors' survey response frequencies. 72

PAGE 86

Through the use of these multiple sources and modes, this research study sought to clarify the findings, enhance confidence in the results, and reduce sources of bias. Table 3.1 Data Collection Strategies Research Questions Qualitative Methods Quantitative Methods What are the common Directors Survey Directors Rankings and structures and Responses Comparisons practices among faculty development programs Directors' Interview Directors Survey at institutions that produce Responses Frequencies and Rates better than predicted student outcomes (DEEP schools)? In what ways are faculty Director's interview Directors' interview development structures and responses response frequencies practices at DEEP schools congruent with best practice Directors' survey Directors Survey as described in the responses frequencies and rates literature? What are the elements of Director's interview Directors' interview faculty development responses response frequencies structure and practice at DEEP schools that differ Directors' survey Directors Survey from best practice as responses frequencies and rates described in the literature? What is the perceived Director's interview Administrators' survey influence of faculty responses response frequencies development best practice at DEEP schools on faculty Directors' survey approaches to teaching? response rates and frequency 73

PAGE 87

Participants The 20 DEEP institutions represent a diversity of missions, selectivity, size, funding, and student characteristics. Nine are private, eleven are public; four are Research I schools while others focus exclusively on undergraduate education. Four are primarily residential colleges; four have a majority of commuter and part time students, while the others have varying combinations of residential, commuter, and part-time students. Two of the schools are historically Black institutions, two are Hispanic serving, two are women's colleges, and one is a men's college. Many of these schools were founded in the 191h century and have a strong cultural tradition, while three were founded in the second half of the 201h century and are still constructing their cultural identity. Several institutions are among the most selective American colleges while others provide educational access to many students, including those who might be marginally prepared for college-level work. Some of the schools have fewer than 700 students while others have more than 30,000 and the tuition at these schools ranges from $1,168 to$26,000 per year. Table 3.2 describes salient features of the DEEP institutions represented in this research study. Subjects were selected specifically from institutions where strong student success had been demonstrated by NSSE data and the DEEP project research findings (Kuh et al., 2005). Kuh et al. also indicated that the teachers at these institutions were exceptional at providing the type of instruction, climate, and 74

PAGE 88

interactions that lead to positive student learning outcomes and persistence. The question remained, however, how these outstanding faculty came to acquire their effective professional skills and practices. ln this research study, I sought to determine how faculty development programs at these institutions contributed to effective teaching approaches that are thought to influence student success demonstrated by the DEEP schools. Integral to this research study was the participation of faculty development directors and administrators from all the above-mentioned categories in the 20 high performing schools identified in the DEEP study. One faculty development representative from each of the 20 DEEP schools was invited to participate in this research; the 13 directors who agreed to participate in phone interviews ( 65% response rate) and the 12 who participated in the online survey (60% response rate) became part of this study. Since the 12 online participants were anonymous, the degree of overlap between the online participants and the interview participants was unknown. The directors who participated in the phone interview provided a good cross section of all the types of institutions in the DEEP study and thus added to the internal validity of these qualitative data and qualitative credibility that the data were accurate representations of the participants. Descriptors of each group are included in Tables 3.2 and 3.3. Table 3.2 describes the DEEP schools and faculty developers who participated in the phone interviews for this study. Participants included directors 75

PAGE 89

of faculty development programs, or, for those institutions lacking a director, the individual responsible for faculty development on that campus. The institution names have been changed to preserve confidentiality. Table 3.2 Faculty Development and Institutional Demographics by Phone Interview School UnderSchool Type Number FT Years FD (pseudonym) grad Faculty/Total Director/ Center/Years Students FT,PT Program Midwest 2,245 Private, religious125/225 15 /assoc. NO women's affiliated, women's Vice Provost ccnter/15 Pacific state 3,618 State, Hispanic 120/300 1.5 FT Center/15 focus Pine state 3,901 State, liberal arts 250 6FT Center/15 Western 3,485 Jesuit, liberal arts 5115% time Center/4 Catholic U. FD Eastern U. 15.312 State research 1000/2024 3 /PT Center/9 university Assoc. provost Central College 1,773 Private, liberal arts 163/263 1.5 FT Center/10 North Central 14,720 State, Research 750/1500 Program 20, Center/10 U. university center 10FT Atlantic 682 Private, women's 71 NO center Women's College Northeast State 2.000 State, liberal arts 140/180 FTIO Center/10 University Great lakes U. 23.000 State. Research I 3300 FT 17 Center/ 50 Texas 17.261 State, large 680/1200 3 Center/IS University Hispanic, Research PT I Associate I provost Midwest Men's 853 Private. men's 85 NO center/ College committee 76

PAGE 90

Table 3.2 (cont) School UnderSchool Type Number FT Years FD (pseudonym) grad Faculty/Total Director/ CcntcrN ears Students FT,PT Program Southeast state 2,320 State, HBU 350 2FT Center/8 university. Twelve anonymous faculty development directors participated in the online survey. Their demographic information is described in Table 3.3. Table 3.3 Faculty Development Program and Director Demographicsfrom Online Survey FD Years Years of No. of Percent Yearly Number of Center as Program Full-of Time Budget Faculty FD time Devoted Excluding Staff to FD Salary Yes 0 lO 2 90 40,000 1200 Yes 15 5 5 100 110 200 Yes I 8 0 100 63,000 2000 Yes I 13 2 30 50,000 900 Yes 0 12 0 50 40,000 130 Yes 6 6 4 100 600,000 250 Yes 6 15 3 50 300,000 250 No 6 20 I 20 5,000 120 No 8 8 6 20 40,000 140 No 1 2 0 25 16,500 350 No 0 many 0 20 200,000 150 No 7 decades 0 5 100,000 194 In addition, 13 administrators from the interview participants' schools were invited to take the online administrators survey. Four of those administrators participated in the survey, for a response rate of 20%. The participating 77

PAGE 91

administrators were from North Central University, Central College, Southeast State University, and Western Catholic University. Although this was a small group of participants, they represent several types of institutions: one large, state research university; one small, private, liberal arts college; one small, state, HBCU; and one small, Jesuit university. Procedures Twenty potential participants were contacted through email to briefly describe the study and invite them to participate in the online survey. A reminder invitation was sent two weeks later to discuss the project and encourage their participation. Twelve directors responded to the online survey. The data were collected in an excel spreadsheet to aid in analysis and comparisons of the 12 participants. Demographic information was collected about the directors' experience and position in order to analyze potential factors that might have impacted the quality or effectiveness of the faculty development program. These data were compared to earlier surveys of faculty development directors reported in the literature (Frantz et al., 2005; King & Lawler, 2003) in order to ascertain similarities and differences between faculty development programs at DEEP schools and those in the literature. After a two-year hiatus because of personal health issues, I again contacted the 20 directors via email to participate in a one-hour phone interview. Thirteen of the twenty responded and interviews were arranged and conducted by phone. Each 78

PAGE 92

PAGE 93

Data from the administrators were gathered in an online survey. Information from the administrators helped to clarify the relationship between the institution and the faculty development program, specifically, what kind of support the administration extended to the program. In addition, administrator input provided another viewpoint about the effectiveness of the faculty development program and how faculty development might contribute to teaching effectiveness and student success on each campus. To reduce researcher bias and improve trustworthiness, accepted qualitative protocol was followed. During the interviews, a record was kept of researcher perceptions and responses to the interviews that helped inform data analysis during coding and the selection of categories (Piantanida & Garman, 1999). This step provided transparency for researcher bias and increased reader confidence that interpretation of the participants' meanings was truly based on participant information rather than the researcher's assumptions or biases. Such credibility is one aspect of internal validity in qualitative research. Data Collection The primary sources of data for this study were semi-structured interviews, lasting approximately one hour, with 13 faculty development program directors and two online surveys, one for faculty development directors and one for administrators of the phone participants. The phone interviews were recorded and 80

PAGE 94

transcribed; then, the transcriptions were entered into the Nvivo computer program for qualitative analysis. The online director survey was hosted by zoomerang and asked directors to rank their practices, goals for their program, and obstacles that were barriers to achieving those goals. See questions in appendix. The raw data were then exported to an excel spreadsheet where they could be more easily interpreted and analyzed. The academic administrators' online survey asked about their support for faculty development and instructional improvement practices as well as their perceptions of the faculty development program at their school and its impact on faculty and students at their institutions. See questions in Appendix A. Lastly, website information was collected from participating programs. This information included structures, practices, and offerings ofthe program found in an online search of the program's website. A table was created to more clearly compare all data across institutions. Measurement Tools-Interview Protocol The open-ended interview questions were based on faculty development best practice literature as well as the research questions about what faculty development programs were doing, how their practices and structures compared and contrasted to best practice, and how their programs and practices might have influenced faculty approaches to teaching (Gibbs, 2005; Helyer & Boschmann, 1993; King & Lawler, 2003; Murray, 2000; Rust, 2001; Sorcinelli, 200 I). A panel 81

PAGE 95

PAGE 96

about which administrators would have special insights. These questions were also reviewed by a panel of faculty development experts and revisions and additions were made. Data Analysis Interview data were analyzed using a process, described in Creswell ( 1998) for analyzing phenomenological data. The first step was to read the transcribed interview data and extract significant statements, which are phrases and statements that pertained directly to the research questions. Next, using a coding scheme and NVivo, a software program for qualitative data analysis, these statements were analyzed for their inherent meaning and compiled into a list of categories or themes. The initial coding scheme was based on information from the literature about recommended best practice in faculty development as well as best practice in teaching and learning gleaned from the DEEP study and others. To improve qualitative reliability, or trustworthiness, several outside assistants from my doctoral lab reviewed the coding scheme and verified the categories and made suggestions. As the interview process progressed, additional categories emerged as important factors and were added to the list of codes. The final list of codes was reviewed by an expert in faculty development for appropriate organization of data and accuracy, however, because of time and work constraints, no inter-rater coding was done to strengthen inter-rater reliability. 83

PAGE 97

Finally, the emerging information, coded into thematic categories, was selected to thoroughly describe the participants' perceptions of how faculty development programs influence faculty approaches to teaching that contribute to successful student outcomes and persistence demonstrated by these DEEP schools. The phenomenological tradition is an iterative process of interviews and analysis, overlapping in time that gradually builds increasing understanding of the phenomenon in question, in this case, the relationship between faculty development and teaching approaches that contribute to student success. Additional categories and organization of data were revised as additional interviews provided new concepts. Using the above process, all transcripts were analyzed for meaning and themes that were common across the 13 schools and that formed the coding categories. The qualitative responses were analyzed using quantitative methods as well. The number of responses and percentage of participants responding in specific categories were counted and compared across all coding categories to provide a view of the relative importance of various themes and topics to the participants. The quantitative data from NVivo and both surveys were analyzed for areas of support for and divergence from the qualitative findings. In this way, precautions or substantiation of findings could be enhanced through the quantitative perspective. The online survey data from faculty development directors about their rating of practices, goals, and obstacles were compared and contrasted with the 84

PAGE 98

outcomes from similar studies of faculty development directors from two large national samples looking for best practices (Frantz eta!., 2005; King & Lawler, 2003). Means from each response to the questions were compared to means from the other two studies. Where means were not available, the responses were rank ordered and compared to the rank order from the other two studies. Patterns of agreement and divergence were noted. All the data, qualitative and quantitative, were viewed and analyzed through the lens of best practice in faculty development. How did faculty development practices and structures at these schools compare to each other, to the non-DEEP schools from other surveys and research, and to recommended best practice? What other data emerged to help explain the successful teaching practices at the DEEP schools? Triangulation of multiple sources, perspectives, and methods, described in this chapter, lent strength to the internal validity or credibility of these data and formed the basis for answering those questions which were then analyzed in subsequent chapters. This multidimensional approach lead to the findings described in chapters 4 and interpreted in Chapter 5. Methods Summary From the literature on teaching approaches that lead to higher-than predicted student outcomes and that of faculty development best practices that impact faculty approaches to teaching, this research study sought to explore the relationship between those two variables and the factors that might mediate the 85

PAGE 99

relationship. The methods used to examine director perceptions of those relationships included a phenomenological exploration of director interview transcriptions and a quantitative approach using online surveys of faculty development directors as well as a survey of their administrators' perceptions. This research design provided one means to evaluate the relationship among the research variables and postulate the conclusions described in Chapter 5. 86

PAGE 100

CHAPTER4 RESULTS Overview The purpose of this study was to explore the relationship between specific faculty development practices and structures and teaching approaches that contribute to student success in institutions with higher-than-predicted student outcomes. The following questions were designed to clarify how faculty development at high perfonnance colleges and universities might share some commonalities and also might differ from faculty development described in previous literature. These similarities and differences might then provide insights about best practices in faculty development. The questions driving this research were as follows: I. What are the common structures and practices among faculty development programs at institutions that produce better than predicted student outcomes (DEEP schools)? 2. In what ways are faculty development structures and practices at DEEP schools congruent with best practice as described in the literature? 3. What are the elements of faculty development structure and practice at DEEP schools that differ from best practice as described in the literature? 4. What is the perceived influence of faculty development best practice at DEEP schools on faculty approaches to teaching? 87

PAGE 101

Summary of Method In online surveys and phone interviews respondents were asked about their faculty development goals, structures, and practices. The survey and interview questions were based on a list of best practices in faculty development gleaned from the literature, described in chapter 2, and summarized here. The following is a summary of faculty development practices and structures common] y cited in the literature as "best practice" in effective faculty development programs. 1. Have clear goals connected to student performance and congruent with institutional mission and have detailed plans to reach those goals. 2. Involve faculty in setting goals, designing plans and assessment tools, and implementing activities. 3. Design and implement ongoing, engaging development activities with appropriate practice, follow up, and support for faculty. 4. Use appropriate tools to measure the effectiveness of their activities and programs. 5. Make appropriate changes as a result of the assessment feedback. 6. Develop connections throughout the institution to help support their mission and goals. The above-listed "best practices" were used as benchmarks to help determine how faculty development programs at the DEEP schools compared to each other and to programs described in other studies. The fonnat of chapters 4 and 5 was organized according to this list. Twelve ofthe twenty DEEP schools' faculty development directors agreed to participate in the online survey for a response rate of 60%. Thirteen of the twenty participated in the phone interviews for this research, for a response rate of 65%. The response rates of 60 and 65% for qualitative investigations provides a good 88

PAGE 102

picture of the DEEP faculty development directors. In addition, the respondents represented every category of higher education institution tracked by the NSSE reports: private, religious-affiliated; state, Hispanic focus; state, liberal arts, Jesuit, liberal arts; state, research I university; private women's; private men's; and state, historically black university. The variety of respondent institutions provides a rich picture of faculty development in higher education. There may have been some overlap among the survey and interview participants; however, the identity of the survey participants was unknown. The phone interviews were transcribed and coded using NVivo software for qualitative analysis. By comparing the responses about their goals, structures, and practices to best practice and to responses from faculty development programs at other schools that have been detailed in the literature (King and Lawler (2003), Frantz et al. (2005), Sorcinelli (2006)), similarities and differences emerged. The data described in this chapter will provide a basis from which to clarify these similarities and differences in Chapter 5 and investigate whether those similarities and differences might contribute to the successful teaching practices and leaming outcomes found on the DEEP campuses. A separate online survey for administrators at the DEEP schools was completed by four of the administrators of the twelve faculty development directors who agreed to the phone interview, for a response rate of 33%. The purpose of this survey was to confirm the responses from the faculty development directors and gain another perspective about the relationship between faculty development 89

PAGE 103

programs and school administrators. Their responses from one set of faculty development directors via online survey, a different set of faculty developers via phone interview, and a third group of administrators can provide a better look at what is true to the data than any single source alone. The described triangulation of data thus improves the accuracy of representation of the qualitative responses and increases confidence that the conclusions of this study emanate from the data and are accurate representations of the data and participants. It is similar to the internal validity of quantitative data. Program Descriptions The DEEP study intentionally selected 20 representative high-performing colleges and universities, from across institutional types and geographical categories, in order to demonstrate that there are high performing institutions in every category. Thirteen of the twenty DEEP schools responded to my request for an interview; these schools represent all the Carnegie categories and have been described in greater detail in Chapter 3. Pseudonyms were created for purposes of anonymity. The following table, 4.1, summarizes the schools and their faculty development programs. 90

PAGE 104

Table 4.1 Faculty Development Program Demographics (phone interview participants) School UnderSchool type FT faculty/ Years Director/ FD center/ (pseudonym) grad Total FT, PT years students program Midwest 2,245 Private, religious125/225 IS /assoc. NO women's affiliated, Vice Provost center/IS women's Paci fie state 3,618 State, Hispanic 120/300 1.5 FT Center/IS focus Pine state 3,901 State, liberal arts 250 6FT Center/IS Western 3,485 Jesuit, liberal arts 5115% time Ccnrter/4 Catholic U. FD Eastern U. 15,312 State research 1000/2024 3/PT Center/9 university Assoc. provost Central College 1,773 Private. liberal arts 163/263 1.5 FT Ccnter/10 North Central 14,720 State, Research 75011500 Program 20, Center/10 U. university center 10 (30 total) FT Atlantic 682 Private. women's 71 NO center Women's College Northeast State 2,000 State, liberal arts 1401180 FT 10 Center/10 Great lakes U. 23,000 State, Research I 3300 FT 17 Center/ 50 Texas 17.261 State, large 68011200 3 Center/IS University Hispanic. PT I Associate Research I provost Midwest Men's 853 Private, men's 85 NO center/ College committee Southeast state 2.320 State, HBU 350 2FT Center/8 university. 91

PAGE 105

To address questions one and two, what are the common structures and practices of faculty development programs at DEEP schools and how congruent are they with best practice in faculty development as described in the literature, this research used an online survey and phone interviews with faculty development directors at the participating schools. The following sections of this chapter will examine the evidence from the online surveys and interviews, in that order, relevant to each of the four questions. From this examination, salient features of successful faculty development programs emerged and formed the basis for the findings discussed in Chapter 5. Faculty Development Directors' Online Survey Responses Practices and structures of the participating faculty development directors and their programs were examined for convergence among the participants and also congruence with best practice in faculty development. Eleven areas examined in the online portion of this study included: faculty development goals, factors in achievement of their goals, obstacles to achieving their goals, successful practices in faculty development, characteristics of faculty development activities, measurement of the effectiveness of their activities, factors affecting the success of faculty development, assessment practices of faculty development directors, faculty development reporting structure at the institution, administrative support, and demographic factors. Questions in the survey were based on the Frantz et al. (2005) study in order to provide head-to-head comparisons of the DEEP cohort responses 92

PAGE 106

with those of the faculty developer responses from outside the DEEP cohort. In some instances, there was ambiguous language, such as in Table 4.6 where "workshop evaluations" and "satisfaction/value surveys" were choices to describe how the directors assessed their activity. In hindsight, these questions may seem too similar to accurately discern. My intent was to determine whether the directors asked "Did you enjoy the workshop experience" in the first case and "Was the activity helpful and will you use what you learned" in the second case. However, it may be that the participants had different interpretations. The phone interview asked similar questions with a different group of directors and that information helped to clarify DEEP cohort directors' perceptions about their structures and activities. With that caveat in mind, note the following tables, 4.2 4.11, present the directors' responses to questions about their faculty development programs, and clarify areas of agreement as well as unique responses. Tables 4.12-4.14 summarize responses from the administrators' online surveys. Faculty Development Program Goals One area where this research sought to find similarities and differences among directors of the faculty development programs at DEEP schools involved program goals: how important their goals were to the program, factors that helped them achieve their goals, and obstacles that were barriers in the achievement of their goals. The following table demonstrates the goals shared by these directors. 93

PAGE 107

Table 4.2 Importance of Program Goals to the Success of Facul(v Development Program. Goal Not Not Somewhat Very Extremely %Total at all very positive Improve student learning 0 0 0 4 8 100 Enhance value of teaching 0 0 3 8 100 effectiveness Improve learning 0 0 2 5 5 100 environment Introduce teaching 0 0 3 8 100 strategies Serve institutional needs 0 3 6 2 92 Provide faculty 0 3 4 3 91 development programs Provide skills training for 0 6 3 2 92 faculty Foster faculty career 0 2 5 4 83 development Serve faculty personal 4 3 4 0 58 needs Improve faculty 0 7 2 3 0 42 evaluations There was considerable agreement among faculty development directors about the goals of their program. "Improving student learning" was rated as very or extremely important by 12 of 12 respondents, followed closely by "enhance value of teaching effectiveness," rated very or important by II of I2 respondents, "improve the learning environment," with IO of 12 rating it as very or extremely important., and "introduce teaching strategies" with 9 of I2 rating it as a very or 94

PAGE 108

important goal. While "serving institutional needs" ranked fifth in the goal importance, 8 of 12 ranked it an extremely or ve1y important goal. While "providing skills training" and "providing faculty development programs" both had 91% and 92% positive responses, respectively, they had more responses in the somewhat important category, than in the ve1y or extremely important categories, compared to the higher ranked goals. The lowest ranked responses, "foster faculty career development," "serve faculty personal needs," and "improve faculty evaluations," had more somewhat important and not vel)' important ratings than the higher ranked responses. Investigating factors that influenced achievement of their goals, data again demonstrate areas of agreement among the 12 directors. Table 4.3 Factors h?fluencing Achievement of the Faculty Development Program Goals Factors Engaged and supportive faculty Adequate budget Strong administrative support Tradition of support and climate of collaboration Programs offered at appropriate I convenient times Table 4.3( continued) Table 4.3( continued) Not Not Somewhat Very Extremely Percent at all ver Total+ 0 0 2 9 100 0 0 7 4 92 0 0 3 4 5 100 0 2 5 4 83 0 2 2 4 4 83 95

PAGE 109

Factors Influencing Achievement of the Faculty Development Program Goals Factors Not Not Somewhat Very Extremely Percent at all very Total ositive Skilled and dedicated staff 2 0 4 5 83 Providing food and 0 3 3 4 2 75 refreshments Grant funding 2 2 5 2 75 Strategic planning and goal 0 3 3 5 75 setting Location and physical 2 4 5 0 75 facilities Student support 4 3 3 2 0 42 Including a variety of delivery 3 2 3 3 0 50 methods The data in Table 4.3 indicated that II of 12 directors believed that "engaged and supportive faculty" as well as "adequate budget" were very or extremely important factors in the achievement of their program goals. "Strong administrative support," "tradition of support and climate of collaboration," and "skilled and dedicated staff' were factors that 9 of 12 directors believed were ve1y or extremely important to the achievement of their goals. The lower ranked factors included "providing food and refreshments," "strategic planning and goal-setting," as well as "location and physical facilities". While those four factors each had 75% positive responses from the directors, the totals were mostly in the somewhat important category rather than in the very or 96

PAGE 110

extremely important categories. The lowest ranked factors, "student support" and "including a variety of delivery methods" were fairly evenly divided between positive and negative ratings. The pattern that began to emerge of the importance of faculty, staff, administrators, and their relationships, is one that will be evident in many of the subsequent responses in the online surveys and the phone interviews. This pattern and its implications will be investigated further in Chapter 5. The responses to perceived obstacles that hindered the faculty development directors' achievement of their goals demonstrated a different pattern. Table 4.4 Obstacle Hindrance to Achievement of Faculty Development Goals Obstacle Not Not Somewhat Quite Strong %Total at all much positive Lack of time faculty could 0 0 3 4 5 100 devote to FD activities Lack of adequate staff 3 2 2 4 67 support Lack of faculty support or 2 5 3 42 interest Research culture rather 4 2 3 50 than teaching culture Budget constraints 2 3 6 0 58 Lack of administrative 6 3 2 0 25 support 97

PAGE 111

The results indicate that only two of the seven choices were perceived by the majority of respondents as hindering the achievement of the faculty development goals. "Lack of time faculty could devote to faculty development activities" was perceived as quite an obstacle or a strong obstacle by 9 of 12 participants and "lack of adequate staff support" was perceived as quite an obstacle or a strong obstacle by 6 of 12 participants. It is worthwhile to note that some of the obstacles commonly assumed to be a hindrance to achievement of faculty development goals, were perceived by this group as not much or not at all a hindrance to the success of their programs. Factors such as "lack of faculty support," "research culture rather than teaching culture," "budget constraints," "lack of administrative support," and "facility problems" were not perceived as obstacles to their success. Analysis of the interview results, later in this chapter, will clarify whether these factors didn't exist on the campuses of these participants or whether they existed but weren't considered obstacles. Practices Used in Faculty Development Programs In an attempt to discern common practices and characteristics leading to success of faculty development programs, the directors were asked to rate the perceived importance of selected practices and activities. The data in the following table indicated the perceived importance of particular practices to the success of faculty development programs. 98

PAGE 112

Table 4.5 Importance of these Practices to the Success of the Faculty Development Program Practices Not Not Somewhat Quite Extremely %Total at all very positive Establish a positive 0 0 2 9 100 climate for teaching and learning Providing resources 0 0 5 6 92 Network among faculty 0 0 3 3 6 100 across departments Collaboration among 0 0 3 3 6 100 faculty Establish learning 0 0 4 5 3 100 communities T caching improvement 0 0 6 3 3 100 workshops Being visible 0 2 2 5 3 83 Adapting and assessing 0 3 3 5 75 needs Providing faculty 0 2 6 4 0 83 mentoring Providing technical 4 5 0 75 support Help write grants 2 3 2 4 58 Personal assessment 4 3 3 58 programs The most important practices that 11 of 12 participants agreed were quite or extremely important to the success of their program included "establishing a 99

PAGE 113

positive climate for teaching and learning" and "providing resources" for their faculty. The next most important practices included "networking among faculty across departments" and "collaboration among faculty," both with 9 of 12 directors agreeing they were quite or extremely important. "Establishing learning communities" while ranking fifth overall, had 8 of 12 directors agreeing it was quite or extremely important to the success of their faculty development program. These results indicate that four of the top five practices involve building relationships among faculty. The director's responses add evidence to the importance ofbuilding a community of support through faculty development efforts rather than deploying their time and energy on individual faculty. The interview responses clarify and add support to that evidence. Characteristics o.f Faculty Development Activities Six characteristics in faculty development activities emerged as the most important contributors to the success of these activities. Table 4.5 displays director responses about those characteristics. 100

PAGE 114

Table 4.6 Importance of Characteristics that Contribute to Success of Faculty Development Activities Characteristic Not Slightly Moderately Quite Extremely %Total at all positive Participant interaction 0 0 0 4 8 100 Time for participant 0 4 3 4 92 reflection Planned in collaboration 0 0 2 5 5 100 with faculty Peer collaboration in 0 6 4 92 activity Prompt feedback for 0 5 4 2 92 participants Follow up sessions/ 0 5 3 2 91 additional information Rewards for changed 3 4 3 42 behaviors Follow up support for 2 3 3 3 58 new behaviors Guided practice for new 5 4 2 58 behaviors "Participant interaction" was judged to be the most important activity characteristic contributing to success; 12 of 12 respondents agreed it was quite or extremely important. Collaboration was the next most agreed upon characteristic of successful activities. Ten of twelve participants agreed that "planned in collaboration with faculty" and "peer collaboration in the activity" were quite or extremely important to the success of their activities. "Time for participant reflection," and "prompt 101

PAGE 115

feedback for participants" were found to be the next most important characteristics, with 7 of 12 and 6 of 12 participants, respectively, agreeing that they were quite or extremely important to the success of their activities. While "follow-up sessions or additional information" was ranked seventh in importance, 11 of 12 participants perceived it to be somewhat (5), quite (3), or (2) important. Once again, the relative importance of interaction, networks, and collaboration in the responses to this question adds to the evidence for community building and group process in the DEEP faculty development programs. Assessment In contrast to research findings from faculty development literature described in Chapter 2, faculty developers in this study use a variety of measures to gain information about the effectiveness of their programs. In response to the question about assessment in the online survey "Please indicate by the following scale, how you assess the effectiveness of the faculty development program on your campus," the directors' aggregate responses included the following data in Table 4.6. They used a variety of assessment methods and they used them with a relatively high frequency. Table 4.7 102

PAGE 116

Faculty Developer Use of Assessment Strategies to Measure Effectiveness of their Programs Assessment Never use 1-2 times 3-5 times 6-9 times > 10 times Total use% Number of workshop 0 0 6 4 92 participants Workshop evaluation 3 5 83 Institutional needs 4 4 3 0 67 assessment Frequency of contact 3 2 3 58 with faculty Strategic planning 2 5 0 2 0 58 andProgram reviews Satisfaction /value 4 4 2 0 58 surveys Focus groups 4 0 2 2 2 50 Program exit surveys to 5 0 4 50 evaluate experience Periodic interviews 4 2 42 Workshop learning 6 0 2 33 assessments Number of subscribers to 8 0 0 17 newsletter Note: My intent was to distinguish between "satisfaction" as assessment of an entire program or series and "workshop evaluation" as assessment of a one-time workshop, this distinction may have been too ambiguous for clear responses. Previous faculty development literature suggests faculty developers rarely measure outcomes of their programs, and those that do use a measurement, most 103

PAGE 117

commonly use a satisfaction survey (Frantz et al., 2005). Compared to those previous studies, the DEEP faculty development directors do a substantial degree of assessment. The most frequently used assessments were: number of participants, workshop evaluation, institutional needs assessment, frequency of contact, strategic planning, satisfaction value surveys, focus groups, and program exit surveys, in that order. In the eight most commonly used assessments, at least half of the respondents used each assessment between 34 and more than 10 times in the past two years. So, not only did this group use more types of assessment than those directors studied in other literature, but they used these assessments at a high frequency. Not only did the participants measure the effectiveness of their overall program but they also measured each activity. When asked the question "How do you determine the effectiveness of your activities?" the respondents enumerated in Table 4.8 a variety of assessment strategies that they commonly and frequently used. "Participant satisfaction" was sometimes, commonly, or always measured by all the participants. "Institutional impact," "faculty behavior change," and "faculty knowledge change," was sometimes, commonl_v, or always measured by more than 66% of the participants. "Student performance," faculty attitude change,'' and "student retention" was sometimes, commonly, or always measured by at least half of the participants. The degree and frequency of activity assessment is in congruence with recommended best practice but differs from the practice of faculty 104

PAGE 118

development directors in previous studies. To reiterate, only 20% of faculty in the Frantz et a!. study (2005) used any assessment at all and most of those measures were satisfaction surveys. Table 4.8 Commonly Measured Outcomes to Determine Effectiveness ofF acuity Development Activities Assessment Never Rarely Sometimes Commonly Always Total use Participant 0 0 8 3 100 satisfaction Institutional impact 3 0 7 2 83 Faculty behavior 3 0 3 3 3 75 change Faculty knowledge 5 3 2 2 67 change Student performance 5 0 3 2 2 58 Faculty attitude 6 0 2 3 50 change Student retention 6 0 3 3 0 50 A follow up question asked how they measured faculty and student changes indicated in Table 4.8. The most commonly used instrument to measure these changes was a questionnaire they constructed: 3. 92 mean ( 12/12 answered sometimes, frequently, or ahvays). Some faculty development directors also used student faculty evaluations: 2.82 mean (5/12 sometimes, .fi-equently, or always). A small percentage (3/12, 25%) commonly used the student knowledge survey to 105

PAGE 119

measure outcomes of particular faculty development activities. One school's director sometimes used the CEQ, TMI, and A TI inventories and one director a/v.:ays used the SEEQ. Contrasted to the previous literature, the faculty development directors in the DEEP schools demonstrated a substantial difference in assessment practices. All the DEEP directors surveyed measured satisfaction but, more importantly, a high percentage regularly used several other assessment methods as well. Eighty two percent of our respondents measured institutional impact, primarily through a variety of whole-school inventories such as the NSSE survey; this is not surprising since all of the DEEP schools were drawn from NSSE participants. Director Perceptions of Support from Administration Funding was only one of the ways that the directors demonstrated support from their administrators. Other modes of administrative support were investigated through the question, "To the best of your knowledge, how do people in your school administration support the faculty development program?" Table 4.9 demonstrated the degree of administrative support for faculty development programs in several venues. Table 4.9 Examples of FD directors' perceptions of Administrative Support for Faculty Development Programs. 106

PAGE 120

PAGE 121

suggested in the questions, participants may have interpreted these questions differently. The questions for the online survey were selected from the Frantz et al. (2005) and King and Lawler (2003) studies specifically to compare responses from the DEEP directors with those from non-DEEP schools. While one question addressed the importance of 10 specific goals to the success of their faculty development program, there was not a choice that specifically reflected collaboration. However, one choice that was loosely linked to cross-campus collaboration was "Serve institutional needs". That selection garnered the following responses: somewhat important, 2/12; very important, 6112; and extremely important, 3112 for a total of 92% of respondents who believe that serving institutional needs contributes to the success of their faculty development program. Yet again, collaborative efforts are perceived as important to the success of these faculty development programs. Further evidence of this trend was noted in the open-ended responses to the survey questions from two faculty to the question of the importance of goals to the success of their program. They added the following goals in their response: build faculty networks and collaboration, help faculty understand profession and share enthusiasm for work, build research collaborations, expand academic interests, deepen pedagogical understanding of interdisciplinary study and create approaches for collaborative teaching (with teaching partners) and students. 108

PAGE 122

Another indicator of the faculty development directors' strong support for collaborative efforts is found in question 11, which states: "Please identify the importance of following practices to the success of the faculty development program on your campus." The research participants ranked the three choices reflecting collaboration among their top five practices important to achievement of their goals out of a total of twelve choices. Their responses are detailed in table 4.10. Table 4.10 Importance of Collaborative Practices to the Success of Faculty Development Program Practice Not at all Slightly Somewhat Very Extremely Total% positive Collaboration among 0 0 3 3 6 100 faculty Network among faculty 0 0 3 3 6 75 across departments Establish learning 0 0 4 5 3 67 communities When asked, in online question 13, to identify the importance of the following 12 factors to achieving their faculty development goals, participants ranked "Tradition of support and climate of collaboration" fourth, out of twelve factors. The mean score was 4.14 out of 5 for this factor. Individual responses were: extremely important 4112: ve1y important 5/12; and sonwtdwt important 1/12; for an 83% total positive response. 109

PAGE 123

Another online question, 26, looked at evidence of collaborative practice in the faculty development activities. Asked, "To what degree do the following characteristics of your development activities contribute to the success of these activities," the respondents ranked "planned in collaboration with faculty" and "peer collaboration" in the top three of nine choices. Table 4.11 details their responses to those two choices. Table 4.11 Importance of Collaboration to Success ofF acuity Development Activities Best practices Not Not Somewhat Very Extremely Total% at all very positive Planned in 0 0 2 5 5 100 collaboration with faculty Peer collaboration 0 6 4 92 in activity The sixth question dealing with collaboration, Question 21, examined the attitude toward collaboration among faculty. When asked how open their faculty were to collaborative teaching, the respondents indicated a mean score of 4.0 I 5. Individual responses were, somevvhat open 6112, ve1y open 3112, for a total positive response of 75%. These results highlighted from the previous six survey questions were indicators of the importance of collaboration on the DEEP campuses in this study. They included the importance of collaboration in achieving their program goals and 110

PAGE 124

in the success of their program, as well as the importance of collaboration among faculty in faculty development activities and across campus departments. The phone interviews, described later in this chapter, added strength to the importance of collaboration found in the survey results. Administrators' Online Survey Responses A survey of administrators of the DEEP directors who participated in the phone interviews confirmed their directors' reports about faculty development goals, practices, impact of their programs, and relationship with administrators. While only 4 of the 12 (33%) invited administrators responded, their collective feedback supported the viewpoints of the faculty development director participants. The triangulation of data added validity to the responses from the faculty developers. The group represents several types of institutions: one large, state research university; one small, private, liberal arts college; one small, state, HBCU; and one small, Jesuit University. The convergence of responses from such diverse institutions lends support to their collective confirmation of director responses. Administrator responses mirrored the directors' positive perceptions of their programs and relationships. The first set of questions in Table 4.12 was used to discern administrators' attitudes about teaching. From literature studies, we know that there are individuals who believe that excellent teaching depends on a set of characteristics that some 111

PAGE 125

people have and some don't. Others believe that excellent teaching can be learned. Clearly, individuals who believe excellent teachers are born, not made, might be less supportive of faculty development efforts and programs. Table4.12 School Administrators' Perceptions of Teaching and Faculty Development Attitude Strongly Disagree Neither agree Agree Strongly Total disagree Nor disagree agree positive Teaching 0 0 0 3 100 performance connected to rewards Champion teaching 0 0 0 2 2 100 excellence Excellent teaching 0 0 0 2 2 100 learned Excellent teaching 2 0 0 0 based on characteristics Faculty development 3 0 0 0 0 and excellent teaching not my job The administrators' responses indicate that they believe excellent teaching can be learned. Thus, these administrators are likely to supp011 faculty development efforts. They also indicate that faculty development and teaching quality are definitely pa11 of their job which adds to the likelihood that they would support faculty development efforts on their campus. The second set of questions for administrators included questions about their faculty hiring practices. Some colleges and universities hire faculty based on their 112

PAGE 126

scholarly publication record and are not concerned about their teaching experience or expertise. The hiring practices of these administrators, however, support the goals of excellent teaching found at the DEEP campuses. All the administrators supported a variety of ways for faculty candidates to demonstrate evidence of excellent teaching. Table 4.13 Use of Required Hiring Practices Related to Teaching Practice included in Never Occasionally Half Frequently Always %Use hiring process the time Teaching 0 0 2 100 demonstration Statement of 0 0 2 100 teaching philosophy Evidence of past 0 0 0 4 0 100 excellent teaching Teach a class with 0 0 2 100 students Input from faculty 0 0 2 2 0 100 development director or staff The results from this group of administrators indicate an emphasis on teaching during the hiring process. This may be related to the teaching excellence found at the DEEP schools. It certainly described administrators who believe excellent teaching is an important factor in faculty candidates. The third set of questions for administrators in Table 4.14 was used to determine their perceptions of factors that contribute to the excellent teaching found on their campuses. These questions also included the degree to which faculty 113

PAGE 127

development was perceived to contribute to teaching excellence. All four administrators agreed definite(v or strongly that faculty development programs helped their faculty learn excellent teaching, created a climate of excellent teaching, and were pivotal elements in creating a norm of excellent teaching on their campuses. It is interesting to note that although these administrators attempted to hire excellent teachers, they perceived those excellent hires as the least important factor contributing to the teaching effectiveness on their campus; whereas, faculty development ranked in 3 of the 4 top factors. Table 4.14 Contribution of Factors to Teaching Effectiveness on their Campus Factor Not Slightly Moderately Definitely Strongly Mean at all Faculty development 0 0 0 3 4.75 programs help our faculty learn excellent teaching Focus on good 0 0 0 3 4.75 teaching sets high standard Table 4.14 (continued) Contribution of Factors to Teaching Effectiveness on their Campus Factor Not Slightly Moderately Definitely Strongly Mean at all Faculty development 0 0 0 2 4.70 programs create a climate of excellent teaching 114

PAGE 128

Faculty development 0 0 0 2 2 4.50 is a pivotal element in creating norm of excellent teaching Campus climate 0 0 0 2 2 4.50 encourages teachers to strive for improvement Reputation attracts 0 0 0 3 4.25 faculty who value excellent teaching We hire excellent teachers 0 0 3 0 3.75 The evidence from administrator responses indicates they had strong beliefs that faculty development was a critical part of the excellent teaching found on their campuses. The analysis indicates that these administrators believe the most important way to get excellent faculty is to grow them through opportunities provided by their faculty development programs and, secondly, to use hiring practices that illuminate a candidate's teaching effectiveness. This mutual respect and admiration between faculty development directors and their administrators was explored and expanded in the phone interview results. Faculty Development Directors' Interview Responses The interview questions were structured around the principles of effective faculty development in order to elicit responses from the participants that would determine their use of these practices and perhaps highlight other details. The participant responses were recorded, transcribed, and then coded by themes in 115

PAGE 129

NVivo. A list of relevant theme categories was then created for each research question. Those responses were then analyzed for relevant examples to use in this chapter. See Appendix A for the themes and node categories. As detailed in chapter 2, a summary of best practices in faculty development includes the following. I) Have clear goals connected to student performance and congruent with institutional mission and have detailed plans to reach those goals. 2) Involve faculty in setting goals, designing plans and assessment tools, and implementing activities. 3) Design and implement ongoing, engaging development activities with appropriate practice, follow up, and support for faculty. 4) Use appropriate tools to measure the effectiveness of their activities and programs and make appropriate changes as a result of the assessment feedback. 6) Develop connections throughout the institution to help support their mission and goals. The above-listed set of best practices were used in this chapter as benchmarks to help determine how faculty development programs at the DEEP schools compare to each other and to programs described in other studies. This chapter is organized by the six categories of best practice, in that order. Goals 116

PAGE 130

The first category of effective practice describes goals: goals connected to student performance, faculty involvement in planning goals, and alignment of program goals with curriculum and institutional mission. Recalling that "improve student leaming" was the most important goal of 12 of 12 faculty development directors in the online survey, it is not surprising that 11 of 13 directors in the interview discussed goals which accounted for 29 responses. Goals Connected to Student Pe1:{ormance Some schools, such as Pacific State and Midwest Women's, explicitly connected their faculty development goals to student leaming. Curriculum is driven by leaming outcomes and all teaching and teaming and faculty development efforts revolve around that goal improved student leaming [which is the institution's primary goal]. Good teaching is the expectation because we need to get to our leaming goals ... l certainly think it's [faculty development] an important venue for ... making the commitment meaningful through ... implementing practices that support the vision. Pacific State Other schools described their goals in terms of reaching more faculty and providing more opportunities for them to explore their teaching practice; the underlying implication was that students would ultimately reap the rewards through improved teaching and teaming. And at the center you might say I throw a spot light on interesting, innovative excellent kind of teaching practices .... we're trying to show faculty ways that they can bring their scholarly lives to bear in the classroom, and the reverse. Eastern University 117

PAGE 131

Some of the 29 responses included efforts to adapt the faculty development offerings to the variety of faculty including adjuncts and those with limited time for development activities. At Texas University, "I offered them just a three-hour orientation, instead of three days ... and everybody came almost. So it became clear to me that I need to be offering an altemative for the non-tenure line". In order to provide a safe environment for teachers to try out and learn better teaching methods, several faculty developers were clear about the need to separate their work with faculty evaluation from the evaluation for promotion and tenure: My goal is to be about formative assessment for faculty. But I don't want to be seen as the person who is, or isn't thinking somebody is tenure [based on] my classroom observation ... we are very clearly a formative place. Eastern University. Other goals of the faculty developers included creating more collaborative methods of inquiry and change such as faculty learning communities. In this study faculty learning communities imply an ongoing, cross-disciplinary group of faculty who engage in an active, collaborative program with a curriculum about enhancing teaching and learning and community building (Cox, 2004). The director at Western Catholic University described his segue into learning communities. We've had informal faculty learning communities in the past, although not for quite some time. But I want to be much more intentional about it in providing money to send people, groups of faculty to conferences for 118

PAGE 132

example, to help support them in development of collaborative teaching efforts. Western Catholic While many of the goals of the directors in this study were congruent with best practice, they ultimately aimed to improve student learning and outcomes through improved teaching practices that were common at the DEEP schools. The director at Texas University provides an example of that connection, "The commitment of the faculty to helping the young people in this region is, you either buy into it, or you leave ... you're the person who helps students learn". Goals Aligned with Curriculum and Institutional Mission. A common thread that ran through most of the respondents' interviews was the importance of the institutional culture on faculty, faculty development, and student outcomes. Campus culture seemed to play a big role in the way faculty development operated on each campus. Several of the respondents, such as this director from Eastern University, mentioned building a culture of teaching excellence as another way that they impacted the quality teaching on their campus. It's some of those kinds ofthings [teaching award celebration] that, they're not impacting right now, success via teacher practices, but what it is doing is its calling attention to the way that teaching is valued on our campus ... It's not sufficient, but it's one of the things that's really important to creating a culture where teaching is something that's visible and talked about and noticed. Eastern University Fifty eight percent of the DEEP schools had faculty development programs or centers that aligned with the campus culture and mission, thus producing a 119

PAGE 133

synergy that enhanced teaching and learning at the school. Seven of the twelve respondents talked explicitly about aligning their faculty development goals to the culture and mission oftheir school. At Midwest Women's College, There is a commitment to a common, coherent, developmental curriculum on the part of all faculty. That means that there's a kind of language we're using with students, that students are transferring learning across courses ... there's a common set of abilities that comprise our core and that everybody commits to teaching and assessing. Midwest Women's College So, faculty development at Midwest Women's College involves mandatory, weekly discussions about the seven core abilities that are part of the institutional mission: analysis, problem solving, valuing and decision making, social interaction, developing a global perspective, effective citizenship, and aesthetic engagement. The constant attention to those goals helped them become embedded in the curriculum, the language, and the culture of the entire institution, including the faculty development program. At Western Catholic University, where the Jesuit mission of''service, ethics, justice, faith, and leadership" formed the foundation for the entire educational experience, faculty were concerned with the "formation'' of their students and themselves. And it [formation] is an individualistic and personally guided journey. So one of the key characteristics of the way faculty at Western Catholic University address or approach students is seeing them as individuals on a journey and that everybody, each individual's formation as a whole person, is going to be a little bit different. Western Catholic 120

PAGE 134

And so, the faculty development center at Western Catholic University became the Center for Teaching and Advising, since the role of advising, of guiding the students through "formation," was so highly valued. Unlike most other colleges and universities, many of the faculty development programs at Western Catholic University revolve around successful student advising. At Texas University, where the institutional vision was a strongly articulated commitment to access and equity for the underserved Hispanic student population in the southwest Texas region, faculty development efforts were aimed at helping faculty learn to teach in ways that promote success for the particular student population. And Texas University's mission of access and excellence is part of the fabric of this place. And the commitment of the faculty to helping the young people in this region is, you either buy into it, or you leave ... you're the person who helps students learn ... what can you do to help students get this stuff? Texas University The focus on "access and excellence" was found across the institution (Kuh, 2005) and the faculty development program at Texas University helped promote it. At Central College, a selective liberal arts school with a mission to help students grow and develop into leaders who will "change the world for the positive," faculty development efforts were aimed at helping faculty teach higher order thinking and writing skills, which are campus-wide goals. We [faculty development center] collaborate with ... the Dean of Academic Programs, particularly around writing instruction and doing 121

PAGE 135

faculty development for faculty who teach first year courses ... where most of the writing instruction is supposed to happen ... we encourage faculty, both in our requirements and in what kind of programming we offer here .. [to do] assignments where students have to revise and resubmit and meet in peer groups. Central College And at Southeast State University, a historically black university that admits most applicants, the institution and the faculty development program were determined to help all students succeed. So, the university invests not only in recruiting excellent students, but also in improving the skills of those that are not so excellent .... so when the students have low profile, we need to raise the profile and when we have raised the profile, then they become, like, at higher levels.[The faculty] know that that is their business. Their business is not to teach but to help the students learn. Southeast State Unlike many higher education institutions, where faculty and administration have an adversarial relationship (Blackburn& Lawrence, 1995), faculty at these schools bought into and supported the goals of their administrators: to provide high quality education for all students. While this is a goal common to many higher education institutions, what was different at these schools was the synergy between the administration and the faculty development programs. Faculty developers at the participating schools helped promote administrative goals and helped faculty learn the skills to realize those goals. One of the keys to this synergy was to involve faculty in the planning and delivery of programs for faculty development. Faculty Involvement and Ownership 122

PAGE 136

The second area of best practice, faculty involvement in planning and implementing faculty development activities, has strong support in the literature. Faculty development programs demonstrating the greatest success were characterized by a strong collaboration between faculty and administration in the planning and operation phases ofthe development programs. Where such collaboration was lacking, programs offered a disconnected smorgasbord of activities without a widespread or lasting impact on the institution (Eble & McKeachie, 1985; King & Lawler, 2003; Murray, 2000; Sorcinelli, 200 I). Faculty developers responding to the online survey indicated strong support for collaboration from faculty: their mean score for this topic was 4.43 out of 5. Twelve of the twelve respondents described planning in collaboration with faculty and peer collaboration in the programs as somewhat, very, or extremely important.. ln the interviews, 12/12 participants had 57 (fourth highest number of comments) responses about faculty participation in and ownership of faculty development efforts. Almost all the participants included faculty in planning and implementation of programs and activities; most of the programs are planned and run primarily by faculty. The following selections provide examples of the different ways faculty were involved in the design and implementation of faculty development programs. Some programs, like those at Midwest Women's, Pacific State and Western Catholic, developed in response to faculty needs. The director at Midwest Women's stated, "It's [faculty development programs] something that grew up as a 123

PAGE 137

solution to problems and issues and things ... It very much is faculty created". At Pacific State the director explained how her faculty commonly approach her with requests for programs. Now more often than not people are coming to me and saying you know we'd like to do a workshop on this with the faculty. Or could we have a session on this? Could we run these concurrent sessions twice instead of once? You know so it, it's really grounded in what people think is important as opposed to my sort of trying to convince people that it's a good idea. Pacific .State University The director at Western Catholic also described faculty requesting and initiating the programs. I often have faculty members who will approach me and say hey here's something I'd like to do, often it's a one-time workshop kind of thing that they will want to do and the CT A will sponsor it. So a lot of our programs are, well all of our programs are faculty driven. Western Catholic University At some schools, the faculty development center also acted as facilitator for activities and programs that were developed and lead by faculty in collaboration with the center. We also have a faculty learning community for every general education course; these are collaborative as well. The faculty development center sets these up from faculty input, and helps faculty facilitators to lead them. Pacific State University Another way these faculty development programs involved faculty was through participation of invited faculty on advisory boards and advisory committees. At Eastern University, the boards were comprised of faculty who had won teaching awards. 124

PAGE 138

PAGE 139

follow up, and support for faculty. We know from adult education and learning literature that active participation, learning in small groups with immediate feedback, engagement in the process, choice in types and content of learning opportunities, and strategies to enhance motivation have a positive effect on faculty attitudes, beliefs, and willingness to try new teaching behaviors (Bess, 1997; Brookfield, 1995; Hativa & Goodyear, 2002; Kember, 1996; Sorcinelli et al., 2005). One way to understand participants' use of effective elements in their activities is by the number of responses summarized in NVIVO for each of the best practice categories. The following list includes the number and percentage (in parentheses) of interview participants talking about the elements included in active, engaging faculty development activities: activity is Ieamer focused (7/12, 58%), activity encourages participant engagement (11/12, 92%), feedback from activity for participants ( 12/12, 1 00% ), opportunities for ongoing intervention as opposed to a one-shot workshop (l 0/12, 83% ), and providing support and rewards for participants in our faculty development activities (7/12,58%). These numbers suggest that a majority of director participants were teaching and modeling active learning principles, participant engagement, ongoing interventions with follow up, and providing support for their faculty participants. Faculty development directors' responses from the phone interview add qualitative support to the numbers above. At Midwest Women's College "we create opportunities to learn together about what we care about as teachers." Faculty 126

PAGE 140

developers at North Central University, where the director pioneered Faculty Learning Communities, have found this engaging, ongoing, faculty-led intervention to be their primary vehicle for faculty development. ... faculty learning communities ... have kept faculty talking to each other across disciplines; they get faculty out of their silos and ... give them a vehicle to talk about teaching and learning. North Central University At Northeast University, a small liberal arts state school with an emphasis on teaching and nursing, much of the faculty development involves collegial discussions about what faculty are doing in their classrooms, sharing ideas and practices, and learning new approaches to teaching . . many of them [faculty development programs] are faculty driven. And I think that there is a lot of opportunity for conversation and discussion. From a teaching perspective, sometimes you think that's not the most productive use of the time, but it actually is. You know, the opportunity to share what's going on in your own classroom. Northeast University At Texas University, a large, state university, the faculty developer began a program that was definitely not in his area of expertise, but was an engaging topic to an underserved group of faculty: mothers . . we started a 'Mama PhD' faculty learning community in the fall. "Mama PhD," there's a book out under the title "Mama Ph.D." And one of my colleagues picked up on it. And so we bought a number of books and, we put the word out, and there was a lot of interest from women faculty who were moms. Texas University Engaging the faculty in the learning programs and activities is the first step in helping them understand teaching and learning in different ways. It lays the 127

PAGE 141

for faculty to incorporate active teaching strategies into their teaching Jractice through the use of the active teaming principles modeled by the DEEP 1lculty development programs. Two examples of the engagement evidenced in the ievelopment programs were from Midwest Women's College and Western :atholic University. I think people would be amazed at the energy and intellectual work and excitement that goes on once people are in those meetings. But it's because they're doing something that makes them feel like they're accomplishing something in relation to student learning and what they care about as teachers. Midwest Women's College ... several of our faculty went a couple of years ago to his summer workshop. So we're thinking about trying to incorporate that and some of these same ideas of high impact teaching practices, what kinds of things matter, and creating significant learning experiences in the classroom. Western Catholic University When faculty at these schools are excited about and energized by their participation n faculty development activities and programs, they may likely bring that energy md excitement into their classrooms in their own teaching. At North Central University, the director describes the results of engaging n a faculty learning community, a year-long cohort of self-selected faculty that ;tudy a topic of their interest and document the changes they make in their :lassroom and then report the outcomes to local faculty as well as at national ;onferences. 128

PAGE 142

They present this on campus and then off-campus at regional teaching conferences or maybe disciplinary conferences, so because of this intensity, it really does make a difference, it's just not a workshop. North Central Pine State College in Washington was begun by faculty and is currently run entirely by faculty who regularly switch their teaching hats for administrative hats in a planned rotation of responsibilities. All teaching is done collaboratively, in teams, and requires hours and days of extra communication and planning. So as the faculty developer at Pine State College concludes," Here, teaching IS faculty development." And so the team members are planning in the summer. There are ongoing planning meetings, all during the year, every week. In fact members are fine tuning the program and dealing with whatever has emerged. But all of our faculty are also obligated to participate in what we call a faculty seminar. And that's where the team meets every week. Pine State Other schools use action research to engage their faculty in thinking about their teaching practices, investigating what works, then sharing their results with a broader audience. At Southeast State University, where much of their funding comes from Title III grants, the faculty developer created a plan to engage his faculty in improved teaching practices through action research . . initially we started just supporting the participation in teaching and learning conferences and things like that. And then we requested to share what they learned, in order to disseminate the type of things that they were getting. The next step was to engage them in doing [action research]. Southeast State 129

PAGE 143

Through these incremental steps, faculty development at Southeast State University grew into a strong scholarship of teaching and learning program which will be discussed further in the section about assessment. One of the outcomes of Ieamer engagement and participant interaction, in these faculty learning opportunities, was that faculty found the topics that concerned them and through their ongoing discussions came to resolutions that eventually impacted the culture at the institution. For example, at Northeast University, the development center lead book discussions with faculty that grew into a revision of the student evaluation system at that institution. Last year we did a whole series of meetings around Ken Bain's new book "What the Best College Teachers Do". And we did a reading group, and discussion groups, and then we did a whole other second round of them. But what came out of it was some concerns about how we do course evaluations ... they basically looked at best practices in student evaluations and collected student evaluations from all departments and looked for common questions ... they created a new version of the course evaluation that's not saying that cheesy question of 'do you think this faculty member is qualified in their field? Northeast University This is just one of the ways faculty development impacts best teaching practice at the DEEP schools. The last part of this third area of best practice in faculty development is building support and recognition for the faculty change process into faculty development activities and programs; that is, creating focus on talent development, not remediation. It makes sense that if faculty developers were asking faculty to take a risk and change something about their teaching, it would be helpful to create 130

PAGE 144

a collegial climate of support and provide appropriate rewards and recognition for their efforts. In the phone interview, almost half of our respondents (7 I 13) talked about providing support for their faculty who participate in faculty development programs. At Western Catholic University, their support was in the form of certificates for advisors who completed an advising development program. Their director explained, "We call it the advising academy. And when advisors complete that, they get a little certificate. The idea is that they can use that ... when they go up for promotion or tenure or reappointment." At Central College, the support came from course relief and buying faculty development resource books for their faculty. They come for lunch and in exchange for coming, every week for a year, they get a course relief in their first year of tenure track teaching ... to the extent that I could, I will give them money, I will buy them books. Central College At North Central University, the support was evident in a highly visible recognition ceremony with certificates and administrative support. They have a multi-dimensional support, not only of these venues to develop innovations, but dollar-wise support ... we have a recognition ceremony every year, where the provost comes, and a provost signs certificates for every member who's been in a community that year, everybody who's received a grant, so there's this big recognition that this is important. North Central University At many of these institutions, the directors described faculty as working harder than ever before and these directors tried to take that into account, not only 131

PAGE 145

through supportive practices, but also with planning, programming, and incentives to support faculty. At Pine State College, the director voiced her concern about how hard her faculty work. And it's something I'm always ... aware of and concerned about for them, how are they balancing everything? ... do they need support? Is there intervention we can do early that might help them before it becomes a problem? Pine State The respondents were aware of faculty strengths and areas of need and worked to provide the support they needed, financial or otherwise. The director at Southeast State University, described his faculty as knowledgeable in their content areas but not necessarily in pedagogy. Everybody here knows the content, but not necessarily everybody here knows to teach it. And that is why we support many faculty with their issues, with technology workshops, with summer institutes ... funding has been critical in that regard, because we are being able to fund nearly one hundred faculty ideas in small grants. Southeast State The respondents in this research demonstrate a strong commitment to all the best practices in what they model, what they teach, and how they reward their faculty. From the responses described in this chapter it was clear that these directors truly cared about their faculty and worked hard to provide them with everything they could to help them become successful educators. The positive attitude of the directors toward their faculty will be reinforced later in this chapter. Measuring the Effectiveness of Faculty Development Activities and Programs 132

PAGE 146

The last two elements of best practice involve assessment. Even though very few programs that have been described in the literature used any recommended measures of effectiveness for their programs, calls for such measurement have been repeatedly suggested by researchers. The participants in this study demonstrated a distinct difference from other programs reported in the literature; they measured and assessed the effectiveness oftheir activities and programs extensively. The interview data provided more detailed information about respondents' :tssessment behaviors and activities. All respondents provided data about outcome measures and their references in this topic totaled 39, the fifth highest of all :;ategories. Midwest Women's College had a culture of assessment; they used student performance measures to help faculty understand teaching effectiveness . . the ongoing assessment of student performance is a way ot: is sort of a feedback loop for the faculty to try and figure out whether what we're doing together is working or not. We've developed a set of expectations for faculty that help us evaluate whether they're developing or not as faculty members. It's grounded in what we know about student learning. Midwest Women's College The entire educational context at Pacific State University was built on student learning outcomes so that was what they measured there to determine the effectiveness of faculty development efforts: ... we're more and more doing assessment of student learning. So we're doing that direct kind of stuff and we 133

PAGE 147

... have probably done more surveying kind of indirect measures of student learning." Even though most of these faculty developers recognized the need for accurate assessment data on their programs, at many schools the conventional type of survey measurement, such as, 'what worked, what will you use?', was a common form of assessment. At Eastern University the director realized that while survey information and feedback from participants was helpful, it was skewed by the self-selection process, thus, it couldn't be the sole indicator of program effectiveness. We do ... an evaluation at the end of all of our workshops that we did about what they got out of it .. .if they thought they were going to use this in their practice ... would they recommend it to somebody else ... attitude change or belief change or how satisfactory this was. There are a couple of items around everything, but that's not the only thing that I rely on. Because for the most part, the people who come to programs voluntarily are, in fact, starving for information and they're always like, 'yes this is great, yes this is great, and yes we need more of it'. So I find that that's nice, it makes you feel good, but it's not necessarily giving you the best information. Eastern University Central College, while using a variety of assessment tools to determine the effectiveness of their programs noted that they were searching for other ways to get the appropriate feedback. Sometimes I run focus groups to find out how things are going ... We assess them but we're only beginning to ask questions like 'okay you came to these programs, do you use any of this?' And the kind of follow up like 'okay I know you liked it. I know you told me it was a good use of your time but, did it just die there?' ... We're moving toward asking those kinds 134

PAGE 148

of questions and trying to figure out how to get that kind of information. Central College At Texas University, the faculty developer also acknowledged the importance of assessment but still hadn't found the time to include serious assessment practices in his program. Like the faculty development director at Eastern University, he was a single-person shop and was also the vice provost for 1200 faculty. While he has tried to use technology to help collect relevant data for feedback about the effectiveness of his program and activities, his focus was on faculty teaching and student learning, not so much on attitude or belief change. I've done some follow-up surveys last year. And it was basically focused on what they used from the workshops. And that's still a weak part ... The other thing that we put in place over the last summer was an information data management system ... the last survey is mostly focused on, did it benefit your teaching and did it benefit the student performance? Not as much on attitude change, belief change. Texas University At the North Central University campus, faculty learning communities have been in effect for 30 years. The director described extensive assessment practices that he developed over the time he was the director; he even tracked cognitive developmental changes in his faculty participants: And a very important factor along the way is that we assess the effectiveness of these communities as far as with respect to the individuals in the program, what sort of faculty development changes occurred, and then what happened with respect to the students in their classes. How did they see learning change, how did they know it was changing, how did their attitudes influence change in the student learning ... with my new faculty, I looked at their applications, their mid-year reports, their final reports, to see if I could tease out any cognitive changes, and it fell right into the Perry model or the Belinky model [of cognitive development]. North Central 135

PAGE 149

Southeast State University also has a strong culture of assessment. In 1ddition to having an outside evaluator, the director there requires evidence of impact from his faculty participants before they receive compensation for their There are, like, two or three levels of assessment. .. about the quality of the event, about the usefulness, and that is immediate. The other is, if we are doing courses, we get other people involved in order to see if there are changes. And if we are doing projects, we ask faculty to get evidence of the impact of what they learn, and evidences of how they measure that, so we are then looking at portfolios, or to write reports. Southeast State At Pine State College which has a long history of collaborative teaching and learning, the results of faculty development were evidenced by the growth of each faculty member and their students. Evaluators used not only reports from faculty nembers about how they had grown but also written evaluations from their ;tudents about the faculty and how they (the students] had grown as a result of )articipating in this faculty member's course . . the key to this whole review process is in the teaching and the teaching documents, evaluations. So for example, a review committee is going to be interested in the student's self evaluations that are in that faculty member's portfolio as much as they are the faculty, or the student's evaluation of the faculty ... they're going to look, not necessarily or literally at the comments, but what kind of reasoning went into that? Pine State These responses provide evidence of the culture of assessment at the DEEP ;chools and in their faculty development programs. At some campuses in this ;tudy, assessment practices in the faculty development program have been the 136

PAGE 150

stimulus to promote assessment throughout the institution. For example, the faculty development director at North Central University credited their extensive faculty learning communities for providing the impetus to expand assessment practices across the university. This culture of assessment is in contrast to the assessment practices among many colleges and universities outside the DEEP cohort. Administrative Support The last set of best practices developed from the literature involves administrative and cross-campus connections and support. The interview data supplied more details that demonstrate the strength, breadth, and depth of the mutual support between our faculty developers and their administrators. The following excerpts from phone interviews provided clear examples of the many ways that their administrators provided support for faculty development programs such as through financial support, providing excellent space, release time, or simply by participating in the programs or helping faculty development directors promote their programs and values. Administrators at Central College provided financial support as well as physical space that met the needs of faculty and the faculty development program. The college is quite serious about this and so in addition to paying the salary of the faculty developer who is a, you know otherwise a full time faculty member, provided staff support and also created a physical for us. Central College The financial support at some of the DEEP schools, like Pine State College, is 137

PAGE 151

guaranteed in the structure of the budget. So in terms of administrative support, there's a very large chunk ofmoney, permanent line item on the budget around faculty, around these institutes. And our provost that supports those and we have one, as the Dean in that position, the college has designated an administrative line for faculty development. Pine State While financial support is considered a key element for any faculty development program, personal and emotional support may be the key ingredient that affords successful faculty developers the courage and confidence to lead change. The comments from Western Catholic and Northeast University directors described these feelings. At Western Catholic University the administrative support is not just financial but emotional and spiritual as well. . the office that I report to, which is the office of the academic vice president, has been generous with financial support and sort of emotional and spiritual support for what we're doing ... I have support from the president on down for the work that the center does. Western Catholic The director at Northeast University described the close relationship she had developed with her provost: I report to the provost ... I love reporting to him and he is very supportive . Oh, I totally could not do my job without him .... And it sounds really cheesy but I've always felt like [my provost] has been a person who really does have my back. Northeast University At Texas University, support from the provost has been a result of that Provost's involvement in the teaching process. This administrator understood the connection between faculty development and teaching effectiveness from his personal experience as a teacher in the classroom. 138

PAGE 152

PAGE 153

PAGE 154

substantial, and was perceived by the directors as sustained and dependable. Administrative support in the form of providing financial resources, space, personal encouragement, recognition, participation in programs, or philosophical support has made a crucial difference on these campuses. These differences included increased visibility of the program or center, increased respect for faculty development programs and activities, increased confidence of the director, increased importance of effective teaching and learning throughout the campus, and higher expectations for excellent teaching to be the norm at each institution. Administrative support, encouragement, and expectations were woven into the fabric of the culture at DEEP schools and allowed for greater acceptance and support from faculty and overall success of the program. Other avenues of faculty development support across the campus are discussed in the next section. Cross-Campus Connections to Support Mission and Goals Analyzing the interview data with NVIYO, this research found 79 responses from all 13 participants about cross-campus collaboration, the highest response rate of any of the best practices, and second only to "quality teaching'' in number of responses from all 63 topic areas of the NVIYO analysis. Those results were an indication ofhow important this topic was to our faculty developers. In the analysis that follows, we see a persistent theme of collaboration among faculty, faculty development, academic departments, other offices, and the administration. 141

PAGE 155

At Midwest Women's College, Northeast State University, and Western Catholic, their faculty development programs facilitated collaborative efforts among faculty and provided the venues for collaboration across the campus. At Midwest Women's College, most of the efforts toward institution-wide improvement were cross disciplinary and faculty development there utilized the cross-campus connections to enhance programs on teaching and learning. We look at what faculty are doing in terms of their own discipline and helping other colleagues in developing their abilities as, as teachers. So it could be anything from kind of formal action research to groups or committees getting together to start an initiative that improves something across the campus or across the curriculum. Midwest Women's College At Northeast State, the title of the Faculty Development Center was changed to Teaching and Learning Collaborative because of their growing emphasis on multidisciplinary collaboration. The faculty development center initiated several different kinds of collaborative venues for their development program. We have faculty teaching cooperatives, like faculty learning communities, that meet every 2 weeks for a semester that include faculty from across the disciplines theme based. There are 4-6 of those every semester. We also have a faculty learning community for every general education course; are collaborative as well. Northeast State The faculty development director at Western Catholic University acted as the catalyst for collaborative groups that they helped form. One example of their role was in the Shared Classroom Initiative. One of the things that we do every spring is called the 'Shared Classroom Initiative' and we just basically organize people: faculty members who invite other faculty members to come and sit in on their classes ... Then we 142

PAGE 156

just go and we sit in and then we get together after the two weeks. We just sort of have a little social and the idea is to just see what other people are doing and start a conversation. So we get people talking about teaching and different ways of teaching and what we're trying to accomplish with teaching. Western Catholic Some of the faculty development programs hosted a variety of events designed to get faculty out of their disciplinary silos, increase collegiality, and stimulate collaborative efforts among faculty across campus departments. And it's collegial in the sense that we know each other better. We get direc1 with each other. We host happy hours here. You know we host workshops here. We've got these four programs a week. We've got book reading discussion groups. We've got lots of stuff happening and so people are intermixing a lot. We also did something that is sort of the lynch-pin program that we run. It's called the New Faculty Seminar. It meets every week, all year long. They come for lunch and in exchange for coming, every week for a year, they get a course relief in their first year of tenure track teaching. So they're with their colleagues who were hired that year as well. Central College At North Central University, the faculty development director used faculty learning communities to enhance collaboration across this large university. His perspective was that the networks developed through learning communities then helped spread changes throughout the university, such as the changes made in assessment practices. So, I think this network really, creates an engaged institution, where people are talking to each other across silos and disciplines, and they really get excited about innovations, and they get excited about determining whether these are really making a difference. North Central 143

PAGE 157

Another way that these faculty development directors facilitated collaboration was to share ideas, planning, and resources with other departments and offices. The directors at North Central, Northeast State, Great Lakes University and Central College formed relationships across the campus to help extend their reach and expand their capacity to spread the values of good teaching and learning across the campus. At North Central University, a large research university, the director elicited support from other academic support units for multi-disciplinary faculty learning communities. We co-sponsor faculty learning communities, for example, with the library . . we put in an equal amount of money, and those learning communities are half librarians and half faculty, and they concentrate on information literacy, that's research literacy. We've done it with our information technology services, we've done it with student affairs, so collaboration then [includes] not only faculty, but across the academic support units as well. North Central The faculty development director at Great Lakes University, another large research university, built collaborative relationships with departments, colleges, and other offices across the campus. The result was to generate strong support for faculty development programs throughout the university. Half of our work is done for individual schools and colleges and departments. We also collaborate with ... dozens of offices across campus, some of which are in the schools and colleges and many of which are other central offices, like the Service Learning Office or the Lesbian Gay Bisexual Office, or whatever. Anybody who is open to collaboration we grab. We work within the departments and schools and colleges, as opposed to centrally, because faculty live in their units, and that's where we try to meet with them and deal with their particular interests and concerns. 144

PAGE 158

The director at Northeast State, a smaller university, personally introduced the new faculty to all the departments and offices with which they would be interacting in order to begin building the relationships necessary to effectively implement the faculty advising program. So one of the things that we did is we set up a program for new faculty advisers where they meet three or four times each semester as a cohort. We have this developmental program that goes through developing them as advisers ... a lot of it is about student support services so we take them to the disability resources office, the health center, the career center, the counseling center, academic services ... Northeast State Location was a critical factor in the way the director at Central College introduced collaborative efforts through her faculty development programs. But one of the benefits of being in the library, we coordinate programming with the library all the time. But it also reinforces the centrality of the scholarly life to be located in the library. Central College At Pine State College and Northeast State University, collaboration was entrenched in the culture and history of their institution. At Pine State College where collaborative teaching was the only format for their primary academic programs, collaboration was in everything they did, including planning, teaching, and advising. To speak about activities, institutes and workshops that support teaching, means, I mean it takes on, its real meaning is clear when you think about our faculty teams. There are two, or three or four faculty members on a team coming from different disciplines. Pine State 145

PAGE 159

At Northeast State University, the name of the faculty development center included collaboration, Teaching and Learning Collaborative. This was a purposeful strateg)' because of the emphasis on collaboration in the school's culture. There is a lot of collaboration around the faculty development agenda. The reason that the space we use is called the Teaching and Learning Collaborative is that it is what we offer and what's available here is a collaborative effort. Northeast State At Southeast Women's College a different picture emerged. Collaborative efforts were primarily initiated from the faculty. There are two ways that we, as an institution, encourage collaborative teaching. One is the honors program, which is by design multi-disciplinary and the other is a set of endowed funds specifically for collaborative teaching efforts. But most of the collaborations are grassroots. Southeast Women's College Collaboration was discussed by every participant in the study. In both the online survey and the high number of responses in the interviews demonstrated the centrality of collaboration in the daily work of these faculty development directors. Collaboration improved communication, understanding, agreement about goals and values, and has helped encourage the collegiality seen on many of the DEEP campuses. The directors have described collaboration as a tool to improve the success of their programs by extending relationships and connections among faculty and staff in all campus departments, colleges, and support units. Perceived Impact of Faculty Development Program on Faculty Approaches to Teaching. 146

PAGE 160

Question four, "What is the perceived influence of faculty development best practice at DEEP schools on faculty approaches to teaching?" was answered through interview responses. The last piece of data from the interviews to be described and analyzed involved the perceived impact of faculty development programs on faculty approaches to teaching that contribute to student success. One of the final phone interview questions asked the participants "What elements of your faculty development program, are the most important factors contributing to teaching approaches that lead to student success?" The responses centered around three main themes: community building, helping faculty understand effective pedagogy, and motivating faculty to become better teachers. While these were discussed as separate factors, they really interacted in different ways and it was the synergy of these interactions that provided faculty developers a strong lever for change. Community Building and Developing a Culture of Teaching Faculty development at the following four schools was based on community building and common pedagogical practices developed through such collaborative efforts. The faculty development program at Midwest Women's College, for example, provided a way for faculty to develop and share a common language, common goals, and common definitions of teaching and learning. A commitment to a common, coherent, developmental curriculum on the part of all faculty. That means that there's a kind of language we're using with students, that students are transferring learning across courses, and that 147

PAGE 161

we design things differently for first semester, first year students and for seniors. The, the second thing is forms of collaboration that support the study of and improvement of teaching and assessment practices across the college. Midwest Women's College The importance of developing a common reference point for discussions about and changes in teaching and learning was echoed by the director at Pacific State University. Responding to the question about the impact of faculty development on effective teaching practices, this director responded: "Overwhelmingly getting cross-disciplinary teams and faculty sitting down together and talking about what they do." Similar to Midwest Women's and Pacific State, the director at Western Catholic University emphasized the cross-departmental conversations about teaching and learning and enhanced communication afforded by the faculty development programs as the most effective way that faculty development fosters good teaching practice. I think the thing that we do best and that we have the broadest impact with is fostering conversations .... part of it is this shared classroom initiative. So we get people talking about teaching and different ways of teaching and what we're trying to accomplish with teaching. Western Catholic At another school, North Central University, the director echoed the strength of collaboration and communication about pedagogy as factors in promoting teaching excellence, and described how these connections helped to foster the initiation of best teaching practices found at that campus. 148

PAGE 162

Well I, again I'd have to say that it's that collaboration and scholarship aspect that is developed in these faculty learning communities. The scholarship of teaching and learning is a component of each of the communities. So they really have to come up with evidence that can make a good change .... they're aware of the scholarship that is out there about how students learn ... if they're engaged learners that it's going to be a deeper kind of learning. . and the fact that faculty have these connections across departments, so that just broadens who they know, who they can call on for support, who they can work with. North Central These common viewpoints, shared perceptions, and common goals for teaching and learning are developed in the discussions and crossdepartmental venues instituted by faculty development programs. These shared experiences and attitudes then become the building blocks for a culture of excellent teaching and learning that we see at the DEEP schools. Several of the directors emphasized the importance of building a common culture of teaching. At Texas University, faculty development helped faculty clarif) what good teaching meant at the institution, thus forming a basis for shared teaching values across the campus. The emphasis of 'you're the person who helps students learn and making it student oriented. So, course design is focused on helping students learn; the message is always on student learning. Alright, it's students, and it's not you,' What can you do to help students get this stuff?' Helping the faculty look at their role as a teacher differently, moving them away from 'oh, I can just suffice with lecturing', to know that's really not effective anymore ... the presence of the Center, and that it's relatively visible, and the culture at Texas, with its focus on serving the young people in this region. Texas U. At Eastern University, the director described how her program helped create and highlight a culture of teaching: 149

PAGE 163

... we do this big teaching award ceremony every spring to honor the people who have been recognized by the teaching awards process. And that kind ofvisibility ... is calling attention to the way that teaching is valued on our campus ... it's one of the things that's really important to creating a culture where teaching is something that's visible and talked about and noticed. Eastern University This faculty developer also found that by strategically supporting department chairs in their work she could enhance the culture of teaching: ... provide more support for our department chairs .... but you know as a department chair you start to build that culture and they start to feel like I'm a resource and that the office is a resource and that there's a website resource and good things are happening. Eastern University As faculty discussed the goals and challenges of their teaching experiences, and share ideas of how to be excellent teachers, in ongoing, regular meetings or discussion groups or learning communities, strong relationships were built. These relationships provided the support faculty needed to take the risks inherent in change when trying new and better ways of teaching. The director at North Central University described how these connections and collaborative opportunities among faculty afforded by faculty development programs have helped promote change by providing a scaffold for trying new pedagogics . . faculty have these connections across departments, so that just broadens who they know, who they can call on for support, who they can work with if they want to get started in cooperative learning. There's a couple people here on campus who have done those communities, facilitated them, and so they are willing to provide help and assistance in trying out new things. North Central !50

PAGE 164

The director at Northeast State University acknowledged that her faculty were already good teachers because people with an interest in teaching self-select to Northeast State due its teaching reputation. However, these faculty lacked confidence in their teaching abilities because they lacked an educational degree. Her faculty development program helped them build a language and understanding of effective educational practice, thereby enhancing their confidence to teach well. They may not necessarily be as informed sometimes about the language of it, and how to describe what they do, but most of them are exceptional teachers. So I think the professional development gives them the opportunity to focus on that, and just share, and learn from one another, what others are doing. Northeast State She indicated that her faculty had extraordinary commitment to the vision of the university and her program provided "an important venue for making the commitment meaningful through implementing practices that support the vision." Facilitating Effective Pedagogy Another factor contributing to teaching approaches that lead to student success is helping faculty understand and implement effective pedagogy. The participant from Central College described how promoting active learning principles was the most important factor in helping her faculty become the excellent teachers found at that campus. l would say probably through trying to encourage faculty to utilize active learning strategies ... [such as] designing assignments that require students to revise their writing, ... running peer groups, and ... faculty student research collaboration. Central College 151

PAGE 165

The faculty developer at Northeast State University indicated that her faculty were very busy and didn't really have the time to stay abreast of effective teaching practice so helping them access and digest educational research became one of her priorities . . staying abreast of new ways of teaching, new ideas, new stuff is not something they always have the time to do and so I spend a lot of time reading a lot of discipline-specific journals, and mailing a lot of articles with highlighted sections. Northeast State At Great Lakes University, a large prestigious, public, research university, the director responded to the question of how faculty development influences faculty approaches to teaching by describing how her program brought student voices to the faculty and helped with teaching efforts any way that they could. Well, I see our role as bringing student voices to instructors. And we provide lots of data and research from ourselves and others about student needs, student interests. We do consultations, workshops, theater sketches, evaluation services and assessments ... the publications ... our availability to help with everything. Great lakes University The director at Great Lakes spent nearly two decades building a strong, comprehensive program that met the needs of a very diverse set of faculty from across many colleges and schools. The size of their program and solid financial support allowed them to provide a wide variety of venues to reach and impact teachers in all colleges and at any phase of their professional career. 152

PAGE 166

Motivating F acuity The third area of impact that faculty development programs have on effective approaches to teaching is motivational. It takes time, effort, and courage for faculty to change teaching practices and try new methods and strategies. The DEEP directors reported that their faculty were extremely busy and lacked time to participate in the many faculty development programs offered. Therefore, it is logical to surmise that the more motivated faculty were to change, the greater likelihood that they would take the time and make the effort to engage in development practices offered on their campus. The director at Southeast State University viewed the impact of his program Dn faculty as primarily motivational. Motivation to improve the quality of teaching is a critical thing. Engagement into this type of continuous improvement is a critical factor. But other is support or leadership from the part of the program chair. Southeast State University He also described providing development opportunities for different phases of faculty professional growth. The tenure track faculty are more into research, so we help them doing research about their teaching. The non-tenure track faculty are more into doing their teaching skills, are more into exploring new technologies. So it [faculty development) depends on their needs and clearly on the way they are approaching their professional careers. Southeast State University Similarly, the director at North Central University used learning communities to meet the diverse interests and needs of faculty from different disciplines and at 153

PAGE 167

different stages of their growth and development. His learning communities provided the motivational stimulus for faculty to become engaged. I think this network across the campus and the particular topics are all involved in teaching and learning. It could be using cooperative learning problem-based learning, or active learning, even technology's second life, all involve connecting with students and working with students. .. this network really creates an engaged institution, where people are talking to each other across silos and disciplines, and they really get excited about innovations, and they get excited about determining whether these are really making a difference. North Central University Responses to question four, how does faculty development impact faculty approaches to teaching that make a difference in student engagement, included three themes: community building, helping faculty understand effective pedagogy, and motivating faculty to become better teachers. Each of these factors was perceived as important, in itself, for improving teaching practices at these campuses; however, when all these elements worked in synergy they created a teaching culture that defined, expected, and supported excellence. Summary of Findings from Online and Interview Data To answer research question l and 2, the following summary provides a sample of the faculty development practices and attitudes found at the DEEP schools; the evidence columns are framed in the context of best practice in faculty development. These directors used various collaborative strategies to build a framework of support and connection among faculty, faculty development, and administration. They then modeled active learning strategies in multiple venues to 154

PAGE 168

provide information about etlective pedagogies. And finally, they attended to faculty motivation to improve their teaching through various models of effective practice, connections with their peers, and collaborative strategies that reduced the risk oftrying new ways of teaching. Table 4.15 Summary of Common Faculty Development Structures and Practices at DEEP Schools Structures and Practices Program goals Factors helping to achieve program goals Obstacles to achievement of program goals Common practices Online Survey Evidence Improve student learning; enhance value of teaching effectiveness; improve learning environment; introduce teaching strategies; serve institutional needs Engaged and supportive faculty; strong administrative support; adequate budget Lack of time faculty could devote to development activities Establish positive climate for teaching and learning; network among faculty across departments; collaboration among faculty; establish learning communities; teaching improvement workshops; provide resources 155 Interview Evidence Curriculum is driven by learning outcomes and all teaching and learning and faculty development efforts revolve around that goal improved student learning. Pacific State There is a commitment to a common, coherent, developmental curriculum on the part of all faculty ... there's a common set of abilities that comprise our core and that everybody commits to teaching and assessing. Midwest Women's Faculty learning communities ... have kept faculty talking to each other across disciplines; they get faculty out of their silos and ... give them a vehicle to talk about teaching and learning. North Central University

PAGE 169

Table 4.15 (continued) Structures and Practices Characteristics contributing to the success of activities Assessment strategies Measurement outcomes Administrative support Online Survey Evidence Participant interaction; planned in collaboration with faculty; time for reflection; peer collaboration; prompt feedback; follow-up SeSSIOnS Number of workshop participants; workshop evaluation; institutional needs assessment; frequency of contact with faculty; strategic planning; satisfaction survey; focus groups Participant satisfaction; institutional impact; faculty behavior change; faculty knowledge change; student performance; faculty attitude change; student retention Advertise; talk about FD in recruitment; participate or sponsor activities; speak nationally 156 Interview Evidence I think people would be amazed at the energy and intellectual work and excitement that goes on once people are in those meetings. But it's because they're doing something that makes them feel like they're accomplishing something in relation to student learning and what they care about as teachers. Midwest Women's College I've done some follow-up surveys last year. And it was basically focused on what they used from the workshops. ... the last survey is mostly focused on, did it benefit your teaching and did it benefit the student performance. Texas University There are two or three levels of assessment ... quality of the event, usefulness, and that is immediate. The other is, if we are doing courses, we get other people involved in order to see if there are changes. And if we are doing projects, we ask faculty to get evidence ofthe impact of what they learn, and evidences of how they measure that. Southeast State The office of the academic vice president, has been generous with financial support and sort of emotional and spiritual support for what we're doing ... I have support from the president on down for the work that the center does. Western Catholic

PAGE 170

Table 4.15 (continued) Structures and Practices Cross-campus connections and collaboration Community building and develop culture of teaching Facilitate effective pedagogy Motivating faculty to become better teachers Online Survey Evidence Collaboration among faculty, network among faculty across departments, and establish faculty learning communities all important to success of FD program. Interview Evidence We also collaborate with ... dozens of offices across campus, some of which are in the schools and colleges and many of which are other central offices, like the Service Learning Office or the Lesbian Gay Bisexual Office, or whatever. Anybody who is open to collaboration we grab. We work within the departments and schools and colleges, as opposed to centrally, because faculty live in their units. Great Lakes University We do this big teaching award ceremony every spring to honor the people who have been recognized by the teaching awards process. And that kind of visibility ... is calling attention to the way that teaching is valued on our campus. Eastern University Staying abreast of new ways of teaching, new ideas, new stuff is not something they always have the time to do and so I spend a lot of time reading a lot of discipline-specific journals, and mailing a lot of articles with highlighted sections. Northeast State This network really creates an engaged institution, where people are talking to each other across silos and disciplines, and they really get excited about innovations, and they get excited about determining whether these are really making a difference. Central University One set of common structures at DEEP schools included substantial use of faculty in the planning and delivery of faculty development activities and 157

PAGE 171

programs. Some schools used faculty advisory boards as ways to input ideas for faculty development efforts. Others sought input and assistance from various faculty across the campus to gather relevant information about the needs of faculty and also to extend the reach of development programs and activities across departments and throughout the campus. A common venue among the DEEP schools for cross campus collaboration was faculty learning communities. Although these faculty development structures were not part of the original set of questions, the majorities of faculty development directors were using them, had used them for a while, or were planning on using them as a means of effective cross campus collaboration. One area that these directors agreed on was the importance of building community among faculty from all departments, administrators, and staff. By acting as a bridge between those various communities on their campus and as a catalyst to bring those divergent viewpoints together, faculty developers at DEEP schools could promote a culture of teaching excellence which was one of their top priorities. Faculty development practices that DEEP schools had in common included goals, priorities, activities, and assessment strategies. The goals that were the top priority for these schools focused on student learning and developing a climate that valued effective teaching: improving student learning, enhancing value of teaching effectiveness, improving the learning environment, introducing teaching strategies, 158

PAGE 172

and serving institutional needs. These schools agreed the only relevant hindrance to the achievement of their goals was lack of time that faculty could devote to faculty development activities. Baniers commonly perceived to interfere with achievement of faculty development goals were not considered baniers for this group of directors; categories such as "research culture rather than teaching culture", "budget constraints", and "lack of administrative support" were ranked as not much or not at all a banier. For another practice, assessment strategies, DEEP directors were in agreement among themselves and with best practices but differed from previous faculty development research of non-DEEP schools. The directors in this research study used many different kinds of assessment to measure outcomes of their programs and activities and they used these tools frequently. The outcomes they measured to determine the effectiveness of their programs included participant satisfaction; institutional impact; faculty behavior, knowledge and attitude change; as well as student retention and student performance measures. All the directors used at least several of these metrics on a regular basis compared to only 20% of directors who used any of these tools in the research reported in the literature. Finally, the faulty development directors who participated in this research study reported substantial and consistent support from their administrators and they described a supportive and encouraging symbiotic relationship with their administration. They supported administration goals and programs and the 159

PAGE 173

administration supported their goals and programs. The directors who participated in this research agreed on which practices they ranked at the top of their priorities. Establishing a positive climate for teaching and learning, networking and collaboration among facu1ty from all departments, and establishing learning communities were among their top priorities. Characteristics of the development activities that these directors perceived contributed to their success included participant interaction, collaboration among faculty, and planned in collaboration with faculty. Chapter 5 will summarize and interpret the findings from the online surveys of the directors and those of the administrators as well as the phone interviews of the faculty development directors. 160

PAGE 174

CHAPTER 5 DISCUSSION Summary and Interpretation Substantial evidence exists in the college student success literature of the necessary antecedents to student success in college as well as the teaching practices that contribute to student success. From the work of Pascarella and Terrenzini (2005) we know that, of the many factors impacting student persistence, the following are determined by the classroom teacher: use of cooperative learning and active learning; instructor organization, skill, clarity, and preparation; classroom discussions and activities requiring higher order thinking skills; absence of classroom racial discrimination or prejudice; concern for and rapport with students; and faculty availability. In addition, students' cognitive growth was positively influenced by a close relationship and frequent interaction with faculty. When Kuh, et al. (2005) reviewed teaching practices at DEEP schools, it became clear that faculty at these institutions did more to facilitate student engagement, retention, and learning than their counterparts at peer institutions measured by NSSE. Their approaches to teaching were congruent with those that were associated with improved student outcomes as described above by Pascarella and Terrenzini (2005). According to Kuh, et al., teachers at DEEP schools use those effective teaching strategies to a greater degree than faculty at other schools. We know, then, that faculty at the DEEP schools are using strategies that improve 161

PAGE 175

student learning but what is less clear, is how they come to possess these skills and talents. Perhaps they come to the university with these abilities or perhaps through faculty development efforts at the DEEP schools, they learn effective teaching practices. The faculty development literature cited in Chapter 2 provides evidence for the following faculty development practices found to make a positive impact on effective approaches to teaching: faculty ownership and community building with support and rewards for the change process; outcome measurements from a variety of sources integrated into the faculty development program; development activities modeled on participant engagement and Ieamer focus which address the full range of faculty roles; and the alignment of faculty development goals with institutional mission (Gibbs, 2003; Hellyer & Boschmann, 1993; King & Lawler, 2003; Levinsosn-Rose & Menges, 1981; Rust, 1999; Sorcinelli, 200 I). The purpose of this research study was to investigate which elements of a faculty development program impact approaches to teaching in ways that contributt to improved student learning and persistence. By examining the faculty development programs at high performing institutions, where faculty were using effective teaching approaches and students were succeeding at higher-than-average rates, this research attempted to tease out commonalities and differences among the DEEP school faculty development programs and between these programs and those 162

PAGE 176

reported in the literature. These comparisons have provided clues to specific facult) development practices and structures that encourage effective teaching practices. The following research questions guided the investigation of the relationship between faculty development and effective teaching approaches. 1. What are the common structures and practices among faculty development programs at institutions that produce better than predicted student outcomes (DEEP schools)? 2. How congruent are the FD structures and practices of DEEP programs with best practice as described in the literature? 3. What are the elements of FD practice at DEEP schools that differ from best practice as described in the literature? 4. What is the perceived influence of FD best practice at DEEP schools on faculty approaches to teaching that contribute to student success? The results described in chapter 4 and summarized in Table 4.14 will be discussed in the context of these research questions. Questions 1 and 2 The first two questions concerned common structures and practices of DEEP faculty development programs and their congruence with best practice. While some overlap exists between structures and practices, for the purposes of this research study faculty development struch1res include reporting structure, location, faculty involvement in design and delivery of program elements, and the relationship between the faculty development program and other institutional units. Faculty development practices include goals, activities, programs, accountability 163

PAGE 177

practices, and institutional communication. ln chapter 5, I will investigate the structures and practices described above, in that order, to discern commonalities among the DEEP programs and congruence with best practice. Structures While most of the participants described similar reporting structures (to the provost or assistant provost) and central or visible locations, neither of those was considered to be a factor in the success of their programs. However, 100 % of the participants perceived faculty involvement and ownership to be important or very important to the success of their activities. Relationships across campus were a primary interest of participants in the interviews as well as in the online survey. All 12 survey participants agreed that "networking among faculty across departments" was somewhat, quite, or extremely important to the success of their programs. [do I reference the tables from ch 4?] Of the 13 interview participants, all 13 added comments about crosscampus collaboration for a total of 79 comments, one of the highest response rates for any node. This is an indication of how important cross-campus collaboration was to the interview participants. Midwest Women's College had an expectation that faculty from across the campus would get together to help each other ... what faculty are doing in tern1s of their own discipline and helping othe1 colleagues in developing their abilities as teachers. So it could be anything from kind of formal action research to groups or committees getting together to start an initiative that improves something across the campus or across the curriculum. 164

PAGE 178

At Pacific State University, they used cross-campus themes to connect faculty in learning groups. We have faculty teaching cooperatives, like faculty learning communities, that meet every 2 weeks for a semester that include faculty from across the disciplines-theme based. Pacific State The entire curriculum at Pine State College was designed and delivered through collaborative teams, where they described "teaching IS faculty development." At Northeast State University, where collaboration was a central feature of their program, the faculty development center was renamed to include the concept of collaboration . . there's a lot of collaboration around the faculty development agenda. The reason that the space we use is called the Teaching and Learning Collaborative is that is what we offer and what's available here is a collaborative effort. Northeast State At Great Lakes University, the collaboration across units and departments is necessary to extend the reach of faculty development across such a large institution The faculty development director there created partnerships with many and units with whom she collaborated to develop and present faculty development programs. Outside the DEEP schools, many colleges and universities continue to be structured in disciplines that form rigid silos (Blackburn & Lawrence, 1995). While disciplines exist at the DEEP schools, all the faculty development programs, 165

PAGE 179

including those at the large, Research I institutions, spent considerable time and ;:[fort collaborating with and forming bridges among the disciplines via faculty :levelopment programs and activities. The director at North Central University ;:xplained his commitment to cross-campus learning communities this way. I think this network really creates an engaged institution, where people are talking to each other across silos and disciplines, and they really get excited about innovations, and they get excited about determining whether these are really making a difference. North Central Practices Faculty development practices include goals, activities, programs, :1ccountability practices, and institutional communication. There were many ::ommon practices among faculty development directors in this study. The first set Jfpractices in this investigation was goals: what were the most important goals and what were the factors that helped or hindered achievement of their goals. Goals The most agreed-upon goal of these participants was to "improve student learning," followed closely by "enhance the value of teaching effectiveness," 'improve the learning environment," and "introduce teaching strategies". All twelve participants agreed that those were somewhat, very, or extremely important to their faculty development programs. These goals and their ranked importance were very similar to those described by King and Lawler (2003) in their survey of 249 POD members from predominantly research I institutions. 166

PAGE 180

When looking at what factors helped or hindered the achievement of their goals, all 12 participants in this research agreed that "engaged and supportive faculty" and "strong administrative support" were somewhat, very, or extremely important factors in achieving their goals. These factors were followed closely by "adequate budget" and "tradition of support and climate of collaboration" with 11 of 12 and 10 of 12 participants in agreement, respectively. Again, these rankings were very similar to those of 109 faculty developers in the Frantz et al. study (2005). However, when we look at the ranking of practices important to the success of their faculty development programs, although the DEEP faculty developers were in high agreement among themselves, distinct differences emerged when compared to the responses of developers in the Frantz et al. (2005) study. All 12 of the DEEP participants agreed that the following practices were somewhat, quite, or extremely important to the success of their faculty development programs: establish a positive climate for teaching and learning, network among faculty across departments, collaboration among faculty, establishing learning communities, and teaching improvement workshops. Again, we see the emphasis on collaborative, connected activities and programs. In contrast, the directors in the Frantz et al. (2005) study ranked the DEEP directors' first three factors (positive climate, networking, and collaboration), ninth, eighth, and tenth, respectively. The Frantz et al. participants' first two factors, 167

PAGE 181

teaching improvement workshops and providing faculty mentoring. were ranked sixth and ninth, respectively, by the DEEP participants. The DEEP participants appeared to place a higher value on collaborative strategies that built a collegial network of faculty as contributing to the success of their programs while the other directors appeared to place a higher value on direct instructional strategies. The only obstacle that directors in this research agreed was a hindrance to the achievement of their goals was "lack of time faculty could devote to faculty development activities". All 12 participants agreed that "lack of time" was somewhat, quite, or a strong hindrance to achievement of their goals. The next most agreed upon factor was lack of adequate staff support, which 8 of 12 agreed was a hindrance to achievement of their goals. Again, there were differences between the directors in this research and those from the Frantz et al. (2005) study. While "budget constraints" and "research rather than teaching culture" were barriers that ranked first and second, respectively, by directors in the Frantz et al. study, the DEEP participants ranked those barriers fifth and fourth, respectively, with mean scores of 2.50 and 2.65 out of 5, respectively. The low ranking and low score indicates that those two barriers weren't considered very important by the DEEP directors but were the most important barriers to the Frantz et al. group. One explanation for the divergence about funding might be that the DEEP faculty development directors regularly responded in interviews that they were well 168

PAGE 182

or adequately funded. The range of funding from the online survey participants was reported as $32/FTE$2400/FTE per year with a median of $297 /FTE per year. In terms of a yearly budget, the DEEP directors reported a median yearly budget of$67,000 for their program. These data come from the online survey, in this research, in which 7 of the 12 programs were from small schools with student populations from 1,000 4,000. It is important to note that the interview data were collected in early 2010, a time when most colleges and universities were facing serious financial challenges. Yet all the participants in this study reported having adequate or excellent financial support for their faculty development programs. No funding data were collected by the Frantz et al. (2005) study; however, in that publication, the authors cited a study in 1998 by Wright that included funding data. The median yearly budget for the 33 faculty development programs participating in the Wright study was \$112,000/year. While Wright's budget was greater than the yearly median budget for the DEEP respondants, the rate per FTE might well be lower for the Wright group, since the participants were drawn from the POD organization which represents a much higher percentage of large, state institutions rather than small, private schools as were represented in this research study. The second highest obstacle that hindered achievement of faculty development goals in the Frantz et al. (2005) study was "research culture rather than teaching culture". This was ranked fourth by the DEEP directors and rated a 169

PAGE 183

2.65 out of 5 mean score. The low score for this obstacle is not surprising since seven out of twelve of the online survey participants were from smaller schools with a strong teaching culture as opposed to the large research I institutions represented by POD members in the Frantz et al. study. However, it is important to note that even the large research I institutions in this study of DEEP programs had strong teaching cultures and expectations of teaching excellence in addition to high research expectations. Activities The second area of practices where this research sought to highlight commonalities was activities, a mainstay of faculty development programs. The online survey demonstrated six out of nine common responses to the question abou characteristics of program activities that contributed to the success of faculty development activities. All 12 of 12 participants perceived that "participant interaction" was quite or extremely important to their activities' success. All 12 of 12 perceived that "planned in collaboration with faculty'' was moderately, quite, or extremely important to the success of their activities. Three other characteristics, "time for participant reflection," "peer collaboration in the activity," and "prompt feedback for participants," were deemed moderately, quite, or extremely important to the success of their activities by 11 of 12 participants. "Follow-up sessions or additional information" was seen as moderately, quite, or extremely important by 170

PAGE 184

l U ot ll partiCipants. Once a gam, these numbers indicate the Importance or collaboration and faculty engagement to faculty developers at the DEEP campuses. Three characteristics were not selected by DEEP participants as important to the success of their activities: "rewards for changed behavior," "follow-up support for new behaviors," and "guided practice for new behaviors". The DEEP directors in this study, then, tended to value or use various processes that shaped the activities but not techniques that attempted to shape faculty acquisition of those behaviors introduced in the activities; that tendency differs from recommended best practice. Data from the phone interviews supported responses from the online survey respondents about the importance of the following characteristics of faculty development activities. Thirteen of 13 interview participants made comments about "faculty ownership" with 61 total comments, the third highest rate of comment of all 68 nodes. Twelve of thirteen participants provided input about "participant engagement" (27 total comments) and "feedback" (31 total comments). Faculty at many of the DEEP schools came up with ideas for programs, developed the programs, and ran them. The faculty development directors encouraged these innovations through financial support, organizational assistance, and recognition efforts. At Great Lakes University the faculty development director depended on faculty for much of the program. "We've got an advisory board that makes our grant decisions. We have faculty running, or co-facilitating almost all oJ 171

PAGE 185

our programs. We have faculty steering committees on all the major things we do." Her examples were echoed by all the participants with faculty development programs. Another way faculty provided input and ownership of the faculty development program was through advisory boards. Some boards were made up of faculty teaching award recipients, some were comprised of faculty and administrative leaders from across the institution, some board members were faculty who were invited to specifically represent a strategic unit in the college or university. All advisory boards met on a regular basis and provided guidance, support, public relations, and direction for the faculty development director and program. Faculty engagement in activities was another important aspect of faculty development programs at the DEEP schools. When describing characteristics that contribute to the success of their activities, "participant interaction" was the most highly agreed-upon characteristic, with 100% of the survey participants indicating it was quite or extremely important. Two other indicators of faculty engagement in activities, "time for participant reflection" and "peer collaboration in the activity" both had 92% agreement that they were moderately, quite, or extremely important. When interview participants were asked how important faculty interaction or engagement was to their activities, on a scale of 1-10, their responses all ranged between 7-10. Some, like the director at Northeast State, talked about the 172

PAGE 186

Importance of havmg taculty share what they are domg m their classrooms, "Many of them [faculty development programs] are faculty driven ... there is a lot of opportunity for conversation and discussion, and sometimes you think that's not thf most productive use of the time, but it actually is ... the opportunity to share what's going on in your own classroom". A prominent theme among the DEEP directors was providing opportunities for faculty to get together and talk about wha1 they were doing or how they were teaching. Other directors talked about engaging faculty to improve their courses. The director at Texas University stated, "I ask them to bring their syllabus and focus on one learning outcome, and go through the steps for good course design, using Dee Fink's model ... They're going to have to think about, how am I assessing that? What are my learning activities?" At North Central, the director explained how learning communities "give them a vehicle to talk about teaching and learning, and to go beyond talk to projects that they designed for their courses and [how] they assess the effectiveness of those projects, innovations for them, and then they present [their findings]". Accountability practices One area that stood out among the participants in this study, both those from the online survey and those who participated in the phone interview, was the area of accountability or assessment. Decades of literature in faculty development have described a lack of appropriate assessment to determine whether these 173

PAGE 187

programs were etlective. Very little empirical evidence exists in the literature to support the effectiveness of faculty development efforts to improve teaching and learning on their campuses. Even as recently as 2005, when Frantz et al. surveyed 109 POD members, only 20% of those directors did any assessment of their activities or programs at all. Of the 20% who used assessment tools, most were in house satisfaction surveys. One of the DEEP participants, from Eastern University, described why relying solely on those kinds of surveys was insufficient, "for the most part, the people who come to programs voluntarily are, in fact, starving for information and they're always like 'yes this is great, yes this is great, and yes we need more of it.' It makes you feel good but it's not necessarily giving you the best information". Many of the DEEP development directors go beyond satisfaction surveys to uncover information about how effective their programs are in meeting the goals of improving teaching and learning on their campuses. To determine the effectiveness of their activities, 100% of the DEEP directors commonly or always measured "participant satisfaction". However, 90100% also measured the following additional outcomes sometimes, commonly, or always: faculty knowledge change, faculty behavior change, institutional impact, student perforn1ance, and faculty attitude change. The tools they used to measure these outcomes included: questionnaire they constructed (100%). student faculty evaluations ( 42%), and student knowledge survey (25%). One director sometimes used the College Experience Questionnaire (CEQ), Teaching Methods Inventory 174

PAGE 188

(TM1), and Approaches to Teaching Inventory (A Tl) inventories, and one director always used the Student Evaluation of Educational Quality (SEEQ) to measure the effectiveness of faculty development activities. A similar pattern emerged when directors were asked what measures they used to determine the effectiveness of their faculty development program, as opposed to activities. Ninety two percent of the surveyed directors use number of participants in faculty development programs, 83% use workshop evaluations, 67o/c use an institutional needs assessment, 58% use frequency of contacts with faculty, strategic planning and program review, and value surveys. Fifty percent use focus groups and program exit surveys to evaluate extended or ongoing programs. The director's use of these measurements ranged from L-2 times to more than 10 times in the past two years. The interview data substantiated the importance of outcome measures for the DEEP directors. Most of the respondents were using a variety of measures as indicated above, but they also were concerned that they weren't doing enough and were struggling with ways to do more or get at specific changes their faculty made as a result of the faculty development experiences. The director at Central University noted, "We assess them but we"re only beginning to ask questions like 'okay you came to these programs, do you use any of this? l know you told me it was a good use of your time but, did it just die there?' We're moving toward those kinds of questions and trying to figure out how to get that kind of 175

PAGE 189

mtonnat10n." These directors are not only aware of the need for accurate and meaningful assessment data; they are working to gather these data in a variety of ways. Other faculty development programs had richer assessment strategies. At North Central, where learning communities have been the venue for most faculty development, the director measured the outcome of the learning communities extensively. "what sort of faculty development changes occurred, and then what happened with respect to the students in their classes. How did they see learning change, how did they know it was changing, how did their attitudes influence change in the student learning". He even has begun measuring qualitatively, evidence of cognitive changes in his faculty: "I looked at their applications, their mid-year reports, their final reports, to see if I could tease out any cognitive changes, and it fell right into those [models of cognitive development], into the Perry model or the Belinky model". His findings would be useful to any faculty development director seeking to determine the effectiveness of their program. Another director used Title Ill funds to encourage appropriate assessment o faculty development efforts. We are able not to pay faculty for learning, but to pay faculty for applying what they learn and for documenting what they get from their experience. There are, like, two or three levels of ... assessment; one is for each of the interactions: we get feedback about the quality of the event, about the usefulness, and that is immediate. The other is, we get other people involved in order to see if there are changes, and if we are doing projects, we ask faculty to get evidences of the impact of what they learn, and 176

PAGE 190

evidences of how they measure that, so we are then looking at portfolios, or to write reports. Southeast State While assessment remained a challenge for the DEEP directors, the amount :md depth of assessment done on these campuses was very different from that reported in the literature. A factor to be considered, however, is that all these used NSSE as a tool to help improve their performance campus wide. Since the scores on the NSSE for these schools were very high, indicating a high level of 5tudent engagement, leaming, and change, it was likely the culture at many of these institutions was one of continuous, measurable improvement. During the interviews l asked if continuous improvement was the norm on their campus. For many of the schools, improvement was the norm. For example, at Southeast State, "Search for excellence is the key focus. We are committed to quality assessment, and we try to leam from feedback that we get and from prior :!Xperience ... and we try to find those critical factors that may help us improve, and that is where we focus the action". However, many of the directors described continuous improvement as a :::hallenging process. At Western Catholic, where continuous improvement is the norm, the director shared his concerns: "While everybody would say 'oh yes I'm all about improving continuously', having these data and the understanding of how to go about that [has been a challenge] and we need to build the structures that will actually allow us to do the kind of improvement we talk about". 177

PAGE 191

Some directors even voiced some of the downsides to a continuously improving organization. The director at Northeast State University stated that continuous improvement was the norm on her campus but, "Sometimes, I wish it wasn't. Sometimes I wish we could pause, for even a millisecond, you know, and breathe. But I guess we're always on a treadmill". Although improvement is described as difficult, perhaps the institutional focus on measuring outcomes that drive improvement has had an impact on the faculty development directors at the DEEP schools. This group definitely uses more assessment than directors at other institutions reported in the literature. They measure more outcomes, they use a wider variety of measurement tools, and they measure more frequently than directors reported in other studies (Frantz et al. (2005). Institutional Communication and Relationships with Administrators Another area of DEEP response that was different from other schools in the literature was in the area of relationships with the administration. At many colleges and universities, the relationship between the faculty and the administration has been one of friction and contention. The administration's efforts to move the organization forward and meet the external demands for quality and accountability have been met with substantial resistance from faculty who perceive these efforts as alien, threatening to their autonomy, and even harmful to the fundamental nature of the academy (Blackburn & Lawrence, 1995). 178

PAGE 192

In contrast, at the DEEP schools, every one of the directors describes a strong and mutually supportive relationship between their faculty development program and the administration. Moreover, they describe their faculty as part of this mutually supportive relationship. These directors find and facilitate collegiality and shared purpose not only among the faculty and across departments, but also between the faculty, support staff, and the administration. This climate ofmutual support, respect, and trust, together with the shared commitment to provide the best educational experience for their students, may be a key component of the high quality teaching and excellent student outcomes found at the DEEP schools. Research by Seldin ( 1995) states that improved teaching is encouraged by a campus climate that rewards teaching in word and deed and accords it equal status with scholarly work and publications. His suggestions for administrators to elevate the importance of teaching included, making good teaching a leadership priority, encouraging faculty dialogue across disciplines, defining scholarship to include scholarship of teaching, and making teaching ability a criterion for hiring new faculty. These areas of influence were among those described by directors in the survey and in the phone interviews and were corroborated by administrators in their survey. When asked, in the faculty development directors' online survey, which factors influenced the achievement of their faculty development goals, l 00% of the directors responded that strong administrative support was somewhat, very, or 179

PAGE 193

PAGE 194

While the level of support at other DEEP institutions may not be as personally close as the two previous examples, the administrative support is substantial. At North Central University, the director claims that the administration has always been sensitive to the needs of teaching and learning. "There's a willingness to support innovative approaches, and a willingness to support straightforward, non-innovative approaches. Another facet that has been very important here is the development office, [they have] sponsored or funded a good part of our particular center's work over the years". At Great Lakes University, the director described having very good resources, including funding and space. The vice provost, to whom I report, shares our vision, and has been very helpful in garnering those resources and space. The President and Provost both speak at new faculty orientation which out program organizes for the University, and they both encourage faculty to use our services. There are many offices, other offices, and deans and department chairs who are providing teaching support and encouragement in many different ways. Great Lakes These positive attitudes were reflected across all 12 participants discussing their administrators. One of the factors in this strong relationship between faculty developers anc their administration might be the small size of many of the DEEP institutions. Small colleges and universities represent sixteen of the twenty schools highlighted as high performers in the DEEP study. At smaller organizations, it is easier to see everyone, make connections, and build the solid relationships that nurture the organization. At many of the DEEP schools, faculty and faculty developers do 181

PAGE 195

PAGE 196

development programs also have the role of associate vice provost. While their primary responsibility is faculty development, they have other administrative duties. Both these schools are large, 1200 and 2000 faculty respectively, and the programs have only the one full-time person to run faculty development. So these developers describe needing lots of collaboration across the university and among faculty to extend their reach and accomplish their goals. As described earlier in this chapter, the large majority of faculty developers in this study work hard to support institutional goals and missions. At Great Lakes University, the director makes an effort to create programs, study groups, and seminars that parallel the yearly themes being promoted by the administration, sud as student-centered teaching and learning, assessment, or critical thinking and writing across the curriculum. Perhaps it is through these efforts that the faculty development directors gamer the administrative support and funding so crucial to the success of their programs. Another benefit of strong connections between faculty, administration, and faculty development is in the potential for faculty development centers to play a role in helping the institution to become a learning organization. A study on the Colorado Boulder campus (Laursen & Rocque, 2009) identified areas of systemic changes that faculty identified as important for their success, such as more flexible reward systems and work-life balance. They then provided examples where facult) development programs were instrumental in helping their universities to make 183

PAGE 197

those changes. One of the examples was from Great Lakes University, where the faculty development program was reshaping recruitment and hiring by creating opportunities for faculty opinion leaders to examine their bias in those hiring processes (Laursen & Rocque). The responses from administrators who completed the online survey strongly support the attitudes and feelings of the directors. All four of the administrators who responded to the survey said that faculty on their campus agreed or strongly agreed that they [administrators] were a champion of teaching excellence. More importantly, these administrators attributed the excellent teaching found on their campus primarily to faculty development programs in three areas: helping faculty learn the skills of excellent instruction (mean 4. 75/5), helping creatt a climate of excellent instruction (mean 4.7/5), and being a pivotal element in creating the norm of excellent teaching (mean 4.5/5). It is clear that these administrators strongly valued their faculty development programs. They also supported them in several additional ways: through initiatives and policies to improve instructional quality, to establish rewards for excellent teaching, and to hold teachers accountable for quality teaching; by attending the faculty development programs; and by setting expectations for deans and faculty to participate in such programs. The strength of their support was demonstrated in the frequency that administrators engaged in these activities. The frequency of their support ranged from one to two times in the 184

PAGE 198

PAGE 199

lacked a shared purpose with their peers, were not only demoralized but found it difficult to provide a supportive learning environment for their students. In contrast, the faculty development directors at DEEP campuses described a positive climate for teaching and learning on their campuses, which they and thei1 administrators helped to develop and which was supported by the administration and other units across the institution. It may well be that these successful relationships with administrators provided a necessary requisite for successful and effective programs that could then impact faculty in ways that helped them become more successful teachers. Question 3 The third question concerned elements of faculty development structure and practice at DEEP schools that differ from best practice; I was looking for something unique about DEEP faculty development programs that was perhaps innovative and outside the list of best practices. Again, I was hoping to clarify what it was that the DEEP faculty development programs offered to their schools that might have made a contribution to the effective teaching practices that have been documented on those campuses. While no magic bullets nor astounding innovatiom were revealed, there were degrees of emphasis that were different on these campuses compared to faculty development programs in institutions outside the DEEP cohort. 186

PAGE 200

Of the best practices used as a baseline for this study, only two areas emerged from the DEEP schools that differed from recommended best practice. From the following recommendation, "Design and implement ongoing, engaging development activities with appropriate practice, follow up, and support for faculty," the DEEP directors didn't say much in the survey or interview to indicate that they provided time or opportunities for their faculty to practice new teaching behaviors. However, all the other elements of that recommendation were strongly supported in the survey and through interview comments. The other area that departs from the recommendations was in the area of assessment. While this group used assessment more extensively than non-DEEP institutions, the DEEP directors did not demonstrate nor describe how they used assessment feedback to make appropriate changes in their activities or programs, which is one of the recommended best assessment practices. They all talked extensively about assessment but didn't indicate the role of those practices in guiding subsequent changes in their programs. I might surmise that they did make changes based on the feedback from assessment, but the narrative evidence was lacking. While at least some ofthe directors may follow these two indicators ofbest practice, discussions about them were noticeably absent. As mentioned earlier in this chapter, these directors didn't seem to focus only on teaching skill development, but rather had a stronger focus on effective pedagogy and changing 187

PAGE 201

and improving the culture of teaching excellence on their respective campuses. While they all described various skill sets that they included in their faculty development activities and pedagogies that they were helping their faculty investigate and adopt, their greater concern and focus was on the bigger picture: building an environment that values and supports good teaching. That emphasis was described in the online survey results of DEEP directors' ratings of practices important to the success of their faculty development program. The highest rated practice was "establish a positive climate for teaching and learning," followed in order by "network among faculty across departments," "collaboration among faculty in activities and programs," "establish learning communities," and last among the top five, was "teaching improvement workshops". While the DEEP directors certainly use workshops to help faculty learn more effective approaches to teaching, their responses indicate they perceive a greater impact on teaching behaviors through creating changes in the culture and peer norms around teaching excellence rather than through direct instruction. Only one unique area of focus that was common among the DEEP schools, and wasn't specifically highlighted as a best practice in the literature, was the focus on learning communities. Because there was scant mention of faculty learning communities (FLC) in best practice literature, it was not included in the original questions nor was it one of the early nodes; however, because of participant responses, questions about FLC were added and that topic garnered 37 responses 188

PAGE 202

from 10 ofthe 13 participants in the interview. Some directors used FLC as their primary mode of faculty development, some used FLC but didn't call them that, many used FLC as a part of their program offerings, and some were just starting to experiment with them. The premise of FLC, while not specifically mentioned in best practice literature, reflects several of the following recommendations for best practice and so makes it a good fit for effective faculty development: faculty ownership in design and implementation and ongoing, engaging development activities with appropriate practice, follow up, and support for faculty. The areas of best practice that are not emphasized through FLC would need the attention of faculty development directors in order to enhance the potential of FLC activities to impact faculty growth. These areas include assessment, use of feedback from assessment to make appropriate changes, and goals of student improvement. Only one of the research participants, the director from North Central University, integrated all those recommended practices into his faculty learning communities. Question 4 The last question was about the perceived influence of faculty development at DEEP schools on effective faculty approaches to teaching. This question tried to clarify ways that faculty development influenced the successful teaching practices 189

PAGE 203

descnbed earlier m this chapter and used to a greater degree at DEEP schools than at other comparable institutions. As summarized at the end of chapter 4, responses to this question included three themes: community building, helping faculty understand effective pedagogy, and motivating faculty to become better teachers. Each of these factors was perceived as important, in itself, for improving teaching practices at these campuses; however, the biggest impact occurred when all these elements worked in synergy to create a teaching culture that defined, expected, and supported excellence. The following interactions may explain how that synergy was created. As faculty discuss the goals and challenges of their teaching experiences and share ideas of how to be excellent teachers, in ongoing, regular meetings, discussion groups, or in learning communities, strong relationships are built. These relationships provide the support faculty need in order to take the risks inherent in change when trying new ways of teaching. The director at North Central University describes how these connections and collaborative opportunities among faculty, afforded by faculty development programs, might help promote change by providing the scaffolding for trying new practices (Wiodkowski, 1999). Faculty have these connections across departments, so that just broadens who they know, who they can call on for support, who they can work with if they want to get started in cooperative learning ... people here on campus who have done those communities, facilitated them, are willing to provide help and assistance in trying out new things. North Central U. 190

PAGE 204

MotivatiOn to try new 1deas and ways of teachmg is increased through this kind of collegial support and the increased motivation, then, decreases the perceived risks inherent in behavior change. Research from social cognitive theories of motivation indicates that the increase in self efficacy or self confidence leads to greater effort and greater persistence in tasks associated with behavior change (A. Bandura in Bess, J.L, 1997). These common viewpoints, shared perceptions, and common goals for teaching and learning, that are developed in the discussions and cross-departmental venues instituted by faculty development programs, can become community norms. These shared experiences and attitudes, then, may form the building blocks for a culture of excellent teaching and learning that we see at the DEEP schools. Kuh (2005) described a climate of institutional focus on improved student learning mirrored across all the DEEP campuses in departments, faculty development programs, school missions, and administrative initiatives. The responses from the DEEP faculty development directors and their administrators corroborated Kuh's results. What then might be the factors that facilitate the synergy found on the campuses in this study? One factor might be the effective, supportive relationships described by participants and fostered through their faculty development programs: among faculty and faculty development programs; between faculty development directors and administrators; and among students, faculty, faculty developers, and 191

PAGE 205

admtmstrators. Just as in any good relationship, these participants and members of their institutions took the time and made the effort to develop, nourish, and sustain these relationships. The relationships became the glue that held the building blocks of cultural change together. The relationship between faculty development directors and their administrators has been thoroughly described earlier in this chapter but the relationship between the directors and their faculty deserves additional representation. Every one of the directors offered glowing comments about their faculty; many of the responses centered on how dedicated their faculty were to student success. "All teaching, learning, and faculty development revolve around the goal of improved student learning. Faculty are extraordinary-committed to students". Pacific State University "Student success is the nature of faculty here ... Our faculty are inherently dedicated to students". Western Catholic University "Student success has grown out of our faculty's high expectations of students ... Our faculty are really committed; they care about the well being of our students. We want to see our students succeed; we want our students to change the world in a positive way". Central College Others perceived their faculty as outstanding people. At Great Lakes University, the director described her faculty this way. They're excellent people in the first place. We have very capable people. And they're very ambitious; they try to do a good job. They do not like to find that students don't like their course or don't like their teaching. They have pride. They don't want to see lousy evaluations at the end of the term. Great Lakes 192

PAGE 206

In the 15 hours of recorded conversation and almost 500 pages of transcribed responses from the 13 participants in this study, none of the directors had anything but positive comments about their faculty. There were no complaints about faculty. All 13 participants in the interviews believed they had exceptional faculty. While that is statistically unlikely, their uniformly positive attitude toward their faculty provides an indication of the collegial, mutually supportive relationships that are the norm across these campuses. The following directors highlighted the culture of collegiality that exists on their campus. In terms of collegiality it's sort of a horizontal academic culture ... I think we create opportunities to learn together about what we care about as teachers. Midwest Women's College And the whole point, and then we get together after the two weeks, we just sort of have a little social and the idea is to just see what other people are doing and start a conversation. Western Catholic We just support our faculty and give them time to meet with each other, plan, work the details of this thing out. And every program, I mean the work we do in the summer, sketches the year for us. But every single week the faculty have to fine tune that thing. We spend a lot of time with our programs, a lot of time; team members socialize. Tomorrow my team and our families are all getting together for dinner, so we are each other's absolute best friends. Pine State So much of what we do, you know it sounds like socializing .... Four days a week we have lunch in this center for faculty to do some kind of program. I'm encouraging collaborative institutional projects ... and those have been wildly successful. I originally started calling [the center], as a joke, the 'Center for Climate Change at Central College.' Central 193

PAGE 207

The faculty development director from Central College may be onto Perhaps the most important aspect of faculty development that has an impact on effective teaching is the climate change. This change in culture, created through all the best practices described by the DEEP directors might be the 11ecessary ingredient that faculty need in order to adopt the teaching approaches we know lead to improved learning. Perhaps faculty developers don't have to reach :!very member of the faculty in their workshops and seminars to create the change they are seeking, but rather could attend to the relationships among faculty and ldministrators, described here, that facilitate cultural change and thus stimulate the process that supports successful teaching and learning. In summary, the faculty development practices and structures that seem to influence faculty approaches to teaching involve collaboration with faculty to jesign, support, and implement various development opportunities; collaboration with administrators to align development programs with institutional themes and goals; creation of a collegial environment for learning; the use of multiple tools and levels of assessment for programs and activities; and the development of mutually mpportive relationships among faculty developers, faculty across the campus, and ldministrators. The directors and administrators in this study demonstrated a focus ::m improving the climate for quality teaching and learning rather than just focusing :m skill acquisition. Improved student persistence and improved student learning 194

PAGE 208

were the pnmary goals of everyone in the institution and assessments of faculty development programs and activities reflected those institutional goals. Implications of Findings The original conceptual framework, described in Chapter I that guided the research in this research study was focused on processes and their interactions rather than people and their relationships. The organizational context was viewed through the lens of research-oriented versus teaching-oriented climate, not the nature of the relationships among faculty, faculty developers, and administrators. The faculty development program was the initial focus of investigation, not the faculty developer as an individual who builds and nurtures relationships. The faculty variable was described as "faculty approaches to teaching, not the individual faculty members and their relationships to each other, faculty developers, and administrators. As data were collected and analyzed, a different perspective emerged. The most important aspects of those three variables became the people and their interactions and relationships, not only the processes. Qualitative research and individual responses opened a window into the personal aspects of these variables, the relationships among them, and their influence on effective educational practices of faculty, faculty developers, and administrators. Therefore, the investigation began to include ways in which individual faculty developers impacted faculty members through interactive and supportive 195

PAGE 209

experiences to build knowledge, confidence, and trust. These elements seemed to be prerequisites to a change in norms and climate that, then, supported individual faculty changes in their approaches to teaching. Further, faculty developers described how engaged and supportive faculty, as well as administrative support, facilitated the achievement of their goals. So the direction of influence in the theoretica] framework described in Chapter I became bidirectional, not just a one way influence. Faculty developers do not do something to faculty or provide something to faculty to make them more effective educators. Faculty developers and faculty work together in symbiotic relationships to construct a climate of respect for teaching and teaching excellence that results in more effective teaching and learning opportunities in the institution. Input from the administrators clarified the importance of their relationships with faculty developers and with faculty. Their views highlighted not only the importance of administrative support for faculty and faculty deve]opers but also the importance of support from faculty and faculty developers for administrative initiatives and goals. All three variables, faculty, faculty developers, and administrators, influence each other and are influenced by each other. Working together they create an institutional climate that welcomes and supports efforts toward change and toward effective educational practices described in Pascarella & Terrenzini's work (2005) and by Kuh et al. (2005) at the DEEP schools. 196

PAGE 210

As research data were collected and analyzed, the processes, of faculty development, approaches to teaching, and organizational context became the venues through which the above-mentioned, mutually complementary relationship5 work. The original research questions were looking for specific structures and practices in faculty development programs that helped faculty learn and use effective teaching practices. It was a somewhat mechanistic viewpoint: if we do this activity, we will get the desired outcome. What has emerged is a more comple;. and interactive view of change and learning for faculty and faculty developers: a holistic web of interactions and interdependent relationships. This view is neither new nor surprising. There is abundant support, in the literature on student learning, persistence, and achievement, for the importance of relationships among students and faculty, students and their peers, and students and organizational units in higher educational institutions (Pascarella & Terrenzini, 2005). The strength or' those relationships impacts the strength of student persistence and achievement. While faculty are commonly more capable and skilled learners than their students, trying new behaviors and approaches to teaching requires effort, time, and thus motivation to be successful. Just as relationships enhance student learning and effort, faculty relationships enhance their willingness and ability to step outside their comfort zone to try more effective approaches to teaching. Currently accepted motivational theories provide a sound 197

PAGE 211

rationale for the best practices that guided this research study. [need citations from my psych book] While the outcomes of this research study clearly apply to the DEEP schools, faculty developers at other institutions might use this information to instigate and facilitate the described relationships through the processes detailed in this research study. They could create the bridges between faculty and administrators as well as the connections among the various disciplines, each with their own unique goals and perspectives, through relationship building and through the supportive structures and processes detailed in this research. Faculty developers in this research study had a central role in the web of these relationships. Through the perspectives, activities, and processes, described in this research study, faculty developers at other colleges and universities could help build institutions where all members are focused on student leaming and collaborate with others across the campus to reach their goals. Not only might their efforts improve student outcomes but through their collaborative endeavors, faculty developers could create a supportive and collegial environment where faculty, students, and administrators could find success and satisfaction in their academic lives. Limitations of this Study 198

PAGE 212

The research ofthis research study worked backwards from student outcomes and teacher behavior to the faculty development practices and structures at the DEEP-defined high performing schools to identify and describe relationships among those variables. However, while this study included faculty and student data from the NSSE and DEEP studies at the participating schools, this research format did not establish a direct or causal link between a particular faculty development practice, specific teaching behaviors of particular teachers, and student outcomes. Also, since faculty at these schools were not interviewed directly, the researcher's understanding of the phenomenon was based on perceptions and, therefore, can not presume cause and effect. Additionally, no faculty developers from schools outside of the DEEP cohort were included for comparison, thus the transferability, or external validity of my research findings is diminished. As with all interview procedures, faculty directors would likely have a degree of bias in favor of their programs that may have affected the validity of the information they communicated to me. While I attempted to diminish the bias through confirmatory interviews with administrators, and triangulation of data frorr interviews, surveys, and artifacts, all information would not be free of self-interest. However, the high response rate from the DEEP schools for the director's survey (60%) and the director's phone interview (65%) as well as a response set from all institutional types in the interview data helped to improve data validity or qualitative credibility. The triangulation of evidence provided through various 199

PAGE 213

viewpoints, such as, administrator surveys, director phone interviews and a different set of director surveys, as we11 as triangulation of methods, lent strength to the findings but could not eliminate all bias. Another obvious limitation is that high performance institutions may distort the role of culture in faculty development. All these institutions had strong cohesive cultures on their campuses. That may be a precondition for them to grow a culture of teaching excellence by using the connections already existing. Furthermore, respondents may not be a representative sample of DEEP schools, even though they did represent a11 the classifications in higher education. The best directors, or those with a predilection for community building may have self selected to become a part of this study thereby skewing the results in favor of co11aboration and community building. Another limitation is that the information in the surveys and interviews came from self report. Even though the directors' responses were corroborated by the administrators on these campuses, all the data from participants was from self report. There was some direct evidence from online information from their websites; however, most of those data concerned their activities and programs but didn't provide a window into the relationships that were integral to the effectiveness of these programs. The language of the survey instruments provided another limitation to this study. Some of the language or phrases in the survey questions may have been 200

PAGE 214

ambiguous to the participants. Therefore, relevant analyses of the survey data was compromised. While the final list of codes was reviewed by an expert in faculty development for appropriate organization of data and accuracy, because of time an< work constraints, no inter-rater coding was done to strengthen inter-rater reliability of my coding scheme. The qualitative trustworthiness or consistency of the data in this study depended on replication within the data set, not, as it does in quantitative reliability, on replication by other people or other times. However, the research results could have been strengthened through a comparison check of response coding between myself and a colleague. In a related issue, some of the survey data tables may have been misleading or unclear because the responses from directors at large schools and small schools, those with a center, and those without a center were combined thereby obscuring potentially relevant results. It might have been more helpful to see how the survey responses varied by demographics. Finally, all the support for the effectiveness of these schools and teachers came from the NSSE study, Documenting Effective Educational Practice (Kuh, et al., 2005). None of my baseline data were linked directly to student outcomes, only those measured by NSSE. Since there are limitations of the NSSE data, discussed in chapter 2, the results of my research should be interpreted with those limitations in mind. 201

PAGE 215

lmpllcatwns tor Further Research Since this research was not designed to establish a causal relationship between faculty development practices and structures, teaching behaviors, and student outcomes, a quasi-experimental research approach or action research would help connect the dots between those variables. The challenges of that research would be lack of time and financial resources. Further research that could help clarify or validate these findings could include investigation of faculty development relationships to teaching effectiveness and student performance at non-N SSE schools, compared to the results of this study. Another avenue of research would be to get input from faculty at the DEEP schools about their perceptions of faculty development efforts at their schools and its impact on their approaches to teaching. It might also be helpful to investigate student perceptions of teacher behaviors following faculty development programs or activities. Conclusions Although this research study has limitations that impact the interpretation of results, there are many insights that are important to faculty developers struggling with the multiple demands of today's students, faculty, and administrators, in a complex and economically challenging environment. The position of faculty developer is pivotal in creating a positive institutional climate for teaching and learning excellence as well as for building the relationships that form the basis for a 202

PAGE 216

high pertormmg umverslty or leammg orgamzation. The faculty developer has a finger on the pulse of all units across the institution and has the opportunity, through those contacts to build strong, mutually supportive relationships among faculty, faculty development, other institutional units, and administrative elements. The successful faculty developers in this study were hubs of communication. They reached out to all faculty, units, departments, schools, and administrative levels to connect and provide venues of engagement and collaboration among these various groups. In addition, they listened carefully to the input from those groups and to the needs of individual members. With the help of representatives from across the institution, the faculty developers in this study created programs, activities, and plans to enhance teaching and learning excellence as well as institutional excellence. They included input and needs from all constituents and also developed the framework for community building and trust, the basis of all relationships. With the help of others, they set the tone and set the pace for supportive institutional change and effectiveness. Admittedly, the challenge for faculty developers at research universities to build a climate for teaching excellence can be daunting. The four faculty developers from research intensive universities in this study had developed strong support from their administrators through personal relationships, alignment of faculty development efforts with administrative goals, and effective faculty 203

PAGE 217

development practices such as extenstve assessment that demonstrated thetr effectiveness. As described in other research, without such relationships and trust, faculty were not only demoralized but found it difficult to provide a supportive learning environment for their students. And in the end, it is the students that feel the results of our efforts in faculty development. As universities strive to help students persist and graduate to be effective 21st century learners, employees, and citizens, faculty developers can take the opportunities afforded by their central position to facilitate not only excellence in teaching and learning but through the connections and venues described here, promote a climate that fosters excellence throughout the institution. Our students thirst for such an environment. Our times demand it. 204

PAGE 218

PAGE 219

PAGE 220

2. In your perception, to what degree and in what ways do faculty development efforts contribute to the high quality of teaching activities and characteristics founc on your campus? C. Campus climate l. Please describe factors that have contributed to resistance to or openness to change on your campus. D. Organization l. Please describe ways that faculty development contributes to a climate of goodwill and trust among faculty, staff, administration, and students. 207

PAGE 221

PAGE 222

Approximately, what percent of your work time is spent on the faculty 6 development program? I Please indicate your approximate program budget, excluding salaries, for faculty 7 development per year. 8 I What do you do to further your own professional development in the area of faculty development? Please check all that apply. Member of POD (Professional and Organizational Development Network in Higher Education) Member of other faculty development organizations. Member of other higher education organizations. Attend faculty development conferences at least yearly. Attend faculty development conferences less than yearly. Subscribe to faculty developmentjoumal(s). Read books on faculty development. Other, please describe. r---Next I .ubrrit I For questions 9, 11, and 13, please rate on a Likert scale, how important each item is to the the faculty development program on your campus. 209

PAGE 223

How important are the following goals to the faculty development program on your campus? Not at all imrortant 2 Not very important Somewhat important Enhance value of teaching effectiveness. Improve student learning. Improve the learning environment. Introduce teaching strategies. 4 Quite important Extremely important 9---------------------------------------------Provide faculty development programs. Serve institutional needs. Foster faculty career development. Provide skills training for faculty. Serve faculty personal needs. Improve faculty evaluations. Please describe any additional goals you have for your faculty development __j =l _j __j II Please identify the importance of the following practices to the success of the 210

PAGE 224

faculty development program on your campus. I 2 Not at all imponant Not very imponant Smne\\hat imponant Teaching improvement workshops Providing faculty mentoring Personal assessment programs Collaboration among faculty Establishing learning communities Adapting I assessing needs Being visible I accessible Providing technical support Providing resources 4 Quite imponant 5 Extremely imponant Establishing a positive climate for teaching and learning Networking among faculty and/or across academic and administrative departments Helping write grants 12 Please describe any other best practices that have made the faculty development program at your campus successful. 211

PAGE 225

[ j _j _j Please identify the importance of the following factors to achieving the goals of the faculty development program on your campus. Not at all important 2 Not very important Somewhat important Strong administrative support Engaged and supportive faculty 4 Qu itc important Extremely important Cultural tradition of support and climate of collaboration Adequate budget Skilled and dedicated staff support 13 __________________________________________ __ Grant funding Location and physical facilities Strategic planning and goal setting Providing food and refreshments Student support Timing of program offerings such as when faculty are available and when instruction is relevant 212

PAGE 226

Including a variety of different delivery methods such as listserves Please describe any additional factors that helped your faculty development 14 [prog_jram achieve its goalsj _j _j 15 Your school participates in the National Survey of Student Engagement (NSSE). We know from NSSE data that there are elements of effective educational practice that increase student engagement. Please indicate below those areas that your faculty devlopment program has helped faculty develop or adopt. Assigning more reading and writing. Active learning techniques. Assigning group work and group projects. Increased contact with instructor. If appropriate, identify how much each of the following obstacles has hindered the faculty development program on your campus from achieving its goals. Not at all a hindrance 2 Not much of a hindrance Budget constraints Somewhat of a hindrance 4 Quite a hindrance s A strong hindrance 16 Faculty perceptions of a research culture rather than an emphasis on teaching Lack of adequate staff support Lack of faculty time to devote to development activities 213

PAGE 227

General lack of administrative support General lack of faculty support and interest Facility problems such as poor location or insufficient space No obstacles Please describe any additional obstacles that have hindered your faculty 17 [el:pment prowam :j achieving its goals. Please indicate the importance of each of the following faculty development director reporting structures to the success of a faculty development program. N01 at all important 2 N,>t very important Som,what important Quite important Extremely important Report to provost, vice president of academic affairs, or chief academic officer 18------------------------------------------Report to vice-provost or associate vice president for academic affairs Report to Dean Report to faculty governing board 19 To whom do you report? 214

PAGE 228

[ j _j _j Please indicate by the following scale, how you assess the effectiveness of the faculty development program on your campus. Never usc Ha' e used times Workshop evaluations Ha' c used 3-5 times Workshop learning assessments II a\ c used 6-9 Have used I 0 or times more times Program exit surveys to evaluate the experience (of an extended program) for participants Satisfaction I value surveys 20 Number of workshop participants Number of subscribers to new letter Frequency of contacts with faculty Periodic interviews Focus groups Institutional needs assessment Strategic planning and program review 215

PAGE 229

Please indicate which of the above assessment strategies you believe are most 21 cant to the program. Next I I Please answer the following questions about the culture on your campus. In general, how open are faculty on your campus to the following teaching improvement efforts? Very closed 2 Somewhat c lased Classroom observations 3 Not sure whether closed or open 4 Somewhat open Videotaping for instructional improvement 5 Very open 22-------------------------------------------Peer mentoring Student feedback Collaborative teaching Small Group Instructional Diagnosis (SGID) 23 Please additional teaching improvement efforts that your faculty may part1c1pate m on your campus. 216

PAGE 230

I Lu j I What is the perception of teaching among faculty on your campus? Please indicate the degree to which you agree with the following examples of faculty perceptions on your campus. Strongly disagree 2 Somewhat disagree Neither agree nor disagree Teaching is a private process 24 Teaching is a collaborative process Teaching is a learnable skill Teaching is as important as research Teaching is as rewarded as research 4 Somewhat agree Strongly agr,'c Please feel free to include a description of any other attitudes about teaching 25 r: n:ay exist on your Next/ubrrit Please answer the following questions about the various faculty development 217

PAGE 231

activities, such as workshops, forums, or training sponsored by your center or school. To what degree do the following characteristics of your center's! school's development activities contribute to the success of these activities. Not at all important Slightly important Moderately important Planned in collaboration with faculty Participant interaction Time for participant reflection Peer collaboration .. Quite important 5 Extremely important 26-------------------------------------------------Guided practice Prompt feedback for participants Follow up feedback sessions or additional information Follow up support for new behaviors Rewards for changed behaviors Other characteristics 27 How do you determine the effectiveness of your activities? Please indicate the 218

PAGE 232

degree to which you commonly measure each outcome. 2 Never measure Rarely measure Participant satisfaction Faculty attitude change Faculty belief change Faculty behavior change Faculty affective change Faculty knowledge change Student performance Student retention Institutional impact Other outcomes Sometimes measure Comnwnly measure Al"ays measure Please describe the tools you use to measure the effectiveness of your faculty development activities. 2 Rarely Sometimes 28 Questionnaires or surveys we construct Student faculty evaluations 219 4 Frequently Always almost ai\\Jys

PAGE 233

Student knowledge survey SEEQ student evaluation of educational quality CEQ course experience questionnaire A TI approaches to teaching inventory TMI teaching methods inventory Teacher Efficacy Questionnaire for faculty Other instruments Next /12ubrrit I To the best of your knowledge, how do people in your school administration support the faculty development program? No! a! 2 Occasionally Commonly bul no! frcqucmly 4 Frcqucnlly Advertise the faculty development program 5 Very 29 Speak nationally about the faculty development program Talk about the faculty development program when recruiting faculty Participate in or sponsor development programs 220

PAGE 234

Please describe any other ways your administration supports the faculty development program at your school. 30L _j : _j_j To the best of your knowledge, how do people in your school administration support teaching excellence? Not at all 2 Occasionally Hire excellent teachers Commonly but not frc
PAGE 235

PAGE 236

PAGE 237

11. In the past two years I have participated in faculty development programs. Not at all 1-2 times 3-4 times 5 or more times 1 2 3 Additional comments: 12. Faculty development and teaching quality are not a significant part of my job responsibilities. 4 Definitely disagree Somewhat disagree Somewhat agree Definitely agree 2 3 4 Additional comments: The DEEP and NSSE studies indicate that your faculty are unusually effective teachers. Please rank the following factors from 1 to 7 to indicate the degree to which each contributes to the teaching effectiveness of faculty on your campus. 1 = strongest factor influencing faculty effectiveness at this campus 7 = weakest factor influencing faculty effectiveness at this campus. __ Our policies reward excellent teachers. We hire excellent teachers. __ The campus climate encourages our teachers to continuously strive to improve. __ Faculty development programs help create a climate of excellent teaching. __ Faculty development programs help our faculty learn the skills of excellent instruction. __ The focus on good teaching at our campus sets a high standard for excellent teaching. __ Faculty development is a pivotal element in creating the norm of excellent teaching. 224

PAGE 238

REFERENCES AAC&U. (2002). Greater expectations: A new vision for learning as a nation goe.s to college. Washington, DC: American Association of Colleges and Universities. ACT Newsroom: College graduation rates steady despite increase in enrollment. 2006. Retreived March 8, 2006, from http:i/www.act.or!.!lncv.osirelcascs/2002/ 11-15-02.html. Allen, D. K. (2003). Organisational climate and strategic change in higher education: Organisational insecurity. Higher Education, 46, 61-92. Astin, A. W. (1980). When does a college deserve to be called "high quality?" In Current issues in higher education: Improving teaching and institutional quality. Washington, DC: AAHE. Austin, A. E., Brocato, J. J ., & Rohrer, J. D. ( 1997). Institutional missions, multipl1 faculty roles: Implications for faculty development. In D. DeZure & M. Kaplan (Eds.), To improve the academy: Resourcesfor facult_v, instructional, and organizational development. Stillwater, OK: New Forums Press. Barr, R. B., & Tagg, J. (1995). From teaching to learning: A new paradigm for undergraduate education. Change, 27, 12-25. Bergquist, W. H., & Phillips, S. R. (1975). Components of an effective faculty development program. Journal of Higher Education, XLV/(2), 177-210. Bess, J. L. ( 1997). Teaching well and liking it: Motivatingfaculty to teach effectively. Baltimore: The Johns Hopkins University Press. Biggs, J. ( 1999). Teaching for quality learning at university: What the student doe1 Philadelphia: Society for Research into Higher Education & Open University Press. Blackburn, R. T., Pellino, G. R., Boberg, A., & O'Connell, C. ( 1980). Are instructional improvement programs off-target? In Current issues in higher education: Improving teaching and institutional quality. Washington, DC: AAHE. 225

PAGE 239

Blackburn, R. T. & Lawrence, J .H. ( 1995). F acuity at work: Motivation, expectation, satisfaction. Baltimore: Johns Hopkins University Press. Solman, L. G., & Deal, T. E. (2003). Rejiaming organizations: Artistry, choice, and leadership. San Francisco: Jossey-Bass. Brookfield, S. D. ( 1995). Becoming a critically reflective teacher. San Francisco: Jossey-Bass. Chism, N. (2004). So what is teaching effectiveness anywaJ? Paper presented at the Annual meeting of Professional and Organizational Development, Montreal, Canada. Chism, N.V.N.(Ed.)(2008). Faculty at the margins. New Directions for Higher Education. (No 143). San Francisco: Jossey-Bass. Cowan, J., George, J. W., & Pinheiros-Torres, A. (2004). Alignment of developments in higher education. Higher education, 48, 439-459. Cox, M.D. (2004). Introduction to faculty learning communities. In Cox, M.D. & Richlin, L. (Eds.). Buildingfaculty learning communities. (p.8). San Francisco: J osseyBass. Creswell, J. (Ed.). (2008). Research Design: Qualitative, quantitative, and mixed methods approaches (3rd ed.). Thousand Oaks, CA: Sage. Dale, E. A. (1998). An assessment of afaculty development program at a research university. University of Massachusetts, Amherst, Amherst: MA. Diamond, R. M., Gardiner, L. F., & Wheeler, D. W. (2002). Requisites for sustainable institutional change. In R. M. Diamond & B. Adam (Eds.), Fie/a gzdde to academic leadership (pp. 15-24). San Francisco: Jossey-Bass. Diamond, R. M., & Adam, B. (2002). Field guide to academic leadership. San Francisco: Jossey-Bass. Eble, K. E., & McKeachie, W. J. ( 1985). lmprming undergraduate education through faculty development. San Francisco: Jossey-Bass. Eckel, P., Hill, B., Green, M., & Mallon, B. ( 1999). Reports from the road: Insights on institutional change. Washington, DC: American Council on Education. 226

PAGE 240

Edgerton, R. (2005). Exploring different dimensions of student engagement: 2005 annual survey results. Bloomington, IN. Elen, J, LindbloomYlanne, S, & Clement, M. (2007). Faculty development in research-intensive universities: The role of academics' conceptions on the relationship between research and teaching. International Journal for Academic Development, 12(2), 123-139. Feldman, K. A. (1998). Identifying exemplary teachers and teaching. InK. A. Feldman & M. B. Paulsen (Eds.), Teaching and learning in the college classroom (pp. 391-414). Boston: Pearson Custom Publishing. Frantz, A. C., Beebe, S. A., Horvath, V. S., Canales, J., & Swee, D. E. (2005). The roles of teaching and learning centers. In S. Chadwick-Blossey & D. R. Robertson (Eds.), To improve the academy. (Vol. 23). Bolton, MA: Anker Publishing. Gaff, J. G., & Morstain, B. R. ( 1978). Evaluating the outcomes. Ne-..t Directions fOJ Higher Education, 24, 73-83. Gardiner, L. ( 1994 ). Redesigning higher education: Producing dramatic gains in student/earning. Washington, DC: George Washington University, Graduate School of Education and Human Development. Gardiner, L. F. (2005). Transforming the environment for learning: A crises of quality. InS. Chadwick-Blossey & D. R. Robertson (Eds.), To improve the academy: Resourcesfor faculty, instructional, and organizational development. (Vol. 23, pp. 3-23). Bolton, MA: Anker Publishing. Gibbs, G. (2003). Researching the training of university teachers: Conceptual frameworks and research tools. In H. Eggins & R. Macdonald (Eds.), The scholarship of academic development. Philadelphia, PA: The Society for Research into Higher Education & Open University Press. Guskey, T. R. (2000). Evaluating prof'essional development. Thousand Oaks, C A: Corwin Press. Guskin, A. E. (1994). Restructuring the role of faculty. Change, 26(5), 16-25. 227

PAGE 241

Guskin, A. E., & Marcy, M. B. (2002). Pressures for fundamental reform: Creating a viable academic future. In R. M. Diamond & B. Adam (Eds.), Field guidE to academic leadership. San Francisco: Jossey-Bass. Hat iva, N. (2000). Becoming a better teacher: A case of changing the pedagogical knowledge and beliefs of law professors. Instructional Science, 28, 491523. Hativa, N., & Goodyear, P. (2002). Research on teacher thinking, beliefs, and knowledge in higher education: foundations, status, and prospects. InN. Hativa & P. Goodyear (Eds.), Teacher thinking, beliefs, and knowledge in higher education. Norwell, MA: Kluwer Academic Publishers. Hativa, N., & Goodyear, P. (Eds.). (2002). Teacher thinking, beliefs, and knowledge in higher education. Norwell, MA: Kluwer Academic Publishers. Hellyer, S., & Boschmann, E. ( 1993). Faculty development programs: A perspective. In D. L. Wright & J.P. Lunde (Eds.), To improve the academy Resources forfaculty, instructional, and organizational development. Stillwater, OK: New Forums Press. Hill, L., Sao La, K., & Lagueux, R. (2007). Faculty collaboration as faculty development. Peer Review, 9(4), 17-19. Holmgren, R. A. (2005). Teaching partners: Improving teaching and learning by cultivating a community of practice. InS. Chadwick-Blossey & D. R. Robertson (Eds.), To improve the academy: Resources for faculty, instructional, and organizational development. (Vol. 23). Bolton, MA: Anker Publishing. Kember, D., & Kwan, K-P. (2002). Lecturers' approaches to teaching and their relationship to conceptions of good teaching. InN. Hativa & P. Goodyear (Eds.), Teacher thinking, belief'>, and knoH"Iedge in higher education. Norwel, MA: Kluwer Academic Publishers. Kember, D., & Gow, L. ( 1994). Orientations to teaching and their effect on the quality of student learning. Journal of Higher Education. 65( I), 58-74. 228

PAGE 242

Kmg, K. P., & Lawler, P. A. (2003). Best practices in faculty development in North American higher education: Distinctions and dilemmas. Journal of Faculty Development, 19{1), 29-36. Kinzie, J, Kuh, G. (2004). Going DEEP: Learning from campuses that share responsibility for student success. About Campus, Nov-Dec. Knafl, K., & Breitmayer, B. J. (1989). Triangulation in qualitative research: Issues conceptual clarity and purpose. In J. M. Morse (Ed.), Qualitative Nursing Research (pp. 193-203). Rockvillle, MD: Aspen. Knight, A.M., Carrese, J. A., & Wright, S.M. (2007). Qualitative assessment of the long-term impact of a faculty development programme in teaching skills. Medical Education, 41(6), 592-600. Krefting, L. ( 1991 ). Rigor in qualitative research: The assessment of trustworthiness. American Journal ofOccupational Therapy, 45, 214-222. Kuh, G. D., Nelson Laird, T. F., & Umbach, P. D. (2004). Aligning faculty activities and student behavior: Realizing the promise of greater expectations. Liberal Education, 90(4), 24-36. Kuh, G. D., Kinzie, J., Schuh, J. H., Whitt, E. J., & Associates. (2005). Student success in college: Creating conditions that matter. San Francisco: Jossey Bass. Laursen, S., & Rocque, B. (2009). Faculty development for institutional change: Lessons from an advance project. Change, 41 (2), 18-26. LevinsonRose, J., & Menges, R.J. (1981 ). Improving'college teaching: A critical review of the literature. Review of Educational Research, 51 (3 ), 403-434. Lick, D. W. (2002). Leadership and change. In R. M. Diamond & B. Adam (Eds.), Field guide to academic leadership (pp. 27-46). San Francisco: Jossey Bass. Mayan, M. J. (2009). Essentials of qualitative inquily. Walnut Creek, CA: Left Coast Press, Inc. McAlpine, L., & Saroyan, A. (2004). Toward a comprehensive framework of faculty development. In A. Saroyan & C. Amundsen (Eds.), Rethinking 229

PAGE 243

teaching in higher education: From a course design workshop to a faculty development framework. Sterling, VA: Stylus Publishing. Menges, R. J. (1980). Teaching improvement strategies: How effective are they? In AAHE (Ed.), Current issues in higher education: Improving teaching and institutional quality. Washington, DC: AAHE. Miles, M. B., & Huberman, A. M. ( 1994). Qualitative analysis: An expanded sourcebook. Thousand Oaks, CA: Sage. Mooney, K. (2002). The ten demandments: Rules to live by in the age of the demanding consumer. New York: McGraw Hill. Murray, J.P. (2000). Faculty development in Texas two-year colleges. Community College Journal of Research and Practice, 24, 251-267. Pascarella, E. T., & Terrenzini, P. T. (2005). How college affects students: A third decade of research. San Francisco: Jossey-Bass. Piantanida, M., & Garman, N. B. ( I999). The qualitative study: A guide for students andfaculty. Thousand Oaks, CA:Sage. Porter, S. R.(2009). Paper presented at the 2009 meeting of the Association for the Study of Higher Education. Ramaley, J. A. (2002). Moving mountains: Institutional culture and transformational change. In R. M. Diamond & B. Adam (Eds.), Field guide to academic leadership. San Francisco: Jossey-Bass. Roche, V. (200 I). Professional development models and transformative change: A case study of indicators of effective practice in higher education. The International Journalfor Academic Development, 6(2), 120-129. Rust, C. ( 1999). The impact of educational development workshops on teachers' practice. International Journal for Academic Development. 3( I), 72-80. Saroyan, A., Amundsen, C., & Li, C. (1997). Incorporating theories of teacher growth and adult education in a faculty development program. In D. DeZun 230

PAGE 244

& M. Kaplan (Eds.), To improve the academy (Vol. 16, pp. 93-116). Stillwater, OK: New Forums Press. Sorcinelli, M. D. (200 1 ). Ten principles of good practice in creating and sustaining teaching and learning centers. In K. H. Gillespie, L. R. Hissen & E. C. Wadsworth. (Eds.), A guide to faculty development: Practical advice, examples, and resources. Bolton, MA: Anker Publishing. Sorcine1li, M.D., Austin, A. E., Eddy, P. L., & Beach, A. L. (2006). Creating the future o.ffaculty development: Learning from the past, understanding the present. Bolton: MA: Anker. Stevens, E. (1988). Tinkering with teaching. Higher Education, 12(1 ), 63-78. Tagg, J. (2003). The learning paradigm college. Bolton: MA: Anker. Tiberi us, R. G. ( 1995). From shaping performances to dynamic interaction: The quiet revolution in teaching improvement programs. ln W. A. Wright & associates (Eds.), Teaching improvement practices: Successful strategies OJ higher education. Bolton, MA: Anker Publishing. Wehlburg, C. M. (2005). Using data to enhance college teaching: Course and departmental assessment results as a faculty development tool. In S. Chadwick-Blossey & D. R. Robertson (Eds.), To improve the academy: Resources for faculty, instructional, and organizational development. (Vol. 23). Bolton, MA: Anker Publishing. Weimer, M., & Lenze, L. F. ( 1998). Instructional interventions: A review of the literature on efforts to improve instruction. In K. A. Feldman & M. B. Paulsen (Eds.), Teaching and learning in the college classroom (pp. 653682). Boston: Pearson Custom Publishing. Wray, M. L., Bates, B. A., Switzer, T., Lowenthal. P., Stevens, E., & Dunlap, J. (2005). Examiningfaculty development attitudes and motivators to challenges of improving higher education in the public interest. Paper presented at the Professional and Organizational Development Network annual conference, Montreal, Canada. Zahorski, K. J. (1993). Taking the lead: Faculty development as institutional change agentry. In D. L. Wright & J.P. Lunde (Eds.), To improve the 231

PAGE 245

academy: Resources for faculty, instructional, and organizational development. (Vol. 12, pp. 227-245). Stillwater, OK: New Forums Press. 232