Citation
Teacher reflection in the twenty-first century

Material Information

Title:
Teacher reflection in the twenty-first century deriving practical relevance from experiences through digital storytelling
Creator:
Renda, Christine Ann
Publication Date:
Language:
English
Physical Description:
xiii, 173 leaves : ; 28 cm

Subjects

Subjects / Keywords:
Reflective learning ( lcsh )
Digital storytelling ( lcsh )
Relevance ( lcsh )
Teachers ( lcsh )
Digital storytelling ( fast )
Reflective learning ( fast )
Relevance ( fast )
Teachers ( fast )
Genre:
bibliography ( marcgt )
theses ( marcgt )
non-fiction ( marcgt )

Notes

Bibliography:
Includes bibliographical references (leaves 164-173).
General Note:
School of Education and Human Development
Statement of Responsibility:
by Christine Ann Renda.

Record Information

Source Institution:
|University of Colorado Denver
Holding Location:
|Auraria Library
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
707492912 ( OCLC )
ocn707492912
Classification:
LD1193.E3 2010d R46 ( lcc )

Full Text
A
TEACHER REFLECTION IN THE TWENTY-FIRST CENTURY:
DERIVING PRACTICAL RELEVANCE FROM EXPERIENCES
THROUGH DIGITAL STORYTELLING
by
Christine Ann Renda
B.S., North Carolina State University, 1977
M.S., Colorado State University, 1984
A thesis submitted to the
University of Colorado Denver
in partial fulfillment
of the requirements for the degree of
Doctor of Philosophy
Educational Leadership and Innovation
2010


2010 by Christine Ann Renda
All rights reserved


This thesis for the Doctor of Philosophy
degree by
Christine Ann Renda
has been approved by
Michael P. Marlow
Alan Davis

Date


Renda, Christine Ann (Ph. D., Educational Leadership and Innovation)
Teacher Reflection in the Twenty-First Century: Deriving Practical Relevance
from Experiences through Digital Storytelling
Thesis directed by Associate Professor Michael P. Marlow
ABSTRACT
This study introduces the concept of practical relevance of reflection upon
learning experiences at the graduate level by practicing or soon-to-be practicing
teachers. Practical relevance encompasses the idea that as teachers think more
deeply about topically relevant matters, their reflections on experiences can be
deemed more relevant to their practice. A mixed methods approach was
employed to devise and test a measure of the concept The measure was applied
to reflections created through traditional and twenty-first century multimedia
approaches, namely journals and digital stories, by teachers attending a space-
science education conference as a group. Practical relevance values for the two
types of reflection were compared. Results indicate that practical relevance
values inferred from digital stories met or exceeded those inferred from journals
for 76% of participants (n=34). In addition, two of five factors used to compute
a multiple regression were found to predict practical relevance with statistical
significance. Results suggest that digital storytelling is a viable and possible
alternative reflection vehicle for some twenty-first century teachers, particularly
if factors predicting practical relevance can be better understood. More work is
warranted on these counts as digital options expand for teachers and teacher
preparers.
This abstract accurately represents the content of the candidates thesis. I
recommend its publication.
Signed.
Michael P. Marlow


DEDICATION
I dedicate this publication to:
My husband, colleague, and best friend, David Sprouse,
for encouraging me every step of the way.
My mother, Sigrid Renda,
for instilling in me a love of education.
My father, Matthew Renda,
for inspiring me to follow my dream.
He would be proud of me, today.
My sons, Jonathan and Steven Goodbread,
for making my world shiny.
My dear friend, Dr. Lloyd Lewan
for nudging me down the path to begin this journey.


ACKNOWLEDGEMENT
I wish to acknowledge and thank those who helped me with this study:
My advisor and chair, Dr. Michael Marlow,
for introducing me to the power and joy of extraordinary experiences
and for providing me with the research opportunity
which led to this dissertation,
My instructor and committee member, Dr. Alan Davis,
for sharing his methodological expertise
and refreshing approaches
to educational research,
My instructor and committee member, Dr. Joanna Dunlap,
for providing essential reality checks
through thoughtful questions,
My committee member, Dr. Karen Johnson,
for grounding me in the world
of teachers,
My writing instructor and editor, Dr. Marcia Muth,
for offering helpful advice on so many matters
and for helping me to be a better writer,
My husband and colleague, David Sprouse,
for tirelessly serving as a
second analyst,
My son and colleague, Jonathan Goodbread,
for generously serving as an
independent third analyst,
and
All of the teachers who consented to be part of this study,
for without them, there would have been no study.


TABLE OF CONTENTS
FIGURES.........................................................xii
TABLES..........................................................xiii
CHAPTER
1. THE RESEARCH PROBLEM...........................................1
Background and Overview.....................................1
Significance of the Study...................................4
Problem Statement...........................................6
The Role of Reflection in Experiential Learning........6
Reflection in the Twenty-First Century.................7
Content, Level, and Relevance of Reflection............8
Needs Identified from the Literature Review............9
Study Purpose and Research Questions.......................10
Methodology................................................11
Structure of the Dissertation..............................12
2. REVIEW OF THE LITERATURE AND THEORETICAL
ORIENTATION..................................................13
Introduction...............................................13
Review of the Literature...................................14
Reflection
14


Narrative as Reflection
17
Digital Storytelling as Narrative Reflection..............19
Evidence of Reflection through Artifacts..................23
Process and Content of Reflection.........................24
Reflection Frameworks.....................................24
Relevance of Reflection...................................26
Theoretical Orientation........................................28
A Multidimensional Reflection Framework...................28
Reflective Aspects of Visual Subelements in Digital
Stories...................................................36
Conceptual Framework of Practical Relevance...............39
3. METHODS AND PROCEDURES............................................46
Introduction and Overview......................................46
Participants...................................................47
Overview of Study Participants............................47
Profiles of Five Selected Participants....................49
2009 and 2010 Settings.........................................53
Overview of Settings......................................53
Researcher's Role in the Class and Recruitment of
Participants..............................................54
Preconference Technology Workshops and Ongoing
Support...................................................55
viii


Reflection Instructions
59
The Houston Experience.....................................61
Digital Story Creation.....................................64
Final Night Presentations..................................66
Data Collection................................................66
Pre and Postclass Surveys..................................67
Online Wiki Reflections and Journal Entries................68
Storyboards, Scripts, and Image Logs.......................68
Picasa and Ning Postings...................................69
Digital Stories............................................69
Final Night Observation Notes..............................69
Research Design................................................70
Overview of Research Design and Methods....................70
Methods for Transformining Qualitative to
Quantitative Data..........................................72
Qualitative + Quantitative (QUAL + quanj Methods...........78
Qualitative Methods........................................79
Quantitative Methods.......................................81
Study Limitations..............................................85
Trustworthiness, Validity, and Reliability.....................86
IX


4. RESULTS OF THE STUDY
88
Overview of Results......................................88
Results for Research Question 1 Concerning Practical Relevance
Derived from Journals and Digital Stories................89
Results for Research Question 2 Concerning Topics of
Deep Importance to Participants..........................94
Themes Emerging from Journals and Digital Stories
of Representative Participants.......................94
A Closer Look at Results for Five Representative
Participants.........................................99
Results for Question 3 Concerning Predictors for Practical
Relevance...............................................118
5. DISCUSSION AND IMPLICATIONS FOR FUTURE STUDY..............122
General Discussion......................................122
Discussion of Results for Research Question 1...........127
Discussion of Results for Research Question 2...........130
Discussion of Results for Research Question 3...........131
Implications for Future Study...........................133
APPENDIX
A. PARTICIPANT CHARACTERISTICS...............................136
B. CALCULATION OF PREDICTOR VARIABLE: TECHNOLOGICAL
COMPETENCY.................................................140
C. PROMPTS FOR JOURNAL REFLECTIONS AND INSTRUCTIONS
FOR DIGITAL STORIES FOR 2009 AND 2010 CLASSES..............142
x


D. PRE AND POSTCLASS SURVEY ITEMS INFORMING THIS STUDY
AND LONGITUDINAL SURVEY ITEMS......................146
E. SUPPLEMENTAL MATERIAL FOR DETERMINING LEVELS OF
REFLECTION.........................................154
F. EMERGING THEMES FOR REPRESENTATIVE JOURNALS AND
DIGITAL STORIES....................................160
REFERENCES................................................164
xi


LIST OF FIGURES
Figure
2.1 Compounding Effect of Multimodal Elements on Reflection
through Digital Storytelling.........................................21
2.2 Level-of-Reflection Criteria and Modes of Connection for Visual
Subelements...........................................................38
2.3 Conceptual Framework Depicting Practical Relevance as a
Product of Topical Relevance and Level of Reflection.................40
2.4 Conceptual Framework Depicting Practical Relevance for
Each Reflection Element as a Product of Its Respective Topical
Relevance and Level of Reflection.....................................41
2.5 Conceptual Framework Depicting Practical Relevance for a Complex
Reflection Artifact with Elements Embedded in a Core Message.........43
3.1 Modified, Single-Phase Mixed-Method Triangulation Design-
Transformation Model Depicting Flow of Analysis Directed
toward Three Research Questions.......................................71
4.1 Practical Relevance of Journals and Digital Stories Plotted in
Order of Increasing Differences Between Respective Pairs..............89
4.2 Practical Relevance Values for Ginas Journal and Digital Story......102
4.3 Practical Relevance Values for Leo's Journal and Digital Story.......106
4.4 Practical Relevance Values for Quinn's Journal and Digital Story.....109
4.5 Practical Relevance Values for Teds Journal and Digital Story.......113
4.6 Practical Relevance Values for Ursulas Journal and Digital Story....116
xii


LIST OF TABLES
Table
2.1 Reflection framework with modified cognitive and
experiential domains..................................................29
3.1 Summary of characteristics of study participants.......................48
3.2 Characteristics of five representative study participants..............49
3.3 Technologies presented during the workshops............................56
3.4 Data sources and uses..................................................67
3.5 Multiple regression variables and their properties.....................83
4.1 Ranges and median values for practical relevance of elements
in journals and digital stories of representative participants........94
4.2 Means, standard deviations, and intercorrelations for practical
relevance and predictor variables....................................118
4.3 Hierarchical multiple regression analysis summary for grade
taught, age, technological competency, controlling for year
class was taught, predicting practical relevance.....................119
A. 1 Participant characteristics...........................................137
E. l Selection of subelement category for level-of-reflection analysis.....155
F. l Emerging themes from elements with practical relevance values
at or above the median level obtained from the journals of five
representative participants..........................................161
F.2 Emerging themes from elements with practical relevance values
at or above the median level obtained from the digital stories
of five representative participants..........................................163
xm


CHAPTER 1
THE RESEARCH PROBLEM
Background and Overview
This study sprang from a seed planted in 2006 when Dr. Michael Marlow
took a group of early- and mid-career science teachers to a National Science
Teachers Association conference in order to study the impact of such an
experience on teacher professional identity (Sala, Marlow, Kirtley, Renda, &
Sprouse, 2008). Teachers were asked to create digital stories as a way to help
them get the most from their conference experiences. A digital story is "a short,
first-person video-narrative created by combining recorded voice, still and
moving images, and music or other sounds (Center for Digital Storytelling,
2010, para. 1). The creation process served a dual role: It provided a mode of
reflection for teachers to process conference content, elements of the
experience, and impact on practice; and it provided a way for teachers to
demonstrate mastery of twenty-first century digital technology (Blocher, 2008;
Marlow, personal communication).
Doctoral students, who were members of Dr. Marlows Inquiry and
Professional Development Laboratory, were invited to pose research questions
1


and pursue those questions under the umbrella of Marlows study [Marlow et al.,
2006}. In this context I wondered, "What level of reflection is evident in digital
stories as compared to more traditional journals? But before I could address
that question, I needed a means to describe and measure depth of reflection.
Informed by Dewey (1933), Schon (1983), and Surbeck, Han, and Moyer (1991),
my work resulted in the development of a preliminary multi-dimensional
framework to assess level of reflection (Renda, 2006, 2010).
This dissertation further develops my framework for level of reflection
and goes the next step by introducing a concept of practical relevance of
reflection, predicated on both level of reflection and relevance of thought
pertaining to teaching practice. The study then uses a newly devised conceptual
framework (a) to derive a measure of practical relevance for elements (e.g.,
paragraphs, complete sequences of thought) found in teacher journals and
digital stories, (b) to measure the practical relevance of the core message of the
artifact, and (c) to generate an integrated measure of practical relevance for each
artifact. Practical relevance encompasses the idea that, as teachers think more
deeply on experiences that are topically relevantabout or having to do with
the day-to-day knowledge of their practice or professional identitytheir
reflections can be deemed more relevant to their practice (i.e., directed toward
practical application), regardless of philosophical orientation.
2


During my earlier work, another, as yet unanswered, question came to
mind: "How can teacher educators leverage the strengths of digital storytelling
to encourage deeper and more relevant reflection? This query begs the more
fundamental question, "What features of digital storytelling can facilitate
meaningful reflection? The latter question helped to direct my dissertation
literature review as I situated my problem in the successively narrowing areas of
reflection, narrative as reflection, and finally digital storytelling as narrative
reflection.
Subsequent experiential learning classes that were offered in 2009 and
2010 by Dr. Marlow fostered conference experiences for new groups of
preservice, early-career, and mid-career teachers and afforded me the
opportunity to develop and test the concept of practical relevance on reflection
in a specific context. This study delves into the experiences and associated
reflections of two cohorts of teachers who attended a space-science education
class, offered as an elective course at the university level. The window into the
teachers' experiences is composed of their reflection artifacts: their journals and
digital stories.
3


Significance of the Study
When coupled with reflection, rich and collegial science-based learning
experiences have the potential to enhance teachers content knowledge, promote
their understanding of scientific concepts, and encourage the sharing of effective
pedagogical strategies [Marlow, 2007]. These learning experiences afford
teachers the opportunity to build understanding in a way that, as expressed by
one study participant, Deidre [pseudonym], "really encompasses what learning
should be-experiential. She went on to say,
I learned more from my journey at NASA than anything
that I could have read from a book. The hands-on activities rise
above any textbook or article that talked about the same thing. I
enjoyed the 'real' experiences, and not just hearing about it from
someone else [Journal, 2010).
In addition, such experiences expand teachers' repertoires of illustrative
stories that can be used to provide context for scientific content and promote
engagement of students in the science classroom [Marlow, 2009).
An associated area ripe for study is that of reflection upon learning
experiences that take preservice and in-service teachers, pursuing masters level
coursework, beyond the walls of the classroom, particularly to study science
[Marlow, 2007). Through reflection upon these experiences, teachers can
explore, envision, and plan ways that their knowledge can be integrated, in a
very practical sense, into their teaching practice [Dewey, 1933).
4


A multimodal way to engage twenty-first century learners (Robin, 2008),
to encourage reflection upon experiences, and to capture stories of import is
through digital storytelling. While researchers are beginning to explore the
viability of using digital stories as vehicles of reflection (e.g., Barrett, 2005, 2006;
Blocher, 2008; Genereux & Thompson, 2008), what has been lacking is a
systematic, integrated way of examining which aspects of their learning
experiences teachers deem most important and how deeply teachers think about
associated topics.
Having a valid and reliable method to gauge practical relevance evident in
reflection artifacts such as journals and digital stories would afford teacher
educators a means to determine the nature of reflection encouraged by each
mode of reflection, to assess the level at which teachers are able (or willing) to
express their thoughts, and to ascertain what topics or issues teachers identify
as being deeply important to them. Informed by my prior work and the
dissertation literature review, this study takes a first step in developing and
applying such a measure to the dissertation problem in a narrowly defined
context.
5


Problem Statement
The Role of Reflection in Experiential Learning
Although educational researchers differ on some aspects of experiential
learning theory [Race, 2005; Yorks & Kast, 2002), they generally agree that
experiential learning includes the learners experience and reflection on that
experience [Dewey, 1933; Kolb, 1984, Chapter 2; McKenzie & Fitzsimmons,
2010). Experiential learning, such as that commonly used in the constructivist
tradition of teacher preparation, is a process through which a learner constructs
knowledge, skill, and value directly from an experience within the environment
[Dewey, 1933; Kolb, 1984, Chapter 2; Kolb, Boyatzis, & Mainemelis, 2000;
Marlow, 2007; Schon, 1983).
When professionals engage in activities that involve open and dynamic
discussion, mutual problem solving, and collaborative learning, the participants
are drawn into a community of learners and develop a greater understanding of
their capabilities [Marlow, 2007). In cohort settings, learning is strengthened
when experiences are supported by reflection, critical analysis, and synthesis as
well as shared knowledge emerging from the interaction of the participants
[Dimova & Loughran, 2009; Lave & Wenger, 1991; Marlow & Renda, 2005; Sala
et al., 2008).
6


Traditionally, as a means to make learning explicit during teacher
education classes at the university level, teachers have been tasked to use
various media including journals, essays, portfolios, and narratives to compose
their thoughts and reflect on their experiences (Renda, 2010; Surbeck et al.,
1991). Reflection upon experiences, especially through narrative, embeds
learning in memory (Schank & Abelson, 1995), a process that in theory can
result in improved practice by providing a path from intention to action (Dewey,
1933).
Reflection in the Twenty-First Century
Because of the increasing ubiquity of technology in everyday lives,
twenty-first century strategies and tools that appeal to current and future
teachers are needed to encourage deep and relevant reflection upon experiences
(Robin, 2008; Strampel & Oliver, 2007). Digital storytelling is one such tool,
showing promise as a vehicle for both individual and shared reflection (Blocher,
2008; Boase, 2008; Li & Morehead, 2006; Renda, 2010; Tendero, 2006). Because
it is multimodal, providing both audio and visual avenues of expression, digital
storytelling is attractive to twenty-first century learners (Prensky, 2001) and has
the potential to encourage deep reflection (Barrett, 2005, 2006; Genereux &
Thompson, 2008; Nordmark, Frolunde, & Milrad, 2010).
7


Content, Level, and Relevance of Reflection
Researchers have explored how participants reflect [e.g., the reflection
process, as described in Dewey, 1933), what participants write [e.g., the
limitations imposed by modes of expression as discussed by Hatton & Smith,
1995), or some combination of the two [e.g., the relationship between content
categories and levels of reflection, as discussed in Korthagen & Vasalos, 2005).
Several frameworks have been proposed to assess levels of reflection evidenced
in traditional reflection venues such as journals [Hegarty, 2009; Kember, McKay,
Sinclair, & Wong, 2008; Surbeck et al., 1991) and the more recently introduced
venue of digital stories [Jenkins & Lonsdale, 2007; MacLaren, 2005; Renda,
2010) although work in the latter area is still experimental [Boase, 2008).
Another topic of interest is the role of relevance in reflection. With
respect to teacher development, some researchers have addressed the concept
of relevance by asking teachers what they perceive as being relevant to their
practice [Kreber, 2004), by providing examples of what is relevant [Jamissen &
Phelps, 2006), by listing appropriate topics for reflection [Korthagen & Vasalos,
2005), or by assuming that relevance of reflection, or lack thereof, is apparent
[Janssen, de Hullu, & Tigelaar, 2008; MacLaren, 2005). In these studies,
relevance is associated with the notion of being about a given topic.
Some researchers have alluded to the idea that deeper reflection occurs
when teachers view the reflection process as being relevant [Callens & Elen,
8


2009) and when teachers think about topics that they view as being more
relevant (Kember et al., 2008). However, a definition of what is meant by more
or less relevant has been lacking.
Needs Identified from the Literature Review
A review of the literature concerning depth and relevance of reflection
demonstrates the following needs relating to the emerging use of digital
storytelling for meaningful teacher reflection on teacher education experiences
conducted in group settings:
further exploration of a connection between levels of reflection and
relevance of content expressed through digital storytelling,
measureable constructs of relevance that can be applied to reflection, as
expressed in traditional and emerging modes of teacher reflection,
further exploration of the ways that images and audio elements can
enhance narrative reflection,
richer understanding of what digital stories can reveal about what
teachers view as being relevant to their practice, and
better understanding of the effects that the use of digital stories can have
on practical relevance and levels of reflection evident in teachers'
reflections on learning experiences.
9


Study Purpose and Research Questions
The purpose of this dissertation study was to investigate how deeply
teachers think on topics relevant to their practice when using digital storytelling
and journaling to reflect on a specific teacher education experience conducted in
a cohort setting. To this end, I have examined emerging themes, topical
relevance, levels of reflection, and practical relevance of reflection elements
evident in teachers journals and digital stories. In addition, I have examined
values of practical relevance for reflection artifacts holistically. To focus this
study, I asked the following research questions:
1. How does practical relevance evident in multimodal digital stories
compare to practical relevance evident in written journals of teacher
reflections for a particular science education experience?
2. What does the practical relevance of digital stories and journals reveal
about what is deeply important to teachers reflecting on a shared science
education experience?
3. For the study population, how well do age, number of years of teaching
experience, grade taught by participant, self-reported levels of
technological competency, and year in which the associated space-science
class was attended predict the practical relevance of digital stories?
10


Methodology
A mixed methods study was conducted to examine artifacts of teachers
reflections on a learning experience, generated by students taking a university
class at the masters level. For this study, qualitative and quantitative data were
collected and analyzed simultaneously.
Qualitative methods included the examination of emerging themes
evident in journals using a modified constant comparative analysis (Krathwohl,
2004; Lieblich, Tuval-Mashiach, & Zilber, 1998) and in digital stories using
narrative analysis (Lieblich et al., 1998). In addition, a novel framework was
employed as a lens through which the qualitative data on content and processes
of post hoc reflection on experience were transformed into a quantitative
representation of a more complex construct of reflection, namely practical
relevance. Themes from digital stories and journal entries, relating to
participation in a space-science class, were examined through this lens to
determine which aspects of this learning experience emerged as being most
meaningful to study participants. Finally, a multiple regression analysis
determined which predictors were the best predictors for practical relevance, as
expressed by the digital stories and journals which comprise the study set. The
selection of predictor variables was based upon a review of the literature.
11


Structure of the Dissertation
This dissertation includes five chapters with supporting appendices.
Chapter 1 provides a broad overview of the study and presents the problem
statement, study purpose, and research questions. Chapter 2 comprises the
literature review and theoretical orientation supporting this study. Chapters 3
and 4 present the methodologies and results of the study, respectively. And
finally Chapter 5 discusses findings and implications of the study.
12


CHAPTER 2
REVIEW OF THE LITERATURE AND
THEORETICAL ORIENTATION
Introduction
This chapter begins with a review of the literature addressing the nature
of reflection and its role in teacher development. From there the focus narrows,
first considering narrative as a form of reflection and then presenting digital
storytelling as a form of narrative reflection.
Next, attention moves to questions of what constitutes evidence of
reflection in reflection artifacts and how researchers have characterized
reflection in terms of content and process as well as reflection frameworks. To
end the literature review, the thin consideration given to relevance of reflection
is discussed.
Following the literature review is a section on theoretical orientation. In
this section, a multidimensional level of reflection framework adapted for this
dissertation is presented and discussed. Finally, the conceptual framework of
practical relevance is introduced.
13


Review of the Literature
Reflection
Much attention has been given to the modes and merits of teacher
reflection on practice and associated reflection on experiences (e.g., Cautrells,
2008; Hatton & Smith, 1995; Hegarty, 2009). There is a growing awareness that,
for meaningful change to occur, teachers must be provided with experiences that
encourage and depend upon self-reflection and that are part of a continuous
process directed toward professional growth (Dewey, 1933; Dimova &
Loughran, 2009; Slepkov, 2008).
The literature abounds with divergent viewpoints on the nature and
definition of reflection. For example, Dewey (1933) espouses a pragmatic
approach, Mezirow (1990) appeals to critical social theory, and Procee (2006)
argues that Kants philosophy better informs an understanding of reflection in
education. Korthagen and Vasalos (2005) discuss systematic core reflection as a
means to enhance professional growth, while Ryan (2005) writes that the key to
growth is structured reflection. Taking a more complex approach, van Manen
(1977) recognizes three main philosophies of knowledge (the empirical-analytic,
the hermeneutic-phenomenological, and the critical-dialetical), discusses forms
of practical action arising from each philosophy, and ranks thinking on the
14


practical use of educational knowledge according to the philosophy which
frames it.
Summing up points of view found in the literature, Hatton and Smith
(1995) have identified four key issues with regard to reflection: (a) whether
reflection is limited to thought about action or is bound up in action, (b) whether
reflection is immediate and short term or extended and systematic, (c) whether
reflection is or is not problem centered, and (d) whether reflection is or is not
critical in nature (taking into account historic, cultural, and political beliefs to
frame and reframe practical problems).
Reflection varies in its purpose, process, and focus (Procee, 2006; van
Manen, 1977). Often reflections target classroom experiences, which may be
narrowly focused on interactions with students or more widely directed toward
institutional or community issues. But Marlow (2007) argues that reflection on
learning experiences situated outside of the classroom have the potential to
foster profound learning in teachers in all stages of their careers, provided the
experiences are structured to require a learner to take initiative, make decisions,
and be accountable for the results.
A common thread running through myriad views of reflection on
experience is that reflection is a mechanism for drawing meaning from
experience. In the pragmatic and constructivist tradition of Dewey (1933), many
15


educational researchers argue that reflection on experience is beneficial to
teacher development in that it promotes a type of introspection necessary for
professional growth (Dawson, 2006; Hatton, & Smith, 1995; Kolb, 1984, Chapter
2; Surbeck et al., 1991; Schon, 1983). Through reflection upon these
experiences, teachers can explore alternatives in the safe realm of thought,
envision consequences, and plan how their knowledge can be applied in a very
practical sense to their teaching practice (Dewey, 1933; Kegan, 1994; Russell,
2005).
But reflection does not happen spontaneously (Russell, 2005). First, a
predisposition to engage in reflection must be present. This predisposition
requires open-mindedness, the ability to consider problems in new and different
ways; whole-heartedness, the willingness to be engaged in and by thinking; and
responsibility, the acceptance of the need to consider the consequences of ones
actions (Dewey, 1933; Dimova & Loughran, 2009). Thus intentionality (Dewey,
1933) is necessary to engage in reflection on experience in order to achieve
purposeful learning (Dimova & Loughran, 2009).
Given the appropriate mindset, reflection on the part of preservice and
in-service teachers pursuing classes can be encouraged through modeling
(Russell, 2005) or a structured approach (Dunlap, 2006; Ryan, 2005). However,
an environment of trust must exist to assuage feelings of vulnerability which
16


may arise from exposing their perceptions and beliefs to others (Hatton & Smith,
1995). A safe environment for shared reflection and learning can be fostered by
using a critical-friends approach (Russell, 2005) or solidifying relationships in a
cohort setting (Marlow, 2004; Marlow & Renda, 2005; Sala et al., 2008). In
addition, some argue that productive reflection requires that a teacher develop a
certain level of skill through repeated opportunities to engage in reflection
(Russell, 2005), call upon a repertoire of life experiences to fuel reflection
(Schon, 1983), and obtain an adequate level of cognitive maturity that allows
framing and reframing of experience (Kegan, 1994; Russell, 2005; Schon, 1983).
Narrative as Reflection
Of the many forms of reflection, narrative is recognized in the literature
as one that solidifies learning (Garrety & Schmidt, 2008; Schank & Abelson,
1995; Smith & Squire, 2007). Narrative is a means of learning from experience
by reflecting upon it, declaring what it means, and distilling it into a symbolic
form to be expressed and remembered (Davis, 2006).
The power of narrative as a mode of reflection stems from its structure
and its shared nature. In narrative, each story has a setting in place and time; a
beginning, middle, and end; a plot that advances the story; and a point. In other
words, it is coherent. The point of the narrative is the seed from which
storyteller and listeners derive meaning. Because of its structure, narrative can
17


address occurrences that happen over time and that may have long-term or
previously unseen consequences (Boase, 2008).
However, storytelling is a fluid process and the stories told are evolving
products. The narrative told at a given moment reflects an individuals
perception of experience at that time and place but may be different at another
time and place (Davis, 2006). Even so, narrative reflection lends itself well to the
constructivist paradigm of reflection: a series of interconnected thoughts
directed toward building understanding, integrating what is known with what is
posited, or solving a problem (Boase, 2008; Dewey, 1933).
In essence, storytelling is a communal experience. Throughout history,
people have shared their memories, experiences, thoughts, and feelings with
others through narrative (Blocher, 2008). When shared, narrative cements
memory and learning in the storyteller through iteration and in the audience
through vicarious experience (Schank & Abelson, 1995). Each time the story is
retold, it is relived. Thus, each retelling provides an opportunity for the
storyteller and the audience to create meaning anew and to develop a richer
understanding of human action and its possibilities (Boase, 2008).
18


Digital Storytelling as Narrative Reflection
Digital storytelling, first introduced in the 1980s by Dana Atchley, is a
multimedia form of narrative that engages learners through technology. While
early researchers in the 1990s found digital storytelling to be a powerful means
of giving members of marginalized populations a voice (Garcia & Rossiter, 2010),
shortly after the turn of the millennium, increasing numbers of individuals and
groups began using this technology for a variety of educational purposes in
PreK-20 classrooms across the United States (Blocher, 2008; Boase, 2008) and
across the globe (Jenkins & Lonsdale, 2007; Susono, Ikawa, Kagami, &
Shimomura, 2010). Aided by rapid advances and greater availability of
technology, digital storytelling has continued to gain acceptance by educators
and is regarded by many as a viable form of narrative for use in the twenty-first
century classroom (Blocher, 2008; Robin, 2008).
As in other forms of narrative, digital storytelling affords opportunities
for iterative reflection through the steps of conceiving, visualizing, creating,
editing, and sharing the story. In contrast to more traditional methods of
reflection such as journaling, digital storytelling has multiple modalities
involving auditory and visual input. The cognitive theory of multimedia
learning, which draws from Paivios dual-code theory and Baddeleys model of
working memory, assumes that humans have separate channels for processing
19


visual and auditory information (Mayer, 2001). Sadoski, Paivio, and Goetz
(1991) maintain that the verbal and visual processing systems can function
independently (e.g., words bring other words to mind and images bring other
images to mind), synchronously (e.g., words and images come to mind at the
same time), or in an integrated manner (e.g., language evokes mental images or
images evoke language). Thus, the combination of the two processing systems
allows for a great variety of cognitive activity.
In line with that theory, it can be argued that digital storytelling invokes
both the sequential, syntactic processing required for creating or understanding
narrative as well as the synchronous or parallel processing required for deriving
meaning from visual information. Mayor (2001) maintains that "in the process
of trying to build connections between words and pictures, learners are able to
create a deeper understanding than from words or pictures alone (p. 5).
Because of its complex set of elements (see Figure 2.1), digital storytelling has
the potential to facilitate deep learning and deep reflection (Barrett, 2005, 2006;
Genereux & Thompson, 2008; Nordmark et al., 2010).
20


Figure 2.1. Compounding Effect of Multimodal Elements on Reflection
through Digital Storytelling
Educational researchers have begun using digital storytelling to
encourage reflection in higher education classes. Jenkins and Lonsdale (2007)
introduced the technique to students in a landscape design class to "encourage
and embed student reflection on the activities in which they were engaged,
recognising that reflection can be enhanced as a collaborative process" (p. 440).
Blocher (2008) reports on early adoption by education technology instructors at
the university level and plans for use of digital storytelling as an opportunity for
teachers-in-training to reflect on their learning as they complete their teacher
preparation courses.
21


Despite the benefits, some disadvantages are inherent in the digital story
form. While a person who delivers a scripted narrative need not depend on
memory to tell the story, the brevity of the short narrative may have a tendency
to limit expression or lead to superficial reflection for some story creators
(Boase, 2008). In addition, level of proficiency in writing (Hatton & Smith, 1995)
or storytelling skills may help or hinder an individuals ability to communicate
the intended message in narrative form (J. C. Dunlap, personal communication,
July 30, 2010).
Digital storytellings reliance on technology also can introduce negative
factors (Cradler, Freeman, Cradler, & McNabb, 2002; Slepkov, 2008; Whittier &
Lara, 2006) which can impede reflection (Renda, 2010). Acknowledged barriers
stemming from the use of technology can include lack of technological
knowledge, resources, and efficacy; increased complexity of tasks; and additional
time needed to complete a task (Blocher, 2008; Hofer & Swan, 2010).
In contrast to those who see the time-intensive nature of digital story
creation as a hindrance, others regard the extended engagement of students in
the reflection process as a benefit (Blocher, 2008). In order to maximize the
benefits of multiple modalities and opportunities for iterative and deep
reflection inherent in digital storytelling, mitigation measures can be employed
22


to bolster both narrative and technological skills (Coutinho, 2010; Li &
Morehead, 2006; Renda & Sprouse, 2010).
Evidence of Reflection through Artifacts
Smith and Hatton (1992), citing Zeichner and Liston, maintain that
reflection is embedded in a particular epistemology which forms a theoretical
framework for any research program and its activities. Thus, each phase of the
research, including the decisions regarding what constitutes evidence of
reflection, will be framed and directed by the definition of reflection employed.
This study has been positioned in a constructivist framework, which espouses a
pragmatic, problem-centered, and decision-making approach to reflection
(Dewey, 1933; Smith & Hatton, 1992).
Reflection artifacts such as digital stories and journals can provide an
illuminating window into the process of individual change, yielding important
information for researchers seeking insight into factors that affect teacher
growth (Dunlap, 2006; Renda, 2010; Surbeck et al., 1991). Some argue that it is
necessary to move beyond self-reports to the identification of ways in which
reflective processes can be evidenced (Kreber, 2004; Smith & Hatton, 1992).
From this point of view, it is not enough to assert that reflection is encouraged
by a given procedure; instead, specific means must be identified to demonstrate
that particular kinds of reflection are taking place (Smith & Hatton, 1992). For
23


example, verbal accounts from study subjects have been used to distinguish
between declarations of reflection and indicators of reflection, denoting
particular behaviors from which reflection can be inferred (Kreber, 2004, p. 35).
Process and Content of Reflection
Acknowledging the importance of both process and content in reflection,
Korthagen and Vasalos (2005) found that teachers who used a structured
reflection process engaged in core reflection on topics that were indicative of
deep reflection. Also exploring levels of reflection in conjunction with thematic
content, Hegarty (2009) analyzed reflections written by teachers who were
required to use a structured approach to reflection to generate reflections for
their portfolios. Similarly, I reported on levels of reflection associated with
content-derived themes of professional identity that were evident in elements of
teachers reflections on a teacher preparation experience (Renda, 2010). In
these studies, frameworks guided the examination of the process and the
content of reflection as discrete constructs.
Reflection Frameworks
Generally speaking, reflection is an internal process (Dewey, 1933; Schon,
1983) that may trigger intention and future action on the part of a reflective
practitioner (Schon, 1983). The purpose of reflection directed toward achieving
a given practical outcome (e.g., building knowledge, making sense of experience,
24


eliminating human misery) may be guided by a particular philosophy of
knowledge (van Manen, 1977). Pursuant to their philosophies, several
researchers have viewed reflection through lenses or one-dimensional
frameworks that employ and rank various indicators of reflection (e.g., Hatton &
Smith, 1995; Hegarty, 2009; Kember et al., 2008; MacLaren, 2005; Surbeck et al.,
1991). The frameworks take different forms such as the sequential models of
Dewey (1933) and Schon (1983), the evaluative model of van Manen (1977), the
criteria-based scheme of Hatton and Smith (1995), the typology of Hegarty
(2009), the thematic model of Zeichner and Liston (1996), or the levels of
reflection proposed by Kember et al. (2008).
With respect to digital storytelling, researchers and educators have
devoted attention to assessing the narrative and presentation qualities of the
products. Only recently have they begun to address the need to assess reflection
or to generate reports on methods used to assess reflection. Hypothetically
speaking, Blocher (2008) discusses points to consider when assessing reflection
evident in teacher candidates digital stories used as e-portfolio products.
Similarly, Boase (2008) describes the potential for using frameworks such as
Moons Map of Learning and McDrury and Alterios Model of Reflective Learning
to evaluate digital stories for evidence of reflection.
25


Jenkins and Lonsdale (2007) used the five levels of Moon's Map of
Learning to assess levels of reflection in the learning processes evident in digital
stories created by students in a landscape design class. They focused on
reflective processes in which students clarified and developed "strategies to
solve problems with a heightened awareness of their personal development and
views on future practice contexts (Jenkins & Lonsdale, 2007, p 441).
Taking a different approach, I conducted a preliminary study comparing
the use of digital storytelling and journaling for individual teacher reflection. As
part of that study, I developed a multidimensional lens through which I
examined the reflective use of digital storytelling as a means to understand
evolving professional identity evident in a cohort of science teachers who
attended and jointly presented at a national conference (Renda, 2010). Drawing
from several authors, I developed a reflection framework for identifying three
levels of reflection in three domains: cognitive, affective, and experiential. This
framework has been adapted and expanded for use in this dissertation study.
Relevance of Reflection
Relevance is broadly defined as "the relation of something to the matter
at hand (Relevance, 2010). With respect to teacher development at the
university level, some researchers have addressed the concept of relevance by
asking teachers what they perceive as being relevant to their practice. For
26


example, Kreber (2004) reports on her previous study in which instructors were
asked to fill out a repertory grid (a matrix which has the potential to make
thinking about a given phenomenon explicit) to rank the perceived relevance of
different types of reflection directed toward improvement of teaching practice.
Others speak to relevance by providing examples of what is relevant (Jamissen &
Phelps, 2006) or by listing appropriate topics for reflection (Korthagen &
Vasalos, 2005). Still others (Janssen et al., 2008; MacLaren, 2005) tacitly have
assumed that relevance is apparent. In these studies, relevance is associated
with the notion of being about or connected to a given topic. Borrowing from
the parlance of Internet search theory, this notion of relevance can be termed
topical relevance (Soergel, 1994).
Some researchers have alluded to the idea that deeper reflection occurs
when teachers view the reflection process as being relevant. Taking a
quantitative approach to understanding reflection, Callens and Elen (2009) built
on Korthagen's preliminary work on structured reflection; their study has
concluded that in-depth reflection, as determined by content category, occurred
when teachers-in-training perceived reflection as being important (or relevant).
Others have argued that deeper reflection occurs when teachers think about
topics that they view as being more relevant (Kember et al., 2008). However, a
measure of relevance has not been made explicit.
27


Theoretical Orientation
This study relies upon the constructivist tradition of experiential learning
and reflection upon learning experiences (Kolb, 1984, Chapter 2; Kolb et al.,
2000), particularly those involving cohorts of teachers who are engaged in
learning about science. By constructivist tradition I mean that which
incorporates a theory of learning whereby "students construct knowledge in the
process of learning through interaction with phenomenon, as they develop
shared-meaning of a phenomenon via interactions within a social context. .
(Geer & Rudge, 2002, Theoretical Perspectives section, para. 1). Within this
tradition, teachers were encouraged to select, engage, and reflect upon
conference experiences in ways that would add to their practical knowledge of
teaching (van Manen, 1995).
A Multidimensional Reflection Framework
To provide a means to measure level of reflection evident in journals and
digital stories, I have adapted and expanded a reflection framework (Renda,
2010) with three levels of reflection in the cognitive, affective, and experiential
domains (see Table 2.1).
28


Table 2.1. Reflection framework with modified cognitive and experiential domains [after Renda, 2010)
Cognitive Domain Affective Domain Experiential Domain
REFLECTION LEVEL Elements of reflection drawn from Dewey (1933) Six levels of revised cognitive domain from Anderson and Krathwohl (2001) Three levels of reflection from Surbeck, Han, and Moyer (1991) Five levels of affective domain from Bloom's Taxonomy (1964) Three levels of reflection drawn from Schon's theory of reflection-in-action (1983) Three levels of reflection drawn from Kegan (1994)
Â¥ experiencing perplexity or mental difficulty (recognizing a problem) knowing reacting (with a personal concern to an event or situation) receiving (becoming aware attending to) recognizing problematic situation but remaining "stuck" (likely to have few experiences) focusing on self without consideration of others
comprehending responding (experiencing zest, pleasure)
i searching or engaging in inquiry (seeking answers) applying elaborating (comparing to a principle, theory, or a moral/ philosophical position) valuing (assigning worth) adjusting activities to the situation with inconsistent results considering others within a particular societal framework
analyzing organizing (ranking, ordering according to value)
I providing warranted evidence and moving toward/ coming to conclusion evaluating contemplating (focusing on constructive personal insights to solve a problem) internalizing (acting consistently within beliefs) reacting reflexively to reframe a problematic situation (likely to have a vast repertoire of experiences) -tolerating and/or accepting other points of view regarding self and society -thinking in terms of systems
creating (synthesizing)


In this framework, the cognitive domain is concerned with acquiring
knowledge in order to make decisions or take action. On the other hand, the
affective domain is value laden, accounting for action predicated on moral or
philosophical considerations. Rounding out the framework, the experiential
domain is concerned with the framing and reframing of experiences. More detail
on each domain of the framework is provided below.
The cognitive domain measures reflection on experience in constructivist
terms and is informed by the writing of John Dewey (1933) on reflection. Dewey
described reflective thinking as being sequential, "the kind of thinking that
consists in turning a subject over in the mind and giving it serious and
consecutive consideration" (p. 33). When viewed in the context of making
decisions, solving problems, and taking deliberative actions, reflective thinking
is a chain of thought aimed at conclusion; it involves (a) a state of doubt,
hesitation, perplexity, or mental difficulty in which thinking originates and (b) an
act of searching, hunting, or inquiring to find material (warranted evidence) that
will resolve the doubt, settle and dispose of the perplexity, and lead to
conclusion (p. 12).
Dewey's (1933) definition of reflective thinking, which mirrored his
views on the scientific inquiry process, fits well with constructivist patterns of
thought involving the acquisition, assimilation, and application of knowledge to
30


solve a problem or engage in a deliberate course of action. I adopted his
definition as the cornerstone of my developing framework, assigned Deweys
stages of reflective thinking to three levels of reflection according to the order in
which they occur during a complete chain of reflective thought, and placed these
operationalized levels of reflection in the cognitive domain of my framework.
In order to account for emotional, moral, or philosophical aspects of
reflection (van Manen 1977,1995) directed at identifying and solving a problem
or dilemma, I incorporated the three levels of reflection of Surbeck et al. (1991)
into the affective domain of my framework. These authors developed a
framework with the following levels of reflection in response to their need to
evaluate depth of thinking exhibited in teachers journal entries:
Level 1: Reacting (commenting on feelings towards the learning
experience, such as reacting with a personal concern about an event),
Level 2: Elaborating (comparing reactions with other experiences, such as
referring to a general principle, a theory, or a moral or philosophical
position), and
Level 3: Contemplating (focusing on constructive personal insights or on
problems or difficulties to arrive at a solution or course of action).
At times, teachers' reflections might relate to practical knowledge (van
Manen, 1977) or sense of professional identity but might not be presented in the
form of a problem to be solved or concern to be addressed. To aid in the
operationalization of the reflection framework in these cases, I turned to
31


Bloom's taxonomies of education objectives in the cognitive (Bloom, 1956) and
affective (Krathwohl, Bloom, & Masia, 1964) domains. I noted that, although
Blooms taxonomies were devised to "classify student behaviors which represent
the intended outcomes of the educational process (Bloom, 1956, p. 12), the
constructs presented in the cognitive and affective domains of the taxonomies
corresponded well with the nature of reflection discussed by Dewey (1933) and
Surbeck et al. (1991) respectively. A learners progression through the
taxonomies indicates growth as expressed through "the changes produced in
individuals as a result of educational experiences" (Bloom, 1956, pp. 12-13).
Therefore, to further operationalize my framework, I parsed the levels from
Blooms taxonomies into the three levels of my framework.
To emphasize behaviors (Anderson & Krathwohl, 2001) that would be
indicative of reflection having occurred at a given level, I adapted Blooms
taxonomies by replacing nouns used in the taxonomies with verb forms to
indicate action on the part of a reflecting practitioner. I also incorporated two
changes recommended by Anderson and Krathwohl: (a) Synthesizing was
replaced by creating and (b) evaluating was made subordinate to creating, as
evaluation is often viewed as being a necessary precursor to the creation process
(Wilson, 2006).
32


To capture the constructs of framing and reframing experiences through
reflection, I added the experiential domain. Operationalization of this domain
was informed by Schons (1983) theory of reflection-in-action to address
problem solving and Kegans (1994) theory of orders of consciousness to extend
consideration of reflection beyond the bounds of problem solving.
The theory of reflection-in-action speaks to a type of reflection that
consists of thinking about something while doing it (Schon, 1983, p. 54) and is
often characteristic of more experienced teachers. Schon has defined this type of
reflection as the ability of practitioners to think on their feet, to reframe and
solve a complex problem, and to change a course of action while in the midst of
that action. He has stated that reflection-in-action is born of a repertoire of
experience and previous reflection on the nature of problems presented and
solved in the past.
In his writing, Schon (1983) provided several scenarios illustrating
reflection-in-action. Relying on these scenarios, 1 operationalized the levels of
reflection in the experiential domain as follows:
Level 1: the practitioner approaches a problem (being stuck) as a unique
case and acknowledges relevant prior experiences but attends to the
peculiarities of the situation at hand,
Level 2: the practitioner reframes the situation in order to construct an
understanding of the problem and responds to complexity in what seems
like a simple, spontaneous way, and
33


Level 3: the practitioner conducts an experiment on a reframed problem
to discover what consequences and implications can be made to follow
from it, and makes new discoveries, which in turn may call for new
reflection-in-action until a solution to the problem is found.
Providing another perspective with which to view the framing and
reframing of experiences, 1 have included concepts drawn from three of Kegans
(1994) five orders of consciousness to operationalize the three levels of the
experiential domain of my framework. Kegans orders of consciousness are tied
to his theory of cognitive maturity and predicated upon a person's ability to
move from a subjective to an objective stance when framing or reframing a point
of view as a way of organizing experiences. The word order is taken to mean
dimension rather than sequence (Kegan, 1994, p. 34).
Level 1, in my framework, draws from Kegan's (1994) second order of
consciousness, which is characterized by a centralized awareness of self and
recognition of others as independent actors. However, an individual operating
at this order of consciousness considers experiences or circumstances as they
relate to self without consideration of others. This order of consciousness is
typical of pre-adolescents and young adolescents (Kegan, 1994).
Thus, a person reflecting at Level 1 frames experiences in terms of self;
the existence of others is incidental or important only in the sense that others
can impact the quality of an experience. The person thinks in terms of concrete
34


examples, thinks linearly in terms of cause and effect, and asks the question,
What happens to me?
Level 2 incorporates ideas from Kegan's [1994) third order of
consciousness, which describes a sense of self that is aware of others and
recognizes a society guided by a consistent set of values. An individual operating
at this order of consciousness is able to consider another's point of view, think in
hypothetical terms, and plan with the future in mind. Transition to this level
usually occurs during late adolescence [Kegan, 1994).
Reflecting at Level 2 in my framework, a person considers experiences in
a way that acknowledges self and others within the context of society and
governing norms. Someone reflecting at this level is able to shift from one frame
of reference to another and thereby "step into anothers shoes" to understand,
but not necessarily embrace, anothers point of view. This person thinks
abstractly, considers alternative outcomes in response to a hypothesis, and asks
the question, What happens to my relationships?
Level 3 draws from Kegans [1994) fourth order of consciousness in
which self-determination becomes possible, as does tolerance and acceptance of
formerly rejected aspects of self and society [Dombeck, 2007; Tinberg &
Weisberger, 1998). An individual operating at this order of consciousness is
35


self-authoring, self-directed, and able to hold multiple points of view
simultaneously. Not everyone will transition to this level (Kegan, 1994).
Reflecting at Level 3, a person is able to think in terms of individual and
system-wide relationships. Such an individual thinks of whole systems, is
inclusive of others as self-governing persons, and asks the question, What must I
do to act within my personally constructed standards of behavior?
Reflective Aspects of Visual Subelements
in Digital Stories
The cognitive model of multimedia learning maintains that humans have
separate systems for processing visual and auditory information (Mayer, 2001)
with different organizational and processing characteristics (Sadoski et al.,
1991). The verbal system deals with the sequential, syntactic organization of
language; while the nonverbal or imagery system is involved in the analysis of
scenes and the generation of visual, auditory, haptic, and affective mental images
(Sadoski etal., 1991).
Sounds, such as spoken words, are converted into a verbally based model
while visual input, such as pictures, motion, or written text, are converted into a
visually based model (Mayer, 2001; Mayer & Moreno, 1998). Nonverbal
information, especially visual information, is cognitively organized in the form of
"holistic nested sets with information available for processing in a synchronous
36


or parallel manner (Sadoski et al., 1991). For example, each star in the sky
could be perceived separately or perceived as being part of the night sky.
Spoken narrative is the heart of digital storytelling. In a storied setting,
images by themselves do not narrate (Banks, 2001), but they do provide
opportunity to reflect and grow from our experiences (Lemon, 2007). To decode
the meaning of an image, entangled in the human action of its creator, requires a
wider frame of analysis than that of the visual element itself. It requires a
reading of the external narrative that goes beyond the image (Banks, 2001, p.
12).
When purposefully selected by a digital storyteller, images may suggest,
depict, or enhance the storys message (Machin, 2007). Recognizing that
thinking more abstractly requires a higher level of cognition (Kegan, 1994),
visual level-of-reflection criteria have been developed which map levels of
reflection to complexity of meaning associated with images (see Figure 2.2).
37


p
R
O
M
P
T
To what extent did each
image or series of images
reinforce the thought or
idea expressed in the text?
Level Modes of Connection
of Visual Criteria between
Reflection Example Image and Example Text
Hi Image was used as filler or selected impulsively, with little thought, or used to add to overall feel or atmosphere of narrative, or used to appeal indirectly to a theme without advancing the meaning of the narrative ^ Adventures in Space ^ 1 1 Vague Connection
Image was used to A photograph of the Earth taken
m depict or Illustrate, by astronaut William Anders in
LU add specificity to, or 1968 during the Apollo 8 mission
provide concrete example of t *
concept(s) expressed in the narrative Concrete Connection
Image was used to advance the meaning of
the narrative through "Perhaps it will be you who wakes
fill connotation up in the morning to see a familiar
IL2JI friend rising in the distance"
xy (i.e., nuance, irony, metaphor), or -Leo, 2010
appeal to prior knowledge
(i.e., something else familiar)
Extended Connection
Permission for educational use of image residing in the Public Domain granted by NASA
http://www.nasa.gov/multimedia/imagegallery/image_feature_1249.html
Figure 2.2. Level-of-Reflection Criteria and Modes of Connection for Visual
Subelements
38


These criteria recognize the mode of visual processing inherent to people
as well as the role that images play to thicken and enrich the meaning of
narrative (Mayer, 2001; Sadoski et al, 1991). The criteria address three levels of
reflection that might be inferred by a digital story viewer as a proxy for the level
of reflection undertaken by the story creator. These levels align image selection
and use with modes of vague, concrete, or extended connections to the narrative.
Conceptual Framework of Practical Relevance
One definition of practical is being "guided by experience and observation
rather than theory (Practical, 2010). Regarding educational practice, van
Manen (1977) maintains that philosophies of knowledge each have associated
with them distinct ways of knowing and distinct modes of being practical. He
speaks of practical knowledge, which embodies the type of knowledge that
teachers use to inform their day-to-day teaching practice. He presents a
hypothetical example of a teachers view of practical knowledge in this way:
My practical knowledge is my felt sense of the classroom, my
feeling who I am as a teacher, my felt understanding of my
students, my felt grasp of things that I teach, the mood that
belongs to my world at school, the hallways, the staffroom, and of
course this classroom (van Manen, 1995, p. 46).
Dewey (1916) argues that reflection involved in practical situations is
concerned with "things to do or be done, judgments of a situation demanding
action" (p. 335). This dissertation study focuses on teachers reflections upon
39


their learning experiences, undertaken for the purpose of informing their
practical knowledge (i.e., classroom practice and professional identity). For this
study, I have derived a conceptual framework (see Figure 2.3) rooted in
constructivism but stretching beyond the boundaries of strict empiricism. This
framework acts as a lens through which the content and processes of reflection
together can be viewed as an interconnected construct of reflection, namely
practical relevance.
Practical relevance of reflection on experience encompasses the idea
that as teachers think more deeply on experiences on matters that are topically
relevant to their practice, their reflections can be deemed more relevant to their
practice, regard less of philosophical orientation.
Figure 2.3. Conceptual Framework Depicting Practical Relevance as a Product
of Topical Relevance and Level of Reflection
40


Practical Relevance of Individual Elements in a Reflection Artifact
If a journal or digital story is viewed as being made up of a set of
reflection elements (e.g., paragraphs, stanzas, images, or combinations of these),
the conceptual framework is further refined to apply values to individual
reflection elements, as shown in Figure 2.4. Here, for a given reflection element,
Practical Relevance = Topical Relevance X Level of Reflection
(PRelement) (TRelement) X (LoRelement)
where
TReiement has a value of 0 or 1 (absent or present, respectively) and
LoReiement has a value ranging from 1 to 3, which is obtained by using
the adapted Level of Reflection Framework, shown in Table 2.1.
r~ Practical
Relevance
(PRelement)
Having a range of
0-3________________
Addressing experiences
in a way that is
deeply connected
t to practice .
Topical
Relevance
(TRelement)
Having a value of
0 or 1
Addressing topics
related to
practice
Level of
Reflection
(LoRe|ement)
Having a range of
1-3
Thinking deeply
as determined by
position in
Reflection Framework
Figure 2.4. Conceptual Framework Depicting Practical Relevance for Each
Reflection Element as a Product of Its Respective Topical
Relevance and Level of Reflection
More specifically, practical relevance for a given reflection element (e.g.,
paragraphs, stanzas, images, video clips) of a reflection artifact (e.g., journal or
41


digital story) is determined by multiplying the level of reflection apparent in
each element by the elements topical relevance. Topical relevance is
determined thematically by noting the presence or absence of an apparent
connection to a selected matter at hand. In this study the matter at hand is that
which impacts teaching practice (e.g., improvement of teacher practice or
teacher professional identity) or adds to practical knowledge (van Manen, 1995).
That is also to say, if reflection is not topically relevant to practice (i.e., Topical
Relevance = 0), neither is it practically relevant, regardless of how deeply an
individual reflects upon it.
Practical Relevance Inferred from a Reflection Artifact
Practical relevance encompasses the overall message derived from the
artifact, the meaning of each element of the artifact, and the functional
relationship between elements of the artifact and the whole. The process to
derive this value is not an exercise in rating the skill of the storyteller to tell a
good story, or the skill of an individual to create a highly polished multimedia
product. Rather it is an attempt to characterize the practical relevance of a
practitioner's underlying thought process and content that may be inferred from
reflection expressed in a journal or digital story.
As depicted in Figure 2.5, practical relevance of a reflection artifact goes
beyond a summation or averaging of the values for a set of elements. It also
42


accounts for the idea that the structure and purpose of an artifact mold its
overarching idea or core message. By contextualizing the composite of its
elements within a core message, practical relevance speaks to the intrinsic
applicability and quality of the reflection on experience whether it is
profound or superficial (Dewey, 1933, p. 44). Thus, practical relevance
considers the backstory of the artifact and acknowledges but is not limited to a
composite of its elements.
Practical Topical Level of
Relevance Relevance Reflection
/ N
PR
rr* message
# * * *
B
% -* a % *
* * # $ m
% m # %
V________/
LoR message
* m
* Q
*
* *
*
PR = Practical Relevance
TR = Topical Relevance
LoR = Level of Reflection
LEGEND
Individual Element 7
Average of
all Elements
[Core] Message
Figure 2.5. Conceptual Framework Depicting Practical Relevance for a Complex
Reflection Artifact with Elements Embedded in a Core Message
For example, in this dissertation study the content, purpose, and
structure of a participants journal, created in fulfillment of class requirements,
were guided to some extent by the instructions provided to the participant
43


(Dunlap, 2006; Ryan, 2005]. In the case of a digital story, the frame of the story
was informed by class requirements (Dunlap, 2006; Ryan, 2005), reflections
recorded in the companion journal, the goal and point of view (Davis, 2006) of
the story creator in telling the story, as well as purposeful observations
undertaken by a participant (Dewey, 1933) during the conference.
Before it can be operationalized, practical relevance, along with its factors
of topical relevance and level of reflection, must be considered within the
construct of core messages. In this study, topical relevance of the artifact's core
message is determined by answering the following question:
In light of assignment requirements and goals articulated by
the author, did the artifact, as a whole, address conference
experiences in ways that were related to teaching practice?
Similarly, level of reflection for the core message can be determined by
summarizing the message expressed by the entire artifact and then locating the
message within the reflection framework (refer to Figure 2.5). Practical
relevance of the core message then becomes the product of its two factors.
Finally, practical relevance of an artifact can be expressed as the average
computed from the practical relevance of the artifacts core message and the
average practical relevance for individual elements of the artifact.
44


Thus, PR (PRmessage + 5]PRelement/n)/2
where
PR is the practical relevance of the artifact and has a scale value
ranging from 0 to 3,
PRmessage is the practical relevance of the core message of the
artifact and has a scale value ranging from 0 to 3, and
SPReiement/n is the average practical relevance of n elements in the
artifact and has a scale value ranging from 0 to 3.
This expression of practical relevance accounts for the big picture as well as the
fine-grained aspects of practical relevance by giving them equal weight. Specific
methods used to infer the practical relevance associated with journals and
digital stories as well as other methods used in this study are presented in the
next chapter.
45


CHAPTER 3
METHODS AND PROCEDURES
Introduction and Overview
For this study, teachers' reflections expressed in journals and digital
stories, additional class artifacts, and survey responses were examined using
both qualitative and quantitative methods (Lieblich et al., 1998). These methods
were employed to infer which aspects of a learning experience were relevant to
study participants and how deeply they reflected on those experiences. The
study employed a modified, single-phase mixed method triangulation design-
transformation model (Creswell & Plano Clark, 2007, p. 85), for which
qualitative and quantitative data were collected and analyzed simultaneously.
This chapter first provides an overview of the study population and
introduces five representative participants in more detail. Next the chapter
describes the setting, taking into account my role in the study and presenting the
major course components of the experiential learning class attended by
participants. Then data collection sources and methods are addressed. This
discussion is followed by a subsection on the research design and mixed
methods used in this study. Then, after addressing study limitations, the chapter
concludes with a discussion on trustworthiness, validity, and reliability.
46


Participants
Overview of Study Participants
Participants comprised a convenience sample (Krathwohl, 2004) of the
17 class members from the spring 2009 session and the 22 class members from
the spring 2010 session of a university-sponsored teacher development class
offered for credit at the master's level. While all class members from both
classes consented to be participants in this study, not all participants fully
completed assignments, submitted complete sets of artifacts, or participated in
online surveys. Table 3.1 summarizes participant characteristics according to
the year in which the class was attended. Table A.l presents characteristics of
each participant.
Each class included preservice and in-service teachers as well as males
and females. In both classes, in-service teachers outnumbered preservice
teachers, and females greatly outnumbered males. Participants ranged in age
from 23 to 59 years, had between 0 and 9 years of teaching experience, and
taught or planned to teach academic levels including kindergarten, elementary
school, middle school, and high school.
47


Table 3.1. Summary of characteristics of study participants
Year of Class
Characteristic 2009 2010 Combined
Number in class 17 22 39
Number of females 12 20 32
Number of males 5 2 7
Average age 31 28 31
Age range 24-50 23-59 23-59
Number of preservice teachers 5 8 13
Number of in-service teachers 12 14 26
Average years teaching experience 2.5 1.4 1.9
Range of teaching experience 0-8 0-9 0-9
Substitute teaching 0 1 1
Teaching/planning to teach kindergarten 2 0 2
Teaching/planning to teach elementary school 3 10 13
Teaching/planning to teach middle school 9 4 13
Teaching/planning to teach high school 3 7 10
Highest degree = bachelor's 12 21 33
Highest degree = master's 5 1 6
Average technological competency score 17.2 17.5 17.4
Range of technological competency Scores 12.7-20.0 13.9-20.0 12.7-20.0
Technological competency for participants ranged from 12.7 to 20.0. For
purposes of this study, technological competency has been defined as the
confident and critical use of technologies for employment, learning, self-
development, and participation in society (Ala-Mutka, Punie, & Redecker, 2008).
Having a maximum value of 20.0, technological competency was computed from
responses to several questions in an online survey administered on the last
evening of class (see Appendix B). Class members reported various motivations
for taking the class, which included learning about space science, learning to use
technology, and earning three credits toward a degree program.
48


Profiles of Five Selected Participants
To provide a sense of the types of students who took the space-science
class, profiles are provided for five representative study participants (see Table
3.2). Pseudonyms are used for all participants. Information on their
backgrounds and motivations for taking the class are included in these profiles.
Table 3.2. Characteristics of five representative study participants
Participant Pseudonym Class Year Age Highest Degree Teaching Status Years of Teaching Experience Grade Level and Subjects Taught
Gina 2009 24 Bachelor's In-service 2 Elementary All subjects (4th)
Middle School
special education
Leo 2009 47 Bachelor's In-service 8 Seeking highly
qualified status for science (6th)
Quinn 2009 38 Master's In-service 3 Kindergarten
Middle School
Ted 2010 24 Bachelor's In-service 1 special education
Substitute
Ursula 2010 59 Bachelor's Preservice 0 Elementary All subjects
49


Gina's Profile
Gina was a 24-year-old in-service teacher with two years experience
teaching at the elementary school level. At the time she took the class, she held
her bachelors degree and was pursuing a masters degree. At the end of the
course, her technological competency score was 16.6 of a possible 20.0 points.
With regard to motivation for taking the class, Gina stated, "Hands on learning is
much better than sitting in a classroom and listening to someone tell me how
wonderful science can be" (Preconference expectations posted on wiki, 2009].
Leo's Profile
Leo was a 47-year-old in-service teacher with eight years experience
teaching special education students at the middle school level. At the time he
attended the class, he held a bachelors degree and was pursuing a masters
degree in education. His goal was to become highly qualified to teach science.
Leo did not own a computer and reported that prior to taking the class he
had not used any computer technology other than e-mail. However, in addition
to learning the applications needed to complete the class assignment, Leo sought
further training and became familiar with GarageBand, a music-creation
application. His self-reported technological competency score on the last day of
class was 16.11.
50


Prior to attending the Space Exploration Education Conference (SEEC)
with his class, Leo stated that he was excited at the prospect of seeing the full-
scale versions of the shuttle and International Space Station. He also said that he
was excited that he would have the opportunity "to 'muddle' [his] way through
the technology and put together a digital story" (Preconference expectations
posted on wiki, 2009).
Quinn's Profile
Quinn was a 38-year-old in-service teacher with three years experience
teaching kindergarteners. At the time she attended the class, she held a masters
degree and was pursuing a second masters degree in education. Her
technological competency score on the last day of class was 17.06 of a possible
20 points. She reported that her digital story did not take much time to create
because she received support from her husband who was "a web guy. Quinn
stated that she was intrigued and excited about learning how to bring science to
her young students. She was motivated to "bring back fun and valuable science
lessons that kids cant wait to do (Preconference expectations: Class wiki,
2009).
51


Ted's Profile
Ted was a 24-year-old in-service teacher who had one year of teaching
experience as a substitute teacher for special education students. He reported
that he was available to teach all grade levels but to date had accumulated most
of his experience in middle school classrooms. Ted had a firsthand
understanding of what it meant to be a student with a learning disability since he
struggled with a coding disability that made it difficult for him to capture notes
in class without the aid of technology. During the conference, Ted assumed the
task of photo documenting the conference-sponsored Taste of Space event and
final banquet, taking more than 600 photos in the process.
At the time he attended the class, Ted held a bachelor's degree and was
pursuing a master's degree. His self-reported technological competency score at
the end of the class was 18.0. Upon return from the conference, he experienced
difficulty in retrieving audio files from his iPod and required the assistance of a
technology facilitator to retrieve the files.
On the first day of class, Ted wrote that he attended Space Camp in
Florida as a child and had a great time there. His motivation for attending the
class was that he "always wanted to be an astronaut, and Huston [sic] is as close
as I'll likely get" (Preconference expectations posted on wiki, 2010].
52


Ursulas Profile
Ursula was 59-year-old preservice teacher who hoped to teach
elementary school. At the time she took the class, she was pursuing teacher
licensure and a master's degree, concurrently. Ursula finished the class with a
technological competency score of 18.44. She reported that she worked "off and
on for over a month on her digital story, learning new things along the way.
Although she was beyond the point of realizing her girlhood dream of
going into space, Ursula felt she could motivate, encourage, and support her
students "who might be the first to go to Mars. In addition, she wanted to have
the firsthand experience of building "rockets, robots, and other flying machines"
(Preconference expectations posted on wiki, 2010).
2009 and 2010 Settings
Overview of Settings
An experiential science education class was offered as an elective course
to in-service and preservice teachers seeking a masters degree in education
from a university in the western United States during the spring semesters of
2009 and 2010. The course had two purposes, the first being to introduce
teachers to space exploration science content and pedagogy. To do so, two
cohorts of teachers participated in events at Space Center Houston, a museum
53


and education center, and the National Aeronautical and Space Administration
(NASA] Johnson Space Center, an astronaut training and mission control center.
The second purpose of the course was to acquaint participants with the use of
technologies they would need in order to develop digital stories as a form of
reflection on their experiences.
The major components of each class included attendance at an eight-hour
technology workshop, attendance at a three-day conference on space
exploration science, creation of a digital story, and presentation of the digital
story to the entire class. Teachers were informed prior to their respective class
sessions that they would be asked to create digital stories but that there was no
prerequisite they be familiar with specific technologies. The class components
are discussed in more detail below.
Researcher's Role in the Class and
Recruitment of Participants
Each session of the class was led by Dr. Michael Marlow, the course
instructor. I served as a technology facilitator for the class, along with another
individual. Together we planned and facilitated the associated technology
workshops, created and managed class Web 2.0 accounts, accompanied the
teacher cohorts to the 2009 and 2010 conferences, and provided technical
support for the duration of each session of the class. As a technology facilitator, I
had no role in assigning grades for student performance.
54


For each session of the course, class members were provided consent
forms at the beginning of their first class meeting. At that time, they were
informed that participation in the study was voluntary and would not influence
their grades. Class members were also informed that a disinterested third party
would collect signed consent forms and place them in a sealed envelope and that
I would not open the envelope before final grades were posted by the course
instructor, Dr. Marlow. Thus neither the instructor nor the facilitators knew
who consented to be part of the study while the classes were in session.
Preconference Technology Workshops
and Ongoing Support
A few days before the 2009 and 2010 conferences, another technology
facilitator and I conducted mandatory eight-hour workshops addressing
technologies that teachers would need to complete their class requirements
successfully [see Table 3.3). Areas of instruction included using a class wiki,
recording audio files on iPods, creating digital images from personal cameras,
appropriately downloading images from the Internet, creating digital stories
using video editing software on Microsoft Windows or Apple Macintosh
platforms, and posting images and movies to shared asset spaces. Participants
had the option of using personal laptops or using university-owned computers in
the classroom.
55


Table 3.3 Technologies presented during the workshops
Units Technology Uses
Wetpaint wiki First-day reflection space Content management system
Web 2.0 Picasa (2009) and Ning(2010) Photo sharing space
Screencast file sharing space Digital story sharing space
Digital camera Image capture at conference
Digital Images Internet Source for images related to space exploration
Audio iPods Audio recording at conference
Digital Story Creation Digital Story Basics Story planning
iMovie MovieMaker Digital story creation tools
The 2009 technology workshop proceeded relatively smoothly. During
the morning session, participants created a personal wiki page on the class wiki
and posted their preconference expectations to it, downloaded images related to
space science and space exploration from the Internet and posted them to
shared space on the class Picasa site, and practiced recording audio files with
iPods and downloading the files to their computers. Subsequently and as an
added benefit, the resulting collection of space science images was maintained as
a source of visual assets for digital story creation.
However, because morning technology sessions ran a little longer than
planned, the afternoon session on digital story basics was somewhat rushed.
Handouts were distributed addressing aspects of good storytelling, but the
56


discussion was cut short because students needed to become acquainted with
story creation software, generate a practice story, and post the story to the class
shared Internet space on Screencast.
In addition, a few participants who normally used computers with
Microsoft-based operating systems and who chose to use the university-owned
laptops, with Apple Macintosh [Mac] operating systems, became frustrated
when trying to use the unfamiliar systems. They opted to leave the workshop
early, offering assurances that they understood the process and felt they would
have no difficulty completing stories and posting them to Screencast.
In contrast to the 2009 workshop, the 2010 workshop got off to a rocky
start as catastrophe struck the personal computers of two members of the class,
Zoe and Kazia. A new and particularly nasty virus, attaching itself to animated
Internet advertisements and transmitting itself to Microsoft operating systems,
attacked and shut down the two women's computers as they were attempting to
access the class Wetpaint wiki site. Computers with Microsoft operating systems
and outdated anti-virus software were vulnerable to attack by the virus. The
workshop was halted temporarily to allow the technology facilitators to
troubleshoot and mitigate the problem.
In response to the situation, several participants, including Zoe, elected to
switch from their personal laptops to university-owned laptops with Mac
57


operating systems. Kazia steadfastly attempted to troubleshoot her situation by
accessing Microsoft technology support via her cell phone throughout the
morning. After lunch, she too switched to a university-owned laptop. The
technology facilitators worked with Dr. Marlow to arrange for repair of the
infected computers at no cost to the impacted participants.
The rest of the day progressed without incident. All 22 participants
created personal wiki pages and posted space science images to the shared
space on the class Ning. The technology facilitators made the decision to switch
from Picasa (used in 2009) to a Ning because the Ning site was easier to use.
During the afternoon, emphasis was placed on aspects of good
storytelling through large-group discussion, supplemented with handouts and
web links posted to the wiki. In addition, multimedia best practices were
covered in tandem with a hands-on introduction to digital story creation
software. The majority of participants completed their practice digital stories
and posted their stories to the class Screencast site before leaving class at the
end of the day.
For both the 2009 and 2010 classes, technical support was made
available to teachers following the workshops and throughout the remainder of
their classes as they developed their digital story assignments. Classes did not
meet formally between the conference dates and the final presentation nights.
58


While participants worked on their own to create their digital stories, technical
support from the technology facilitators was offered to them through wiki
postings, e-mail, telephone, and one-on-one meetings.
Following the 2010 conference, an optional four-hour technology
workshop was offered for participants who desired additional help with
technology use. Six of the 22 class members availed themselves of this option.
Reflection Instructions
To guide their journal reflections and digital stories, participants in both
classes were provided with the following information:
prompts for preconference expectations to be posted on the class
Wetpaint wiki,
prompts for journal reflections to be written during the conference,
requirements for practice digital stories, and
requirements for final digital stories
The prompts were designed to encourage participants to consider and reflect
upon ways that (a) their conference experiences could inform teaching practice
and (b) their conference experiences could inform their digital stories. Actual
instructions and prompts are provided in Appendix C.
First-day wiki reflection prompts and conference journal prompts were
the same for both classes. However, written instructions accompanying the
2010 journal prompts added two statements: [a) All reflection topics were to be
59


addressed at least once, and (b) reflection topics need not be limited to the topic
list. The first-day wiki prompts addressed participants' expectations, goals, and
motivations for attending the class. The journal prompts addressed personal
and professional impacts of the conference, newly attained content knowledge,
and ideas for digital stories. In addition, teachers were asked to consider what
they were bringing back for their students.
Instructions for practice digital stories generated during the workshop
were generally consistent across classes, but the 2010 instructions were more
detailed. One significant difference between instruction sets was that 2009
students were encouraged to post their completed practice stories on the class
Screencast site within a few days of the workshop, while the 2010 class
members were required to post their practice stories before leaving the
workshop. Each participant chose of one of the three wiki reflection prompts as
a practice story theme.
Final digital story instructions were consistent across classes with two
main exceptions. The 2010 instructions added the stipulations that digital
stories (a) be suitable for all audiences [i.e., "Rated G") and (b) incorporate the
elements of good storytelling. The first stipulation was prompted by incidents of
adult humor occurring in the 2009 final digital stories. The second stipulation
60


was consistent with the greater emphasis being placed on development of basic
storytelling skills during the 2010 workshop in contrast to the 2009 workshop.
Participants were asked to generate digital stories that drew something
meaningful from their Houston experiences and that included at least one space
science concept. In addition, 2009 participants were asked to create stories that
could be incorporated into a lesson for their students. This requirement was
modified slightly for the 2010 class: Participants were to create digital stories
that either could be incorporated into a lesson or simply shared with students.
The Houston Experience
Each session of the course required group participation in the annual
Space Exploration Education Conference held in Houston, Texas. Class
participants spent their time at the conference engaging in the following
activities:
attending hands-on conference sessions on space exploration science,
obtaining space exploration science curriculum materials,
participating in tours of NASA operations at the Johnson Space Center,
gathering informally to share experiences,
making journal entries, and
collecting digital assets [images, video clips, audio clips] for possible
inclusion in their digital stories.
61


Since 1994, teachers from across the United States have converged at
Space Center Houston for the annual Space Exploration Education Conference.
Many teachers have come back, year after year, because this three-day
conference offers so many sessions, activities, and events from morning until
night each day that no one can do and see everything available in just one trip.
In the words of one participant, Ursula,
Just reading through the list of sessions and tours before the
conference made me realize my preferences would take nearly a
month to complete. I had to make choices, but it was so difficult.
(Journal entry, 2010)
Each day during the conference, attendees began boarding busses from
their hotels to Space Center Houston at 7:30 a.m. Once there, they were treated
to a continental breakfast before they dashed to the morning's two-hour hands-
on sessions and tours of NASA facilities. Teachers choose sessions according to
the grade level and topics offered. For example, an elementary school teacher
might choose the Bubble Trouble session and come away with curriculum and
pedagogical strategies to introduce students to concepts such as surface tension
and Bernoullis Principle. On the other hand, a high school teacher might choose
to attend the session entitled Black Holes, Supernovae, & Dark Matter to learn
about the physics behind exotic phenomena in space.
After a break to enjoy a buffet lunch and regroup, teachers dispersed to
attend four hours of back-to-back sessions in the afternoon. Participants had
62


mixed reactions to the structured, hands-on sessions. For example, Patrick
wrote,
I would also say that my goals met would be a fifty/fifty average. I
was very impressed by some of the presentations and subjects and
others just left me wanting more. (Journal entry, 2009}
Many teachers participated in tours offered at the Johnson Space Center.
A universal attractor was the trip to the Neutral Buoyancy Laboratory (NBL}.
Leo, a 2009 class participant, had this to say about the NBL:
I was very excited to see the Neutral Buoyancy Lab. 1 was very
impressed with the size of the pool and could see the outline of the
space station under the water. We were in an enclosed hallway
with glass windows that looked down on the pool. I was glad they
had TVs placed in the hall where we could see the astronauts
training under water. But that was the tour. No more. I wanted to
see more. I wanted to get closer to the water. I wanted to feel the
weight of the suit and get a feel for all the different layers of
clothing they had on. I wanted more. (Journal entry, 2009}
Other attendees toured the NASA Food Laboratory, Historic Mission Control
(intact from the 1960s), or the Vehicle Mock-up Training Facility where
astronauts train in a full-scale mock-up of the International Space Station.
Each evening brought great food, excitement, and opportunities to
network with other teachers, meet astronauts including those from NASA's
"Teachers in Space program, and explore the myriad exhibits at the museum.
Describing her experience, Gina wrote:
63


I was able to meet new people and have a blast. I went there
knowing only a few people and left there with friends. Going to the
food tasting and dinner was beyond my expectations, (Journal
entry, 2009)
In 2009, study participants attended the first Taste of Space gourmet food
tasting and thrilled to the premier screening of the documentary film, Weightless
Flights of Discovery, which told the story of teachers experiences of simulated
weightlessness as they flew in the Zero-Gravity plane. The participants in 2010
met Dr. Sally Ride, the first American woman in space, and heard firsthand about
her dedication to the education of young boys and girls in the fields of math,
science, and technology. Regarding his impressions, Ted wrote:
She was inspiring. She provided help for teaching and new
teaching and working on including more girls in STEM. I even got
to ask her a question. I asked her if she had any advice for 1st year
teachers. (Journal entry, 2010)
Once back at the hotel, study participants gathered in an informal setting to
unwind, discuss the day's events, and share ideas for the classroom.
Digital Story Creation
For their final class projects, the teachers were asked to create digital
stories that related something meaningful about their conference experiences,
involved some aspect of space science, and could be integrated into a lesson or
shared with students. Participants were allowed to work independently or in
64


small groups to create their stories. Three class members opted to work as a
group during 2010. All others in both classes worked individually.
Participants in both classes were given the choice of creating storyboards
or scripts and were required to generate image logs. The purpose of the
storyboards or scripts was to guide participants in their digital storytelling
efforts. The purpose of the image logs was to document origin and copyright
information for each image used. To create their stories, participants were
permitted to use image and audio files that were downloaded from the Internet
(provided copyright restrictions were honored), captured at the conference, or
obtained from assets posted to the class online shared space.
Subsequent to the 2009 conference trip to Houston, class participants
were asked, but not required, to post five personally generated images on Picasa
to share with the group. Teachers were not required to post images because not
everyone owned a digital camera. It was hoped that this strategy would foster a
sense of shared experience and encourage collaboration among participants as
well as generate a rich pool of images available to all participants for use in their
digital stories. This effort resulted in a shared pool of 88 images.
Because the 2009 effort met with limited success, in 2010 all study
participants were required to generate and post between five and ten images
from the conference on the class Ning within five days of returning from the
65


conference. Class members had agreed that those who did not have cameras at
the conference would arrange with other classmates to borrow cameras at some
point during the conference. This strategy resulted in the creation of a shared
pool of 610 pictures available for use in participants digital stories.
Final Night Presentations
Both classes reconvened roughly seven weeks after their respective
conferences in order for participants to enjoy pizza and comradery, present their
digital stories, and share their experiences with their classmates. To set up their
stories, participants in the 2009 class were asked to tell the class "a little bit
about their digital stories. Participants in the 2010 class were asked to relay
the purpose of the story, the rational for selecting images, and the challenges
faced when creating the story.
Data Collection
During 2009 and 2010, data were collected under an approved human-
subjects protocol written for two previous studies and subsequently amended
for dissertation purposes. Data took the form of pre, post, and longitudinal
online surveys; class-generated artifacts; and researcher observation notes from
the final night of each course offering. Because this dissertation study builds on
data collected from previous studies, not all survey items in the pre and
66


postclass surveys were pertinent to the dissertation. Data collection efforts are
summarized in Table 3.4 and described in detail below.
Table 3.4. Data sources and uses
Data Source Class Purpose Research Purpose
Pre and postclass surveys Background information on Demographic data
Longitudinal surveys class members Triangulation data
Online reflections Journal entries Class deliverables Triangulation data Focus of research analysis
Storyboards, scripts, and image logs Intermediate work products Triangulation data
Picasa and Ning postings Shared digital assets Triangulation data
Digital stories Class deliverables Focus of research analysis
Final night presentations Group reflection Triangulation data
Pre and Postclass Surveys
Pre and postclass online Zoomerang surveys were administered to 2009
and 2010 class participants prior to the technology workshops and after the final
night of each class session. The purpose of the surveys was to collect data on
participants' demographics and their self-reported experience, competencies,
and comfort levels in using various technologies. Both Likert Scale items and
open-ended questions were included in the surveys. (See Appendix D for survey
67


items relating to the dissertation study). Invitations to take the online
longitudinal survey were issued on September 4, 2010, to all participants, using
the most current e-mail addresses on record; however, only 17 of 39
participants responded. Missing data were obtained for an additional five
participants through phone calls and for six more participants through a search
of publicly available Internet sites, including school websites. Table A.l provides
information on methods employed to obtain data for each participant.
Online Wiki Reflections and Journal Entries
During the preconference workshops, participants were asked to write
briefly about their course goals and expectations on their personal pages of the
class Wetpaint wikis. In addition, participants of both classes were instructed to
keep journals during the three days of their respective conference trips.
Participants from the 2009 class were given the option to submit their journals
to the class instructor as typed hard copy or to post their entries on the class
wiki, while those from the 2010 class were asked to submit their journals in both
hard copy and electronic format.
Storyboards, Scripts, and Image Logs
Participants in both classes were given the choice of creating storyboards
or scripts and were required to generate image logs. Participants were asked to
submit these artifacts in hard copy format.
68


Picasa and Ning Postings
Participants in the 2009 class were asked to post at least one image
downloaded from the Internet during the technology workshop along with a few
personally generated images from the conference to share with classmates
through a private Picasa photo-sharing account. Those in the 2010 class were
required to post at least one image downloaded from the Internet during the
technology workshop and at least five personal images to the class Ning, a social
networking site that allows sharing of images.
Digital Stories
Participants from both the 2009 and 2010 sessions of the class posted
copies of their final digital stories to Screencast to promote ease of sharing with
classmates. In addition, class members made copies of their stories on CDs or
DVDs to submit to the course instructor.
Final Night Observation Notes
Notes were taken on participants comments during the final night of
each class when digital stories were shared and discussed.
69


Research Design
Overview of Research Design and Methods
Figure 3.1 summarizes the single-phase mixed-method triangulation
design-transformation model (Creswell & Plano Clark, 2007, p. 85) used in this
study. The figure links research methods to research questions and illustrates
the path from data collection through data analysis to results. For this study,
qualitative and quantitative data were collected and analyzed simultaneously.
The design employs qualitative to quantitative (QUAL to quan) transformation
as well as use of qualitative data to triangulate quantitative survey results. Use
of qualitative data is dominant in this design (Creswell & Plano Clark, 2007).
70


Question 1: How does PR of teacher reflections evident
in digital stories compare to that evident in journals?.
QUAL
data
collection
Digital stories
Journals
Photo logs
Story scripts
Online reflections
Survey responses
Final discussion
QUAL
data
analysis
Content
analysis
QUAL
triangulated
with
additional
data
QUAL
data
results
Themes
Transform
QUAL
into quan
TR dichotomous
LoR ordinal
PR ordinal
Question 3: How well do age, teaching experience, grade
taught, technological competency, and year in which class was
attended predict the PR of teacher reflections in digital stories?
QUAN (scale) +
qual
data collection
Surveys
QUAN + qual
data analysis
QUAN
triangulated
with qual
QUAN + qual
data results
quan quan
data data
analysis results
Descriptive
statistics
Comparative
results
Question 2: What does the PR of
digital stories reveal about what is
deeply important to teachers?
QUAL + QUAL +
quan > quan
data data
analysis results
Use PR to
rank and
select themes
Themes most
important to
teachers
Age (interval)
Teaching experience (interval)
Grade taught (ordinal-^scale)
Technological competency (ordinal->scale)
Year class was taken (dichotomous)
QUAN + QUAN +
qual qual
data data
analysis results
Multiple
regression
Best
predictors
Legend: TR-Topical Relevance LoR Level of Reflection PR Practical Relevance
Figure 3.1. Modified, Single-Phase Mixed-Method Triangulation Design-Transformation Model Depicting
Flow of Analysis Directed toward Three Research Questions


Research Question 1 was addressed by using methods transforming
qualitative data to quantitative data to compare the practical relevance values of
journals and digital stories. To examine what is deeply important to teachers,
Research Question 2 was approached using practical relevance values
(determined from Question 1) to sort emerging themes generated with
qualitative methods for a representative set of artifacts. Practical relevance
considerations also guided the in-depth discussion of these artifacts. Finally,
Research Question 3, exploring factors of practical relevance, required the use of
statistical methods operating on quantitative predictor variables generated
primarily from surveys and practical relevance values determined when
answering the first research question. Methods used in this study are discussed
below by category as they pertain to the research questions.
Methods for Transforming Qualitative
to Quantitative Data
To answer Research Question 1, numerical values for topical relevance,
level of reflection, and practical relevance were generated holistically for
journals and digital stories (qualitative artifacts) and for each element in those
artifacts. Practical relevance was determined by following the steps that are
listed here, highlighted below, and presented in more detail in Appendix E:
72


1. Divide artifacts into elements.
2. For each element, determine topical relevance, level of reflection, and
practical relevance.
3. For core message of the artifact, generate values for topical relevance,
level of reflection, and practical relevance.
4. Generate values for topical relevance, level of reflection, and practical
relevance for an artifact by combining the average values of the elements
with corresponding core message values.
As intermediate steps in determining practical relevance of reflection
artifacts or their individual elements, values for topical relevance and levels of
reflection were obtained through processes that transformed qualitative data
into quantitative data. This transformation was accomplished for topical
relevance by subjecting data to a set of criteria and for level of reflection by
passing data through an operationalized reflection framework. Because the risk
of subjectivity was great in these steps of the analysis, particular attention was
paid to interrater reliability of practical relevance. The method employed to
determine interrater reliability of practical relevance is discussed in the
Quantitative Methods section of this chapter.
Division of Artifacts into Elements
Prior to determining topical relevance, level of reflection, and practical
relevance of artifacts, digital stories and journals were segmented into elements.
Digital story elements consisted of chunks of narrative, in the form of sentences
73


or stanzas (Riessman, 1993), along with supporting images, transitions, and
music that embodied a complete thought or idea. Similarly, journal elements
were composed of sentences, paragraphs, or other groupings of text that
embodied a complete chain of thought.
Initially, delineation of elements was guided by the paragraphs and
headings used in participants' journals and by groupings of narrative and images
in participants scripts or storyboards. Digital story elements were further
subdivided into narrative and visual subelements. Then, if necessary,
adjustments were made to the groupings. For example, in some cases the
paragraphs in journals were split into two or more elements that each expressed
one fully formed idea.
Topical Relevance
Elements of journals and digital stories were assigned a value of 0 or 1 to
indicate absence or presence of topical relevance, respectively. To make this
determination within the context of this study for elements in reflection
artifacts, the following question was asked: Does the idea expressed in the
element accomplish one or more of the following?
draw from conference experiences to inform teaching practice
draw from conference content to inform practice
draw from the conference to inform professional identity
74


An answer of "yes" in any case equated to the presence of topical relevance and a
resultant score of 1. An answer of "no to all cases equated to the absence of
topical relevance and a resultant score of 0.
The next step was to assign a value of topical relevance to the core
message of each journal and digital story by asking the following question:
In light of assignment requirements and goals articulated by the
author, did the artifact, as a whole, address conference
experiences in ways that could inform teaching practice or
professional identity?
As in the case of elements, an answer of "yes" equated to the presence of topical
relevance and a resultant score of 1. An answer of "no" equated to the absence
of topical relevance and a resultant score of 0.
Level of Reflection
Elements of journals and digital stories were assigned values ranging
from 1 to 3 to indicate their levels of reflection. In this process, each element in a
reflection artifact was evaluated using the reflection framework [Table 2.2) and
visual criteria for level of reflection [Figure 2.1) to infer a level of reflection on
the part of a participating teacher. To operationalize the six columns or threads
in the reflection framework, the following questions were applied to narrative
subelements in an artifact:
Cognitive Domain [column 1): Is there evidence of a problem to be solved
or solution to be found? What level in the process is inferred?
75


Cognitive Domain (column 2): Is there evidence that cognitive skills were
being employed? What level is inferred?
Affective Domain (column 3): Is there evidence of a personal concern
prompting or requiring action that is predicated upon an emotional
trigger? What level in the process is inferred?
Affective Domain (column 4]: Is there evidence that reflections were
being driven by emotional, philosophical, or ethical factors? To what
level?
Experiential Domain (column 5): Is there evidence of spontaneous
reaction, occurring in the moment and being informed by experience? To
what level?
Experiential Domain (column 6): In which frame of reference do the
reflections reside?
To operationalize the visual reflection criteria, this question was asked:
To what extent did each image or series of images reinforce the thought or idea
expressed in the associated text? Step-by-step examples of determining level of
reflection for elements in an artifact are provided in Appendix E.
The next step was to assign a value of level of reflection to the core
message of each journal and digital story. This value was determined by
summarizing the overarching message of the artifact and evaluating the message
using the reflection framework. The approach used to refine the
operationalization of the reflection framework is discussed next.
76


Refinement of Operationalized Procedures
To begin the process, five pairs of digital stories and companion journals
were selected for independent evaluation by me and a second analyst. The
second analyst possessed substantial familiarity with the project. He had
participated in previous studies relating to Dr. Marlows study (Marlow et al.,
2006) and also had acted as the second technology facilitator during the 2009
and 2010 technology workshops. Artifacts were those created by members of a
representative subset of the study population, previously introduced in this
chapter (refer to Table 3.2).
The core messages of the ten artifacts were evaluated for levels of
reflection and topical relevance. Results from the two analysts were compared
and reconciled for each artifact. Then each element within an artifact was
evaluated for its level of reflection and topical relevance. Results from the two
analysts were compared and reconciled element by element. As inconsistencies
in interpretation between the two analysts were examined and discussed,
clarifying modifications were incorporated into the procedures and criteria
presented in Appendix E.
77


Practical Relevance
As a final step in the analysis of elements, values of practical relevance for
each element of the representative artifacts were computed by taking the
product of the associated values for topical relevance and level of reflection.
Similarly, the value of practical relevance for the core message of an artifact was
computed by taking the product of the corresponding values for topical
relevance and level of reflection. Finally the practical relevance of a reflection
artifact was computed by adding the value of practical relevance for the core
message to the average practical relevance for all elements in the artifact and
dividing that sum by 2.
Qualitative + Quantitative (QUAL + quan) Methods
To answer Research Question 2 and thereby understand what the
practical relevance of journals and digital story elements revealed about what
was deeply important to teachers, the artifacts generated by the five participants
in the representative subset (identified earlier in this chapter) were examined.
Qualitative themes that emerged from their journal and digital story elements
were ranked according to values of practical relevance derived when answering
Research Question 1.
Because the ranges of practical relevance values were different for this
small subsample of artifacts, the cutoff point for inclusion in the analysis was
78


established by determining the median value of practical relevance for each
artifact. Elements with scores falling at or above the median for each artifact
were included in the analysis.
Themes were then grouped according to qualitative codes, depicted
graphically, and analyzed using descriptive statistics. Qualitative methods used
to identify emerging themes are discussed in the next section.
Qualitative Methods
Lieblich et al. [1998) define narrative research as "any study that uses or
analyzes narrative materials" [p. 2). For purposes of this study, narratives in
digital story form were analyzed to investigate which experiences associated
with the space-science class were afforded the highest values of practical
relevance by participating teachers. Through a categorical-content type of
narrative analysis, which follows along the lines of traditional content analysis,
stories from multiple narrators were examined for emerging themes (Lieblich et
al., 1998). Similarly elements of journal entries were analyzed for emerging
themes with a modified form of constant comparative analysis (Krathwohl,
2004; Lieblich et al., 1998). Topical relevance of those elements had been
established previously by coding them according to a code set developed
specifically for that purpose from journal prompts and digital story assignment
requirements.
79


The journals and digital stories from the five participants who were
introduced earlier in this chapter were used for the thematic analysis to provide
a representative sense of which experiences resonated most deeply with
participants. To begin the analysis, journal and digital story elements were
coded sequentially, using a new code set derived from codes and subcodes
emerging through open coding (Creswell, 1998). These codes were predicated
on indicators of conference impact such as type of activity, feelings, or space-
science content. Codes drawing from the implicit content of the stories were
generated by considering what the images and music or other sound (if present)
brought to the story (Lemon, 2007; Lieblich et al., 1998; Richards, 2006).
The second analyst and I coded the elements independently, discussed
the coding results, and subsequently reconciled the two sets of codes "to create
higher sensitivity to the text and its meaning to different readers (Lieblich et al.,
1998, p. 114). Codes were then recombined for artifacts of each individual
through axial coding (Creswell, 1998). Online participant reflections, qualitative
survey responses, and notes from final presentation nights were used to
triangulate qualitative results.
80


Quantitative Methods
This section discusses methods used to address interrater reliability as
well as the multiple regression analysis used to investigate how well certain
independent variables predicted practical relevance of digital stories.
Establishing Interrater Reliability of Practical Relevance
A third analyst was engaged to analyze artifacts independently for
interrater reliability purposes. He had little familiarity with the project and
worked in the private sector as the lead statistician and manager of the data
processing department for an international market analysis firm.
The third analyst was introduced to the study and trained in the
procedures to determine topical relevance, levels of reflection, and values of
practical relevance from artifacts. Training consisted of familiarizing the analyst
with the theoretical underpinnings of the procedure and coaching the individual
through the procedure using one digital story and companion journal from the
previously coded representative set. As each element was evaluated by the third
analyst, his results were discussed in light of the results previously reconciled
from the work of the first two analysts.
After training, the third analyst independently evaluated the remaining
four digital stories and companion journals (eight artifacts) in the representative
set for topical relevance, level of reflection, and practical relevance. The
81


independent practical relevance results of the third analyst and the reconciled
results from the evaluation effort of the first two analysts were used to
determine interrater reliability of practical relevance values.
A Pearsons correlation was computed to assess the interrater reliability
of the scores for each of the following data sets: (a) practical relevance for
artifacts, r(8) = .78; (b) average practical relevance of elements in artifacts, r(8)
= .88; and (c) values of practical relevance for core messages of artifacts, r(8) =
.62. These results indicate that there is good interrater reliability for practical
relevance and its factor of average practical relevance of elements, and there is
acceptable interrater reliability of practical relevance of core messages.
Pearson's correlation was selected because the process to determine practical
relevance involved subjective judgment (Leech, Barrett, & Morgan, 2008, p. 46).
Hierarchical Linear Regression Analysis
Pursuing an answer to Research Question 3 required a method to
determine which independent variables, listed in Table 3.5, best predicted
practical relevance for reflection artifacts in the sample set.
82


Table 3.5. Multiple regression variables and their properties
Variable IV vs. DV Type Values or Range
Practical Relevance Dependent Interval 0 to 3
(PI) Year class was taken Independent Dichotomous 2009 or 2010
(P2) Grade taught Independent Ordinal treated as Interval 0-10 (K-HS)
(P3) Age Independent Interval 23 to 59
(P4) Years of teaching experience Independent Interval 0 to 9
(P5) Technological competency Independent Ordinal treated as Interval 1 to 20
(end of the class)
Legend: IV = Independent variable K=Kindergarten
DV = Dependent variable HS = High school
Predictors were selected based upon the assumptions supported by a synthesis
of the literature presented in Chapter 2 and are discussed below.
Predictor Variable 1: Year Class Was Taken
The year in which the associated space-science class was attended by
participants {year class was taken) could be a proxy for differences in curriculum
content and delivery during 2009 and 2010. These differences could have
influenced teachers reflection processes and products (Hegarty, 2009;
Korthagen & Vasalos, 2005; Ryan, 2005].
83


Predictor Variable 2: Grade Taught
The content, purpose, and structure of a participant's digital story were
guided to some extent by course requirements (Dunlap, 2006; Ryan, 2005].
Because teachers were asked to create stories that could be shared with their
students, the grade taught by a participant {grade taught] and the associated
cognitive maturity of the intended audience (Kegan, 1994) could inform the
complexity and practical relevance of a story's message.
Predictor Variable 3: Age
Teachers of different ages {age] could think differently as a result of their
familiarity with technology (Prensky, 2001) or could have a larger or broader
cache of life experiences to inform their reflections (Marlow & Renda, 2005).
Predictor Variable 4: Years of Teaching Experience
Teachers with a greater number of years of teaching experience {years of
teaching experience] could have a larger or broader cache of experiences to
inform their reflections and could have reflection skills superior to those of
teachers with less experience (Marlow & Renda, 2005; Slepkov, 2008).
Predictor Variable 5: Technological Competency
Teachers' ability to express themselves through digital storytelling could
be helped or hindered by their technological competency (Renda, 2010).
84


Because there was a risk of collinearity [a] between the variable year
class was taken and each of the remaining predictor variables, a hierarchical
linear regression analysis [Leech et al., 2008, p. 94) was conducted. The analysis
was structured to investigate how well practical relevance of digital stories was
predicted by grade taught, years of teaching experience, age, and technological
competency, when controlling for year class was taken.
Study Limitations
A number of limitations inherent to this study were considered as the
study progressed:
Level of reflection was both defined and limited by the framework
developed for this study.
Potential for bias, on my part, was introduced to the study through
my direct interaction as a technology facilitator for participants.
Trustworthiness, validity, and reliability of the data and results of
this mixed methods study were highly dependent upon analysts
abilities to assign level of reflection and practical relevance in a
consistent and meaningful fashion.
Generalizability of study results to other populations was not
possible because of the small study population and limited setting,
characteristic of qualitative research.
The multiple regression analysis was limited by a small sample set.
85


Trustworthiness, Validity, and Reliability
Krathwohl [2004] defines trustworthiness as "the judged credibility of a
qualitative research study based upon the appropriateness of the data gathering
and analytic processes and their resulting interpretations" (p. 694). To promote
trustworthiness in this study, raw qualitative data were coded in a methodical
way, using a categorical-content type of narrative analysis (Lieblich et al., 1998)
for digital stories and modified constant-comparative analysis of emerging
themes (Krathwohl, 2004; Lieblich et al., 1998) for journals. One goal of using
these analytical approaches was to conduct the analyses in such a way that
another person could understand the themes and arrive at similar results.
To advance the credibility of the study, I employed triangulation.
Specifically, I used multiple sources of data including participants' digital stories,
journal entries, online reflections, responses to survey items, and comments
during the final class discussion to corroborate results. Although I had no role in
assigning grades to class participants, my role as technology facilitator and the
associated potential for researcher bias has been acknowledged as a threat to
credibility (Onwuegbuzie & Leech, 2007).
Challenges to validity and reliability of quantitative data and results are
compounded when quantitative data are derived through the transformation of
qualitative data. In this study, values of topical relevance, level of reflection, and
86


practical relevance were derived from qualitative data for each element of each
journal and digital story. In addition, values of practical relevance were derived
for the core message of each journal and digital story and for each journal and
digital story as a whole. Thus validity, or ability to draw meaningful inferences
from the data to the study population, was predicated upon the construct
validity of the topical relevance criteria and the level of reflection framework
(Krathwohl, 2004]. In this study, construct validity was supported by findings in
the literature.
In this study, reliability (consistency of results] was assured through the
calculation of interrater reliability. An acceptable result for interrater reliability
shows that analysts acting independently can achieve acceptably consistent
results when using a method involving some measure of subjective judgment
(Leech et al., 2008].
The methods described in this chapter were employed to answer the
research questions that framed this study. Results of the study, organized by
research question are presented in the next chapter.
87