Citation
The effects of embedded generative learning strategies and collaboration on knowledge acquisition in a cognitive flexibility-based computer learning environment

Material Information

Title:
The effects of embedded generative learning strategies and collaboration on knowledge acquisition in a cognitive flexibility-based computer learning environment
Creator:
Higgs, Jr., John Robert
Place of Publication:
Denver, CO
Publisher:
University of Colorado Denver
Publication Date:
Language:
English
Physical Description:
153 leaves : illustrations ; 28 cm

Subjects

Subjects / Keywords:
Knowledge and learning ( lcsh )
Computer-assisted instruction ( lcsh )
Learning, Psychology of ( lcsh )
Team learning approach in education -- Case studies ( lcsh )
Computer-assisted instruction ( fast )
Learning and scholarship ( fast )
Learning, Psychology of ( fast )
Team learning approach in education ( fast )
Genre:
Case studies. ( fast )
bibliography ( marcgt )
theses ( marcgt )
non-fiction ( marcgt )
Case studies ( fast )

Notes

Bibliography:
Includes bibliographical references (leaves 144-153).
Thesis:
Educational leadership and innovation
General Note:
School of Education and Human Development
Statement of Responsibility:
by John Robert Higgs, Jr.

Record Information

Source Institution:
|University of Colorado Denver
Holding Location:
|Auraria Library
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
40462335 ( OCLC )
ocm40462335
Classification:
LD1190.E3 1998d .H54 ( lcc )

Downloads

This item has the following downloads:


Full Text
THE EFFECTS OF EMBEDDED GENERATIVE LEARNING STRATEGIES AND
COLLABORATION ON KNOWLEDGE ACQUISITION IN A
COGNITIVE FLEXIBILITY-BASED COMPUTER LEARNING ENVIRONMENT
by
John Robert Higgs, Jr.
B.S., University of Maryland, 1980
M.A., State University of New York, Plattsburgh, 1991
A thesis submitted to the
University of Colorado at Denver
in partial fulfillment
of the requirements for the degree of
Doctor of Philosophy
Educational Leadership and Innovation
1998


1998 by John Robert Higgs, Jr.
All rights reserved.


This thesis for the Doctor of Philosophy
degree by
John Robert Higgs, Jr.
has been approved
by
Laura D. Goodwin
Michael P. Marlow

l_9V/
Date
Daniel R. Ambruso


Higgs, John Robert, Jr. (Ph.D., Educational Leadership and Innovation)
The Effects of Embedded Generative Learning Strategies and Collaboration on
Knowledge Acquisition in a Cognitive Flexibility-Based Computer Learning
Environment
Thesis directed by Associate Professor R. Scott Grabinger
ABSTRACT
Inert knowledge is typically produced in learning environments that simplify
content and context. Several theories and methods exist to establish learning
environments that overcome the negative effects of inert knowledge. This study
examined the effects of combining three instructional approaches on knowledge
acquisition by advanced learners in a computer-based learning environment.
This study used a 2 X 2 experimental design. The two independent variables
were program version and method of instruction. Two qualitatively different
computer programs were used: a base program developed according to cognitive
flexibility theory (CFT) and a generative program containing embedded generative
learning strategies (explanation and summary cues). Students completed these
programs either individually or collaboratively.
One hundred and thirty-two second-year medical students participated in the
study. The two treatment variables were randomly assigned to the students. Students
received computer diskettes that permitted access to the computer module, "Handling
Transfusion Hazards," and were given instructions on how, when, and where to
complete the program. The computer program consisted of an orientation section, a
IV


pretest, a learning phase, and a posttest. The pretest and posttest consisted of the
same three clinical transfusion cases; the learning phase contained six cases.
One hundred and one students were included in the final analysis. Analysis of
covariance was used to test for the presence of main effects or an interaction. The
ANCOVA yielded a significant main effect (p=.008) on posttest performance for the
version of program treatment variable. No significant differences in achievement
were found for method of instruction, nor was an interaction present between the two
treatment variables.
This research suggests that advanced learners did not benefit equally from the
two computer programs. Learners using the base computer program performed
significantly better than students using the generative version on the posttest. Also,
although the students felt there was an advantage to working with a peer, no
significant advantage materialized. Recommendations for future research based on
these findings are presented.
This abstract accurately reflects the content of the candidate's thesis. I recommend its
publication.
Signed
v


DEDICATION
To my family, Kathy, Stephanie, and Katie. Thank you for your unyielding support
and understanding during this doctoral process. Also a special thanks to Lieutenant
Colonel Michael Whyte, who supported my doctoral pursuit.


ACKNOWLEDGEMENTS
I would like to acknowledge Scott Heath who provided the programming
expertise to develop the generative version of the program used in this study.
Additionally, I would like to thank Lisa Traditi and her staff (Kathei, David, and Jim)
at the Dennison Library Learning Resources Center for the support they provided
during the data collection period.
To my dissertation committee members Mike Marlow, Judy Duffield, Laura
Goodwin, and Dan Ambruso thank you for your active involvement during the
dissertation process. I would particularly like to thank Dan Ambruso for the
opportunity to work with the Transfusion Medicine Team. Laura Goodwin deserves
special recognition for the significant time she spent guiding the research design and
data analysis used in this investigation.
Finally, Scott Grabinger served as program advisor and dissertation chair during
my doctoral program. He invested three years as advisor and mentor, and provided
the support, encouragement, and friendship necessary for me to successfully complete
this study. Thank you Scott for your wisdom and guidance.


CONTENTS
Figures...........................................................xiii
Tables............................................................xiv
CHAPTER
1. STATEMENT OF THE PROBLEM.........................................1
Introduction..................................................1
Purpose of the Study..........................................4
Problem Statement......................................4
Problem Solution.......................................5
Cognitive Flexibility Theory..................................6
Generative Learning Theory....................................6
Collaborative Learning........................................7
Research Issues...............................................8
Research Questions...........................................10
Research Methodology.........................................11
Experimental Intervention.............................11
Dependent Variable....................................12
viii
Definition of Terms
14


2. REVIEW OF LITERATURE
18
Introduction...................................................18
Cognitive Flexibility Theory: Assumptions and Theory...........21
Research on Cognitive Flexibility Theory.................27
Generative Learning Theory.....................................30
Compatibility of Cognitive Flexibility Theory and
Generative Learning Theory.....................................32
Individual Student Focus.................................34
Collaborative Learning.........................................34
Research on Collaborative Learning.......................37
Research Questions.............................................40
Research Hypotheses............................................41
3. METHODOLOGY........................................................43
Introduction...................................................43
Materials......................................................43
The Program: Handling Transfusion Hazards................44
Program Development and Evaluation.......................53
Participants...................................................56
Assignment to Treatment........................................56
Design.........................................................58
IX


Variables
59
Independent Variables.....................................59
Dependent Variable........................................60
Instruments......................................................62
Pretest and Posttest......................................62
Procedure........................................................63
Pilot Study...............................................63
Instructional Intervention.......................................66
4. RESULTS.............................................................68
Introduction.....................................................68
Data Collection and Analysis Screening Procedures................69
Primary Data Analysis............................................70
Assumptions for Analysis of Covariance....................70
Summary and Descriptive Statistics........................76
Analysis of Covariance....................................77
Secondary Data Analyses..........................................80
Multivariate Analysis of Variance.........................80
Program Effectiveness.....................................83
Time Spent on Instruction.................................87
Analysis of Explanations and Summaries....................88
x


Analysis of Student Comments..........................89
Summary of Results...........................................91
5. DISCUSSION AND RECOMMENDATIONS..................................93
Introduction.................................................93
Discussion...................................................94
Effect of Embedded Generative Learning Strategies.....94
Effect of Collaboration...............................98
Interaction of Generative Learning Strategies
and Collaboration....................................101
Other Consideration: Influence of the Overall
Instructional Environment............................102
Summary..............................................106
Recommendations for Future Research.........................106
Study Limitations...........................................109
xi


APPENDIX
A. SAMPLE PRACTICE CASE....................112
B. SAMPLE GENERATIVE SCREENS...............121
C. SAMPLE TEST CASE...................... 124
D. STUDENT INSTRUCTIONAL HANDOUT...........132
E. PRETEST/POSTTEST DATA SUMMARY...........135
F. STATISTICAL INFORMATION.................137
G. SAMPLE STUDENT COMMENTS.................142
REFERENCES.......................................144
Xll


FIGURES
Figure
3.1 Sample Case Screen from Practice Case Three.............................46
3.2 Sample History Screen from Practice Case Three..........................47
3.3 Sample Perspectives Screen from Practice Case Three.....................48
3.4 Sample Similar Cases Screen from Practice Case Three....................49
3.5 Textbook Index Screen...................................................50
3.6 Experimental Design.....................................................59
3.7 Sample Screen from Test Case B..........................................61
xiii


TABLES
Table
4.1 Pearson Correlation Coefficients..........................................73
4.2 Summary Statistics........................................................77
4.3 Analysis of Covariance Summary............................................78
4.4 Adjusted Cell and Marginal Means..........................................78
4.5 Tests for Equal Cell Variance............................................81
4.6 Univariate Tests of Significance........................................ 82
4.7 Posttest Case Scores for Version of Program...............................82
4.8 Aggregate Pretest/Posttest T-test Results.................................85
4.9 Mean Time Spent on Instruction............................................87
4.10 Summary Statistics for Explanations and Summaries.........................89
4.11 Student Comments..........................................................90
xiv


CHAPTER 1
STATEMENT OF THE PROBLEM
Introduction
A common criticism of the traditional model of teaching and learning involves
the lack of effective instructional methods to facilitate critical thinking and higher
levels of learning (Bransford, Franks, Vye, & Sherwood, 1989; Grabinger, 1996;
Resnick, 1987; Scardamalia & Bereiter, 1994; Spiro, Coulson, Feltovich, &
Anderson, 1988; Weinstein, 1978). The goal of instruction in the traditional model of
education involves efficiently and effectively transferring knowledge to learners by
breaking instruction down into simple, basic units devoid of content and context
(Bednar, Cunningham, Duffy, & Perry, 1992). This form of teaching produces
students who treat information as facts to be memorized and recited rather than as
tools to solve problems... (Grabinger, 1996, p. 666), which results in a use-of-
knowledge gap (Perkins, 1992). The traditional model of teaching, therefore,
produces learners unable to recall, use, or transfer knowledge and skills to new and
novel situations. Whitehead (1929) calls this inert knowledge since it is
information students in fact possess, but because it is memorized devoid of context, is
not transferable, even to relevant situations (Bransford et al., 1989; Bransford & Vye,
1989; Cognition and Technology Group [CTG], 1992; Grabinger, 1996; Perkins,
1


1992; Whitehead, 1929). Thus, due to the absence of context, inert knowledge maybe
the predominant form of knowledge acquired by students in the traditional model of
education.
In contrast to the traditional model of education, many educators and scholars
believe that todays complex world requires the ability to use tools and knowledge in
a variety of domains and situations. These critical-thinking skills allow people to
fully participate in our modem, information-age society by giving them the ability to
analyze and synthesize information to solve technical, social, economic, political, and
scientific problems (Grabinger, 1996, p. 665). The question then becomes how to
encourage critical-thinking skills development in students rather than the formation of
inert knowledge. Resnick (1987) suggests that the key to developing critical-thinking
ability involves using instructional strategies that (a) emphasize socially shared
intellectual work organized around joint task accomplishment, (b) encourage student
observation and commentary which makes usually covert cognitive processes overt,
and (c) tailor the treatment of the subject matter to engage students in the processes of
meaning construction and interpretation. Such student-centered strategies increase
time on task, increase the amount and relevance of feedback to each student, use a
performance-based system to assess student achievement (rather than a system where
all students spend the same amount of time studying instructional material), and
incorporate self-paced instruction geared to student capabilities rather than class
average (Reiser & Salisbury, 1995). This approach to learning promotes achievement
2


of the basic goals of education retention, understanding, and active use of the
knowledge and skills (Perkins, 1992). Perkins (1992) writes:
Surely we want what is taught retained, else why teach it? Unless knowledge
is understood, to what purposes can it be put? Finally, having and
understanding knowledge and skills come to naught unless the learner actually
makes active use of them later in life. (p. 18)
A student-centered approach considers learning to be an experiential process
(Dewey, 1938) that promotes the social structures and dynamics required for
knowledge building to occur among students (Scardamalia & Bereiter, 1994). Rather
than focusing on the teacher-centered model of education, which is based on the
assumption that students will learn because they are asked or told to learn (Weinstein,
1978), a student-centered approach focuses on students sharing perspectives with the
teacher and other students and then modifying internal representations of knowledge
in response to this sharing experience. In a student-centered environment, learning
becomes a process of constructing, interpreting, and modifying representations of
reality based on experience (Jonassen, 1994). Wilson (1996) defines such a student-
centered learning environment as a place where learners may work together and
support each other as they use a variety of tools and information resources in their
guided pursuit of learning goals and problem-solving activities (p. 5).
3


Purpose of the Study
Problem Statement
For many years now, research (Feltovich, Spiro, & Coulson, 1989; Spiro et al.,
1988; Spiro, Vispoel, Schmitz, Samarapungavan, & Boerger, 1987) has been
conducted on learning at the stage of advanced knowledge acquisition (i.e., the post
introductory or intermediate stage of learning in a subject domain). The goals of
advanced knowledge acquisition shift from recognition and recall to attaining a deep
understanding and flexible application of content material (Spiro et al., 1988);
however these goals have been hard to reach. This has been especially true of
advanced students trying to learn in ill-structured knowledge domains (e.g.,
medicine). For example, Feltovich, Spiro, and Coulson (1989) found that medical
students had problems transferring knowledge previously learned in one context, such
as medical school coursework, to new situations, such as clinical problem solving.
Spiro and his colleagues attribute the deficiencies in learning outcomes (i.e., inert
knowledge formation) of students at this advanced stage of learning to
oversimplification (reductive bias) of complexity.
A predominant share of the misconceptions (and networks of misconception)
that we have identified reflect one or another kind of oversimplification of
complex material... .Misconceptions of advanced material result from both
interference from earlier, simplified treatments of that material and from a
prevailing mode of approaching the learning process in general that fosters
simplificational strategies and leaves learners without an appropriate cognitive
repertoire for the processing of complexity. (Spiro et al., 1988, p. 376)
4


Thus, ill-structuredness (the combination of breadth, complexity, and irregularity of a
content domain) and the goals of advanced knowledge acquisition unite to create a
difficult problem for both teachers and students who tend to rely on simple strategies
to present and learn complex subject matter.
Problem Solution
In response to the problem of reductive bias, Spiro and his colleagues (1988)
proposed cognitive flexibility theory as a means to create learning environments that
provide the complexity and multidimensionality of content required for advanced
knowledge acquisition in ill-structured domains. This study investigated the effects
of a computer-based microworld on knowledge acquisition among advanced learners.
The computer program was developed according to principles of learning, instruction,
and knowledge representation espoused by cognitive flexibility theory. The program
was further enriched with embedded generative learning strategies (explanation and
summary cues) and collaborative activity to produce a computer-based microworld
that presents information, stimulates exploration, facilitates high level thought
processes, and promotes collaboration (Grabinger, 1996). The remainder of this
chapter briefly outlines cognitive flexibility theory, generative learning theory, and
collaborative learning, frames the research questions, and provides an overview of the
research methodology.
5


Cognitive Flexibility Theory
Cognitive flexibility theory (CFT) targets the stage of advanced knowledge
acquisition, and, as such, provides a framework for addressing the difficulties
inherent in complex and ill-structured knowledge domains (Spiro et al., 1988; Spiro,
Feltovich, Coulson, & Anderson, 1989; Spiro & Jehng, 1990). Evidence suggests
that learners develop conceptual misunderstandings during the advanced phase of
learning, which seriously affects their ability to transfer knowledge to new situations
(Jonassen, Ambruso, & Olesen, 1992; Spiro et al., 1988). Cognitive flexibility theory
emphasizes case-based instruction with multiple representations of content in ill-
structured knowledge domains to avoid oversimplifying instruction (Spiro & Jehng,
1990). This enables learners to investigate the multiple perspectives inherent in a
knowledge domain in an exploratory manner (Jonassen et al., 1992), which promotes
knowledge acquisition and transfer.
Generative Learning Theory
Generative learning theory is based on the notion of an active learner (Wittrock,
1974a; 1974b; 1985; 1990; 1992). In this case, the learner works to create meaning
by drawing relationships between existing information and experiences stored in
cognitive structures and information presented in the learning environment
(Grabowski, 1997; Wittrock, 1985). Wittrock (1974b) demonstrates the importance
of active learner participation in the learning process by writing: Although a student
6


may not understand sentences spoken to him by his teacher, it is highly likely that a
student understands sentences that he generates himself (p. 182). Thus,
understanding cannot be directly transferred to students, but must arise from the
interactions between the learner and the environment (Wittrock, 1974a; 1974b; 1985).
The new meaning constructed from the interactions between the learner and the
environment results in learning with understanding (Wittrock, 1974a; 1974b; 1985),
characterized by the creation of meaningful cognitive structures, better retention and
retrieval of information, and better explanations of the information (DiVesta, 1989;
Grabowski, 1997).
Collaborative Learning
Collaborative learning actively involves learners in the construction of their
knowledge, but, in contrast to most applications of cognitive flexibility theory and
generative learning theory which focus primarily on the individual student's cognitive
development, collaborative learning promotes socially shared intellectual
development. Collaborative learning involves the instructional use of small groups
where students work together to maximize each others learning (Johnson & Johnson,
1992). To accomplish the goal of maximizing learning, the students are responsible
for learning the assigned material themselves and ensuring that all other members of
the group leam the material as well (Johnson & Johnson, 1992). Such peer
groupwork allows the students to actively engage in interactive higher-level thinking
7


and problem-solving activities (Damon & Phelps, 1989) leading to enhanced
academic achievement and cognitive growth, better attitudes and motivation toward
learning, and a host of positive social-emotional benefits (Johnson & Johnson, 1992;
Nastasi & Clements, 1991; Slavin, 1996).
Research Issues
What makes learning so hard and what can be done about it? The answers are
teacher-centered education and student-centered education respectively. Learning
requires intellectual effort on the part of the learner (Osborne & Wittrock, 1985), yet
with its emphasis on passive absorption of information, the teacher-centered model of
education devalues the effort required. In contrast, the student-centered approach to
learning activates the learners information-processing strategies and stores of
relevant specific memories related to the information to be learned (Wittrock, 1978).
This results in mindful (Salomon, 1985) processing. Thus, the student-centered
approach values effortful learning and uses strategies designed to promote
generative learning (i.e., knowledge construction).
Cognitive flexibility theory is a student-centered approach specifically targeted
toward knowledge construction and acquisition by advanced learners in ill-structured
knowledge domains. Cognitive flexibility theory advocates creating case-based
computer learning environments that emphasize the conceptual complexities and
interrelatedness of subject matter content (Spiro et al., 1988). The use of authentic,
8


multiple cases permits students to see many contexts in which concepts occur or
apply, and index meaning accordingly. Students, therefore, develop the ability to
adaptively re-assemble diverse elements of knowledge in response to changing
situational demands (i.e., the needs of a given problem-solving or knowledge
application situation) (Spiro & Jehng, 1990). Thus, cognitive flexibility theory-based
(CFT-based) computer learning environments stimulate mindful processing which, in
turn, facilitates the generation of fluid, flexible, and usable knowledge structures.
The research to date (Hartman & Spiro, 1989; Jacobson, 1990; Jacobson,
Maouri, Mishra, & Kolar, 1996; Spiro et al., 1987) supports this premise; computer
programs based upon cognitive flexibility theory principles do indeed promote higher
levels of knowledge acquisition and transfer (this research is discussed more fully in
Chapter Two). However, given the multitude of student-centered instructional
strategies available that also promote generative learning, might advanced learners
benefit from the simultaneous integration of compatible learning strategies in a CFT-
based environment? Might a synergy develop that results in a greater degree of
mindfulness and consequently, deeper processing of the content material? If so,
would greater knowledge acquisition result?
9


Research Questions
This study examined the effectiveness of a computer-based microworld (see
definition on p. 15) designed to integrate Resnicks (1987) keys for developing
critical thinking and increasing learning. Specifically, this study addressed the
question: Would knowledge acquisition among advanced learners increase if a
cognitive flexibility theory-based computer program was enriched with (a) additional
generative learning strategies designed to facilitate comprehension and understanding
and (b) collaborative activity to attend to the social nature of learning and the
synergistic development and application of knowledge among groups of students? In
this context two main questions were explored:
1. Would the inclusion of embedded explanation and summary cues
(generative learning strategies) result in differences in learning in a
CFT-based computer micro world?
2. How would the addition of collaborative learning affect learning in this
environment?
10


Research Methodology
Experimental Intervention
This study used a 2 X 2 factorial design consisting of two independent variables:
(a) method of instruction (single learners versus collaborative learners) and
(b) program version (CFT base program versus CFT generative program). This
design resulted in four treatments consisting of:
1. Single student, base program (Single/base). Subjects in this treatment
used the CFT base computer program (i.e., the computer program without the
embedded generative learning strategies).
2. Single student, generative program (Single/generative). Subjects in this
treatment used the CFT computer program with the embedded generative
learning strategies. These subjects were asked to explain the choices they
made during the program, as well as to summarize the main ideas and
concepts for each case presented.
3. Paired students, base program (Pair/base). Subjects in this treatment
collaborated with a partner while using the CFT base computer program.
4. Paired students, generative program (Pair/generative). Subjects in this
treatment collaborated with a partner while using the CFT generative
computer program. These students were asked to explain and summarize in
the same manner as the single/generative subjects.
11


Dependent Variable
The dependent variable for this study was the knowledge constructed
(i. e., acquired) by advanced learners. Subjects took a posttest consisting of three
transfusion medicine cases similar in structure and content to the six cases contained
in the practice segment of the computer program. However, the present study
departed from the traditional practice of requiring the students assigned to the
collaborative condition to test individually based on the following rationale:
1. Collaborative learning is presumed to provide positive group-to-individual
transfer. Group-to-individual transfer occurs when students...demonstrate
mastery of the material being studied on a subsequent test taken individually
(Yager, Johnson, & Johnson, 1985, p. 61). Research on collaborative,
computer-based instruction generally supports this assertion; students working
collaboratively in computer-based instruction perform as well as students
working alone (Carrier & Sales, 1987; Makuch, Robillard, & Yoder, 1992;
Webb, 1985; 1987; 1989) and often perform better (Dalton, Hannafin, &
Hooper, 1989; Hooper, Temiyakam, & Williams, 1993; Johnson, Johnson, &
Stanne, 1985; 1986).
2. Collaborative learning is also premised on the notion that collaborative
activity leads to emergent knowledge a level of knowledge that results from
the interaction between the understandings of the group rather than a
summation of the knowledge of the individual participants (Whipple, 1987).
12


Scaffolding enables the group to solve a problem or achieve a goal which is
beyond the capabilities of the individuals involved. If, in fact, collaboration
leads to higher performance of the group, why test collaborating students
individually? After all, research demonstrates that group-to-individual
transfer occurs; therefore, if the learning experience is positive for students
working individually, the learning experience should also be positive for the
students working in groups.
3. The computer program used in this study was entirely self-contained.
The learning and testing phases of the program were developed and delivered
as one complete module, with the pretest and posttest immediately preceding
and following the learning phase respectively. Factor in the medical students
tight schedules and it would have been awkward and inexpedient to require
separate testing. Furthermore, sentiment exists among some researchers that
promoting shared cognitive activity but assessing learning individually in the
same study leads to inconsistencies in results (Forman & Kraker, 1985). This
study, therefore, accepted the assumptions of a positive learning experience
for the students working individually and that transfer would occur for the
collaborative pairs. This provided the opportunity to compare individual
versus socially-mediated knowledge acquisition.
13


Definition of Terms
Advanced knowledge learning beyond the introductory stage for a subject
area, but before the achievement of practiced expertise that comes with massive
experience (Spiro et al., 1988, p. 375).
Case a contextualized piece of knowledge representing an experience that
teaches a lesson fundamental to achieving the goals of the reasoner (Kolodner,
1993, p. 13).
Case-based learning involves problem solving, understanding, and learning.
Case-based learning can mean adapting old solutions to meet new demands,
using old cases to explain new situations, using old cases to critique new
solutions, or reasoning from precedents to interpret a new situation (much as
lawyers do) or create an equitable solution to a new problem (Kolodner,
1993, p. 4).
Cognitive flexibility the ability to spontaneously restructure ones knowledge,
in many ways, in adaptive response to radically changing situational demands
(both within and across knowledge application situations) (Spiro & Jehng,
1990, p. 165).
Cognitive flexibility theory a conceptual model for instruction that is based
upon cognitive learning theory. Its intention is to facilitate the advanced
acquisition of knowledge to serve as the basis for expertise in complex and ill-
structured knowledge domains (Jonassen et al., 1992, p. 312).
14


Collaborative learning Collaboration is a coordinated, synchronous activity
that is the result of a continued attempt to construct and maintain a shared
conception of a problem (Teasley & Roschelle, 1993, p. 235). Collaboration
involves a coordinated and mutual effort to problem solve (Teasley & Roschelle,
1993).
Comprehension results from formulating connections, rather than from placing
information or transforming information in memory. The subtle difference lies
in the creation of new understanding of the information by the learner, rather than
changing the presented information (Grabowski, 1996, p. 898).
Computer-based microworlds computer programs that are a small but
complete subset of reality in which one can go to learn about a specific domain
through personal discovery and exploration (Rieber, 1992, p. 94). Computer-
based microworlds permit students to explore, experience phenomena, and
formulate hypotheses (Hannafm, Hannafin, Hooper, Rieber, & Kini, 1996).
Computer-based microworlds present information, stimulate exploration, facilitate
high level thought processes, and promote collaboration (Grabinger, 1996).
Cooperative learning cooperative learning involves placing students in small
groups to facilitate working toward common goals (Johnson & Johnson, 1992;
Nastasi & Clements, 1991). Cooperative work involves an activity where,
through division of labor, each participant is responsible for a portion of the
problem solving (Teasley & Roschelle, 1993).
15


Generative learning activities require students to work with, manipulate, and
change information, relate information to existing knowledge structures, and use
information to support problem solving (Dunlap & Grabinger, 1996).
Generative learning theory is based on the assumption that the learner is an
active participant in the learning process where comprehension and
understanding result from the processes of generating relations both among and
between experience or prior learning and new information (Wittrock, 1992,
p. 532).
Ill-structured knowledge domains knowledge domains in which many
concepts (interacting contextually) are pertinent in the typical case of knowledge
acquisition, and their patterns of combination are inconsistent across case
applications of the same nominal type (Spiro et al., 1988 p. 375). Examples
include history, biomedicine, and literary interpretation (Jacobson & Spiro, 1994).
Inert knowledge is knowledge that can usually be recalled when people are
explicitly asked to do so but that is not used spontaneously in problem-solving
contexts even though it is relevant (CTG, 1992, p. 136). Inert knowledge is
passively learned by students and available for tests, but not available for
application in new situations (Bransford et al., 1989).
Learning with understanding Wittrock (1974a) writes: Learning with
understanding, which is defined by long-term memory plus transfer to
16


conceptually related problems, is a process of generating semantic and distinctive
idiosyncratic associations between stimuli and stored information (p. 89).
Mindfulness the voluntary, controlled employment of nonautomatic
processing operations....It is manifested in attention to details, in the careful
examination of a problems given conditions, in consideration of alternatives, in
the generation of hypotheses and inferences, in the reading of text for deeper
meanings, in linking new information to remote knowledge structures, in
processes of abstraction and decontextualization, and the like (Salomon, 1985,
P- 213).
Reductive bias the tendency to oversimplify important aspects of complexity
resulting in misconceptions of advanced material. Reductive bias leaves learners
without an appropriate cognitive repertoire for the processing of complexity
(Spiro et al., 1988). Spiro and his colleagues identify the following forms of bias:
(a) oversimplification of complex and irregular structure, (b) overreliance on a
single basis for mental representation, (c) overreliance on top down processing,
(d) overreliance on precompiled knowledge structures, (e) context-independent
conceptual representation, (f) rigid compartmentalization of knowledge structures,
and (g) passive transmission of knowledge.
17


CHAPTER 2
REVIEW OF LITERATURE
Introduction
Since the 1930s, competitive and individualistic learning has dominated all
levels of education (Johnson & Johnson, 1992). This view of education (commonly
called the traditional model of education) assumes learning is a passive, individual
activity (Brown & Campione, 1996), where the teacher simply transmits knowledge
to students as efficiently as possible (Bednar et al., 1992). To accomplish this,
educators simplify instruction by neatly defining and prescribing subject matter
domains in order to teach only the critical attributes of the domains (Duffy & Knuth,
1990). The traditional model of education, therefore, assumes (a) that learning
involves forming simple associations based upon external reinforcement (Brown &
Campione, 1996) and (b) that learning occurs most efficiently if the excess baggage
of irrelevant content and context are eliminated (Bednar, Cunningham, Duffy, &
Perry, 1995).
Many researchers believe the traditional approach to education is obsolete
(Bednar et al., 1992; 1995; Bransford et al., 1989; Bransford & Vye, 1989; CTG,
1992; Grabinger, 1996; Perkins, 1992; Reigeluth, 1995; Resnick, 1987; 1989;
Scardamalia & Bereiter, 1994). A major criticism focuses on the assumption that
18


learning occurs devoid of context. Whitehead (1929) believed the lack of relevant
context provided by the traditional model of education produces inert knowledge.
Inert knowledge is knowledge that students possess, but not used in problem-solving
contexts, even when relevant, unless the students are specifically asked to do so
(Bransford et al., 1989; Bransford & Vye, 1989; CTG, 1992). Inert knowledge exists
as an island of information, which while retrievable, provides little value to the
learner for interpreting, modifying, or influencing performance (Hannafm, 1992).
John Dewey similarly criticized the traditional approach to education because
teachers present dry problems of little relevance to students (Phillips & Soltis, 1991).
Dewey believed information presented this way is committed to memory as static,
cold-storage knowledge and unless students have the opportunity to use this
information in meaningful problem-solving activities, the information remains
sterile (Phillips & Soltis, 1991). Removing the context, therefore, limits students
abilities to leam and apply concepts (i. e., successful learning).
Successful learning requires more than literal encoding of formal instruction in a
context-free environment (Hannafm, 1992). Successful learning occurs by engaging
in activities that encompass the concepts students are trying to leam (Bransford et al.,
1989; Brown, Collins, & Duguid, 1989; CTG, 1992; Spiro et al., 1988). Successful
learning occurs when students retain, understand, and actively use the knowledge and
skills learned Perkins goals of education in the world beyond the school walls.
Perkins (1992) defines these goals as generative knowledge; knowledge that does
19


not just sit there but functions richly in peoples lives to help them understand and
deal with the world (p. 5). Generative knowledge requires existing knowledge to be
evaluated concurrently with new knowledge and reconstructed accordingly, that
knowledge be assimilated, and that perceptions of meaning, value, and importance be
derived (Hannafm, 1992; Hannafin & Rieber, 1989). Generative knowledge
therefore, is the antithesis of inert knowledge. Thus, how can educators promote the
development of generative knowledge? What theories or instructional methods exist
to promote the development of generative knowledge?
Current cognitive theory (Bednar et al., 1992; 1995; Bransford et al., 1989;
Bransford & Vye, 1989; Brown et al., 1989; CTG, 1992; Duffy & Knuth, 1990;
Grabinger, 1996; Perkins, 1992; Jonassen, 1991; Resnick 1987; 1989; Spiro et al.,
1988) emphasizes three essential and interrelated aspects of learning: (a) learning
involves knowledge construction rather than knowledge absorption and recording,
(b) learners build on current knowledge to construct new knowledge (i.e., learning is
knowledge dependent), and (c) learning depends on the context or situation in which
it occurs. Many researchers (Bransford et al., 1989; Brown & Campione, 1996;
Grabinger, 1996; Hannafm, 1992; Jonassen, 1991; Spiro et al., 1988; Wilson, 1996)
advocate creating student-centered learning environments that embody the three
principles of learning just discussed. This chapter presents a selective review of the
research and theory of three strategies designed to capture the complexities of
learning and promote generative learning.
20


The chapter begins by examining cognitive flexibility theory (CFT), a
conceptual model for instruction that facilitates advanced knowledge acquisition
(Spiro et al., 1988) by activating generative learning processes. The chapter then
moves to a discussion of generative learning theory, which assumes that learners
actively participate in the learning process by constructing meaningful
understanding of information found in the learning environment (Wittrock, 1974a;
1974b). Next, research on collaborative learning is reviewed. The reviews of
literature provide the rationale for the development of the student-centered computer
programs used in this study. The chapter concludes by restating the research
questions and offering the hypotheses of interest.
Cognitive Flexibility Theory: Assumptions and Theory
Spiro et al. (1988) believe that schools simplify representations of knowledge to
allow students to see the main point. Simplifying the content removes the context,
which rather than simplifying the concept, results in teaching a different concept.
Spiro and his colleagues (1988) argue that case-based learning captures the
complexity of real experiences. Students work through several cases examining the
concept from different points of view in different contexts. This allows the students
to see multiple contexts in which the concept occurs and applies, and thus index its
meaning.
21


Cognitive flexibility theory is based on two elements: (a) that learning occurs
along a continuum from novice to expert and (b) that knowledge domains can be
defined by two conceptual characteristics increased complexity and increased ill-
structuredness (Jacobson & Spiro, 1994; Spiro et al., 1988; Spiro, & Jehng, 1990).
Cognitive flexibility theory posits that knowledge acquisition follows a continuum
from beginner to expert, but focuses on the intermediate or advanced learning stage.
Introductory learning typically involves learners with very little transferable
knowledge in a content area (Jonassen & Grabinger, 1993). The goal of instruction
for beginning learners involves providing a general orientation and exposure to
content followed by objective assessment measuring recognition and recall of facts
(Spiro et al., 1988). In contrast, during the intermediate stage of learning, learners
acquire advanced knowledge in order to solve complex domain-oriented problems
(Jonassen & Grabinger, 1993). During the advanced learning phase the learning
goals shift to (a) mastery of content complexity (i.e., acquisition of the conceptual
complexity necessary for understanding important concepts) and (b) knowledge
applicability (i.e., the ability to adaptively transfer knowledge to novel and realistic
situations) (Spiro et al., 1989). Advanced knowledge acquisition, therefore, involves
attaining a deeper, richer understanding of content material, as well as the ability to
intelligently reason with and apply it in diverse contexts (Jacobson & Spiro, 1994;
Spiro et al., 1988).
22


The second distinguishing element of cognitive flexibility theory involves the
conceptual characteristics of increased domain complexity and increased ill-
structuredness (Jacobson & Spiro, 1994; Jonassen et al., 1992; Spiro et al., 1987;
1988; 1989; Spiro & Jehng, 1990). Complexity occurs when the domain content
places unusual demands on cognition as compared to the cognitive loading in the
introductory phase of learning (Feltovich et al., 1989). Complexity can result from
(a) task multidimensionality, which places unusual or excessive demands on working
memory, (b) the abstractness of the content or the semantic difference between
concepts and their formal symbolic representation, (c) heavy reliance on the learner's
prior knowledge (which may be faulty or at odds with the concepts), or (d) irregular,
inconsistent concepts or the interaction of many concepts (Feltovich et al., 1989;
Jonassen et al., 1992).
Related to domain complexity is the notion of ill-structuredness (Jacobson &
Spiro, 1994; Jonassen et al., 1992; Spiro et al., 1987; 1988; 1989; Spiro & Jehng,
1990). By ill-structuredness we mean that many concepts (interacting contextually)
are pertinent in the typical case of knowledge acquisition, and that their patterns of
combination are inconsistent across case applications of the same nominal type
(Spiro et al., 1988, p. 375). Characteristics of an ill-structured knowledge domain
include the following:
no general rules or defining characteristics exist that cover most cases,
inconsistent hierarchies of relationships exist among cases,
23


the same features of cases assume different levels of importance in
different contexts,
no prototypic cases exist or they are misleading, and
interactions among features cause case novelty.
(Spiro et al., 1987; Jonassen et al., 1992).
Ill-structured knowledge domains place demands on learners that are at odds with the
cognitive modes and instructional practices appropriate for introductory learning
(Feltovich et al., 1989).
The notion that advanced learning differs from introductory learning with
respect to goals, complexity, and ill-structuredness suggests different approaches to
learning are necessary. However, all too often the same teaching methods and tactics
used for initial knowledge acquisition are also used for intermediate learning.
Learning and instruction for mastery of complexity and application in a
complex and ill-structured domain cannot be compartmentalized, linear,
uniperspectival, neatly hierarchical, simply analogical, or rigidly prepacked.
Yet it much too often is, and the result is the development of widespread and
serious misconceptions and difficulties in knowledge application. (Spiro &
Jehng, 1990, p. 168)
Teaching methods geared toward introductory learning lead to oversimplification
(reductive bias) of complex material (Spiro et al., 1988) which contributes
significantly to examples of learning failure (Spiro et al., 1988; Jacobson & Spiro,
1995). Cognitive flexibility theory offers several principles designed to overcome
24


reductive bias and promote the goals of advanced knowledge acquisition (Spiro et al.,
1988):
1. Avoid oversimplification and overregulation. Advanced knowledge
acquisition requires that learners be made aware that knowledge is not simple
and orderly, but complex and irregular. It is important to lay bare the
limitations of initial, first pass understandings, to highlight exceptions, to
show how the superficially similar is dissimilar and superficial unities are
broken (Spiro et al., 1988, p. 377). Additionally, rather than decomposing
and reassembling information, instruction should reflect the intricate patterns
of component interactions.
2. Provide multiple representations of content. Single depictions of complex
and ill-structured knowledge will miss or misrepresent important aspects of
complex concepts. Cognitive flexibility requires the use of multiple themes,
concepts, analogies, and points of view (Jacobson & Spiro, 1995) to provide a
diversified repertoire of ways of thinking about a conceptual topic
(Spiro et al., 1988).
3. Use and emphasize cases. In complex, ill-structured knowledge domains,
great variability may exist between cases concerning the applicability of
relevant concepts. Cases may not be linked together by general principles,
therefore, multiple cases are necessary to illustrate abstract concepts
associated with ill-structured knowledge domains. The more variegated
25


these cases are, the broader the conceptual basis that they are likely to
support (Jonassen et al., 1992, p. 312).
4. Emphasize conceptual knowledge as knowledge in use. In an ill-structured
knowledge domain the way a concept is used or applied may vary greatly
across cases, which makes it more difficult to extrapolate the concepts from
the features of the cases. Therefore, if a concept's meaning cannot be
determined universally across cases, then one must pay attention to the details
of how the concept is used knowledge in practice, rather than in the
abstract (Spiro et al., 1988, p. 380).
5. Emphasize knowledge construction rather than knowledge transmission.
Given the emphasis placed on case-based learning and knowledge in use,
learners must be able to assemble (construct) meaningful knowledge
representations in order to adaptively fit the situation at hand. In other words,
the complexity and irregularity inherent in ill-structured domains requires
flexible, recombinable knowledge structures. Thus the storage of fixed
knowledge is devalued in favor of the mobilization of potential knowledge
(Spiro et al., 1987, p. 181). Knowledge construction implies active learner
involvement in knowledge acquisition accompanied by expert guidance and
cognitive support.
6. Emphasize noncompartmentalization of cases and concepts. The
complexity and irregularity inherent in the cases and examples in ill-structured
26


knowledge domains precludes compartmentalizing knowledge. Rather than
mapping knowledge onto the learner, the learner must map contexts onto
his/her own knowledge as it is being acquired in order to support the
transferability of knowledge (Jonassen et al., 1992, p. 313). Multiple
interconnectedness and differing thematic perspectives among different cases
and concepts enable the situation-dependent and adaptive schema assembly
that promote knowledge transfer (Spiro et al., 1988).
Cognitive flexibility theory prescribes using a case-based instructional approach
that provides access to thematic information structures to create learning
environments that stimulate the development and application of flexible knowledge
structures. Such environments stimulate the mindful processing of information
(Salomon, 1985) necessary for generative learning to occur. In sum, by focusing on
students generating flexible, usable knowledge in an information-rich learning
environment, cognitive flexibility theory is a generative activity.
Research on Cognitive Flexibility Theory
Evidence validating the effectiveness of CFT is generally positive in the area of
higher level thinking skills. For example, Spiro et al. (1987) conducted two studies
with high school students in which control groups studied the same cases as the
experimental groups, but the cases were presented linearly, while the cases for the
27


experimental groups contained case-to-case linkages. Control group learners
outscored experimental group learners on reproductive tests of memory,
but the experimental groups outscored the control groups on six different tests of
application and transfer. Spiro and his colleagues (1987) concluded that conventional
methods produce better results on traditional tests stressing rote memorization, but if
the goal of education is generative knowledge, then it would seem that methods like
ours are far preferable to the conventional ones (p. 191). Hartman and Spiro (1989)
expanded on this research effort by studying the effects of multiple perspectives,
flexible representation, and assessment of transfer. They found no differences in
reproductive memory, but in knowledge application and transfer, the flexibility group
significantly outperformed the control group. Finally, Jacobson (1990) compared a
linear, computer-based drill treatment with an experimental treatment emphasizing
multiple representations, linking abstract ideas to case examples, and the
interrelationships between surface and structural knowledge. Again the drill group
recalled more facts, but the experimental group attained higher scores on all transfer
tests of knowledge.
More recent investigations concentrate on expanding our understanding of
cognitive flexibility theory and computer-based microworlds. For example, Jacobson
& Spiro (1995) investigated the effects of learning from a minimal hypertext/drill
condition versus a cognitive flexibility hypertext. The study consisted of two main
parts. The first part was a reading stage where both groups read the same
28


instructional content. The second stage consisted of a study stage, in which the
control group completed a computer-based drill program based on the facts and
concepts of the reading stage, while the experimental group received a hypertext
treatment stressing knowledge interrelationships and knowledge assembly. The
results revealed that the control group performed higher on measures of memory for
factual knowledge, while the experimental group demonstrated greater knowledge
transfer.
These research efforts appear to validate the efficacy of cognitive flexibility
theory; instruction based upon CFT principles promotes development of certain types
of generative knowledge. The studies demonstrate that CFT-based learning
environments require students to manipulate and relate information to existing
knowledge structures which supports the problem-solving and application processes
endemic to advanced knowledge acquisition. However, opportunities for deep
learning are not always fully taken advantage of, even when presented as unique
learning environments (Salomon, 1985), such as a cognitive flexibility learning
environment. In other words, although a learning environment may be based upon
cognitive flexibility theory (or some other student-centered strategy) students may not
employ the mindful processes required of generative learning. Given the necessity to
activate mindful processes for generative learning to occur, the question becomes:
Would the integration of compatible generative learning activities into a CFT-based
learning environment promote deeper processing of material, and hence, increased
29


knowledge acquisition by advanced learners? The next section of this chapter
discusses generative learning theory and collaborative learning, two generative
activities that, in combination with cognitive flexibility theory, should increase
learning by advanced learners.
Generative Learning Theory
Generative learning theory provides an understanding of the process of
comprehension (Wittrock, 1992). Comprehension requires learners to formulate
connections between the different parts of the information being perceived and
between that information and what exists in memory (Wittrock, 1985). This causes
learners to reorganize, elaborate, and/or reconceptualize information rather than
stuff more information into memory which results in meaningful learning and
comprehension (Grabowski, 1996). The important pedagogical point is that
knowledge cannot be transferred directly from the teacher to students, but has to be
created within each student (Harlen & Osborne, 1985). Learning with understanding,
therefore, requires intellectual effort on the part of the learner to generate
relationships between stimuli and stored information (Osborne & Wittrock, 1985).
Wittrock (1992) sums up generative learning theory in the following manner:
30


At the essence of this functional model are generative learning processes
people use actively and dynamically to (a) selectively attend to events and
(b) generate meaning for events by constructing relations between new or
incoming information and previously acquired information, conceptions, and
background knowledge. These active and dynamic generations lead to
reorganizations and reconceptualiztions and to elaborations and relations that
increase understanding, (p. 532)
Wittrock (1990) believes that there are two categories of activities that stimulate
generative processes. The first category, designed to stimulate processing between
the different parts of the information being received, includes activities such as
composing titles and headings, writing questions, stating objectives, writing
summaries, drawing graphs and tables, and constructing main ideas. The second
category consists of demonstrations, metaphors, analogies, examples, problem
solving, explanations, paraphrases, and inferences which generate relationships
between the external stimulus (instruction) and prior knowledge. These categories
can be used in either a teacher-provided or learner-generated format (Grabowski,
1996; Wittrock, 1990). For example, the teacher can provide summaries or ask the
learners to summarize. However, Wittrock (1974a) cautions that understanding
cannot be given directly to students; if understanding is to occur, students must make
the connections themselves. Di Vesta (1989) concurs:
What is learned is not necessarily that which is stated in the title of the course,
what is described in the curriculum or syllabus, or the behavior described in a
behavioral objective or the content of the course defined by texts,
assignments, delivery system, and curricular materials. What is learned
depends on processing requirements that are actually carried out by the
learner, (p. 55)
31


In summary, generative learning theory posits that learners put forth intellectual
effort to actively construct knowledge by generating relationships (a) between the
different parts of the incoming information and (b) between the new information and
prior knowledge and other memory components (Grabowski, 1996; Wittrock, 1974a;
1974b; 1985; 1990; 1992). Unfortunately, most of the research on generative
learning theory concentrates on reading comprehension and individual students.
Fortunately, the model emphasizes transforming static information (text) into flexible,
usable knowledge (Grabinger, 1996), thus there may be transferability to other
student-centered strategies designed to increase learning.
Compatibility of Cognitive Flexibility Theory and Generative Learning Theory
Cognitive flexibility theory advocates constructing/applying knowledge in a
multiple case format to encourage development of flexible representations of content
(Spiro et al., 1988). Learners manipulate the content information to make the
multiple connections between and among the information and prior learning. Thus
cognitive flexibility theory is a generative activity and conceptually, very similar to
generative learning theory. However, the two models differ with respect to practice.
First, there are obvious contextual differences between the two theories. Cognitive
flexibility theory advocates using multiple cases in a complex learning environment
to increase learning, whereas, generative learning theory prescribes specific strategies
designed to increase student reading comprehension and understanding of text.
32


Second, Spiro and his colleagues specifically target the advanced stage of learning for
CFT application, while generative learning theory application appears unrestricted
regarding the stages of learning. Given the conceptual similarities, but practical
differences, how might cognitive flexibility theory and generative learning theory
interact to affect learning?
The limited cognitive flexibility theory research discussed earlier demonstrates
the effectiveness of cognitive flexibility theory in promoting advanced knowledge
acquisition. Since generative learning theory offers strategies designed to make the
learning processes explicit (i.e., stimulates mindful processes) and increase
comprehension and understanding, it is reasonable to expect that generative learning
strategies would enrich a cognitive flexibility learning environment. In other words,
using strategies to stimulate one or both of the generative processes should help
advanced learners develop the rich knowledge structures cognitive flexibility strives
to promote. The strategies of interest in this study are summarization, which
facilitates making organizational connections between the different parts of the new
information, and explanation, which supports making connections between prior
knowledge and the new information.
33


Individual Student Focus
One interesting observation concerning the theories of learning just
reviewed especially cognitive flexibility theory is the focus on the individual
student. Given the proclivity to use computers to deliver CFT-based instruction, the
focus on students interacting individually with computers to learn the respective
subject domains is especially intriguing since much, if not most, research on
computer-based learning focuses on collaborative learning. The concentration on the
individual learner within the cognitive flexibility theory literature may reflect the
initial empirical evaluation of cognitive flexibility theory, or the assumption that
advanced learners (the stage of learning hypothesized to benefit from CFT-based
instruction) learn best individually, or perhaps it is based on the belief that computers
represent the ideal medium to individualize instruction. Whatever the reason, the
reality is that students often work in small groups at computers (Hooper, 1992).
Therefore, an investigation into the effects of collaborative learning on cognitive
flexibility theory is warranted, especially since social interaction is such an integral
part of a student-centered learning environment (Resnick, 1987; Wilson, 1996).
Collaborative Learning
At the outset it is important to make the distinction between the terms
cooperative learning and collaborative learning because they differ with respect
to how they encourage student interaction. Cooperative learning refers to group
34


learning approaches where a division of labor among participants requires individual
students to take responsibility for different portions of the task (Crook, 1994; Damon
& Phelps, 1989; Stodolsky, 1984; Teasley & Roschelle, 1993). Collaborative
learning, on the other hand, involves students working jointly in a coordinated effort
on the same problem (Damon & Phelps, 1989; Teasley & Roschelle, 1993). In both
cases, the grouped students share a common goal, interact, and contribute to the group
activity (Stodolsky, 1984), but the differences in task structure result in differences in
equality and mutuality (two indexes of peer engagement). Equality means that both
parties...take direction from one another rather than one party submitting to a
unilateral flow of direction from the other; and mutuality means that the discourse in
the engagement is extensive, intimate, and connected (Damon & Phelps, 1989,
p. 10). According to Damon and Phelps (1989), cooperative learning is high on
equality, but variable on mutuality due to the task subdivision that requires learners to
accomplish much of their work individually. In contrast, students work jointly on the
same problem in a collaborative environment creating an engagement rich in mutual
discovery, reciprocal feedback, and frequent sharing of ideas (Damon & Phelps,
1989, p. 13). The higher degrees of mutuality and equality present in a collaborative
environment results in interaction between (not summation of) the understandings of
the students which leads to emergent knowledge (Whipple, 1987).
Collaborative learning is rooted in the assumption that knowledge is socially
constructed and that social interaction is necessary for learning to occur (Vygotsky,
35


1978). Two different theoretical perspectives offer possible explanations for the
cognitive growth associated with collaborative learning. The first, socio-cognitive
theory, posits that interindividual conflict facilitates cognitive growth (Nastasi &
Clements, 1992; 1993). Development occurs as students enter into a conversational
process of negotiation and justification in an attempt to resolve differences of
agreement (Crook, 1994; Nastasi & Clements, 1992; 1993). As the students
reevaluate their positions and seek new information to resolve conflict, they learn the
material better (Webb, 1987) and restructure old knowledge structures into new
knowledge structures (Hanafin, 1992). Co-construction is the second theoretical
explanation for improved achievement. In this case interpersonal conflict is absent
and partners integrate their differing task conceptualizations into a mutual plan for
solving a problem neither could solve alone (Nastasi & Clements, 1993). Thus
collaborating students scaffold or guide and correct each other and build on each
other's ideas until they reach the solution. In either case, the process of collaborating
requires students to make their thinking public and explicit (Crook, 1994), which
facilitates intellectual growth through a process of declaring and justifying ideas,
opinions, and interpretations (Damon & Phelps, 1989). Collaborating students,
therefore, retrieve prior knowledge, seek new information, evaluate their own and
others' answers, ideas, and opinions, confront their own misunderstandings and lack
of knowledge, and as a consequence, restructure their thinking (Webb & Lewis,
1988, p. 181) which leads to higher levels of understanding.
36


Research on Collaborative Leamine
Literally hundreds of studies and research reviews exist and most demonstrate
the value of collaborative learning. For example, meta-analyses comparing the
effects of small group and individual learning environments on achievement indicate
effect sizes of .75 (Johnson, Maruyama, Johnson, Nelson, & Skon, 1981) and .63
(Johnson & Johnson, 1989) favoring small group learning. In addition to improved
learning, numerous other studies demonstrate that collaborative learning positively
benefits the social-emotional (positive relationships and psychological health)
aspects of instruction (Johnson & Johnson, 1992).
The cognitive and social benefits documented for traditional classroom use of
collaborative learning appear to transfer to computer-based learning environments as
well. Some studies demonstrate significantly higher achievement for collaborative
groups (Dalton et al., 1989; Hooper et al., 1993; Johnson et al., 1985; 1986), while
others find no significant differences between individuals and small groups (Carrier
& Sales, 1987; Makuch et al., 1992; Webb, 1985). Webb (1987) reviewed 14 studies
comparing achievement between group and individual computer work and found
similar results; only five studies found differences favoring the group activity.
However, Webb concludes:
The important result is that no study found greater learning among students
working alone than students working in groups. This suggests that, averaging
over all students, group work is not detrimental to students learning, and may
be beneficial, (p. 195)
37


For example, groups spend more time interacting on task (Johnson et al., 1985) and
express more positive attitudes toward working at computers (Hooper et al., 1993).
Although variable, these results mirror the positive results found for collaborative
learning in the non-computer classroom. Thus, computer-based collaborative
learning appears to be an enriching experience and may increase achievement.
Given the volume of studies investigating the effects of computer-based
collaborative learning, as well as the large number of factors involved (age,
experience with computers, the computer activity, achievement measures, etc.) it is
no wonder that results vary. This disparity of results notwithstanding, the general
consensus is that the collaborative use of classroom computers is the preferable
implementation strategy (Watson, 1990). One reason for grouping students at
computers involves solving logistical problems (Hooper, 1992); in many instances,
there simply are more students than computers. A second and more educationally
sound reason for grouping students, rests on the assumption that learning occurs in a
social context where learners interact and draw on one another's expertise (Krajcik,
Blumenfeld, Marx, & Soloway, 1994). In the latter case, grouping students at the
computer is based on the roles that communication and interaction play in creating
meaning, while in the former case, grouping students may be based solely on
classroom expediency. In both cases, however, teachers expect some educational
benefit which presumably results from the quality of communications and interactions
that occur between the grouped students.
38


In summary, considerable evidence exists demonstrating the efficacy of
collaborative learning, both in a computer-based learning environment and the non-
computer classroom. The benefits of collaborative learning result from the active
engagement of students in an equal and mutual discourse of conversation and
communication where learners make connections between incoming information and
between the incoming information and prior knowledge. Clearly collaborative
learning is a generative activity.
Despite the considerable literature devoted to computer-based collaborative
learning, most of this research is conducted with courseware designed for individual
use (Jonassen, 1988). Additionally, computer-based learning is dominated by
strategies that facilitate only superficial levels of processing (Hooper & Hannafin,
1991). These types of software may be inappropriate for group use (Hooper, 1992)
and may explain the variability of results found for computer-based collaborative
learning. Several researchers (Hooper, 1992; King, 1989; Nastasi & Clements, 1992;
Webb, 1987) call for research efforts to investigate how to facilitate and strengthen
the use of effective collaborative processes. It is my contention that combining
strategies designed to engage students in generative learning activities should
facilitate student interaction and increase achievement.
39


Research Questions
Several themes emerge from the literature review just concluded. First, although
limited at this point, research supports the central assertion of cognitive flexibility
theory that case-based learning produces greater knowledge acquisition and transfer
than a linear, computer-based approach to learning the same material. Could student
achievement be improved by modifying a CFT-based computer program with
additional generative strategies? Additionally, the cognitive flexibility theory
literature only evaluates students working individually in the CFT-based
environment. What effect, if any, would pairing students have on the effectiveness of
a CFT-based computer micro world?
Second, research demonstrates that generative learning strategies are effective
for increasing comprehension (Di Vesta & Peverley, 1984; Doctorow, Wittrock, &
Marks, 1978; Grabowski, 1996; Johnsey, Morrison, & Ross, 1992; Linden &
Wittrock, 1981; Stein & Bransford, 1979; Wittrock, 1990; 1992; Wittrock &
Alesandrini, 1990). However, most generative learning studies focus on learners
working alone. Given the evidence demonstrating the effectiveness of collaborative
learning, might collaborating pairs generate more effective explanations and
summaries and therefore, reach higher levels of knowledge acquisition than single
students?
40


Finally, a common theme running through the vast literature on computer-based
group work is the intense focus on children.
Energetic computer-based group work has been most thoroughly documented
within the earliest years of education..., and in the middle and secondary
years....There is very little relevant research on this topic that considers further
and higher education or work in training communities. (Crook, 1994, p. 124)
Given the relative paucity of research regarding adult-oriented, computer-based
collaborative learning, is collaborative learning effective for advanced learners?
Also, can the collaborative process, and hence achievement, be improved by
integrating the opportunity for collaboration with other generative activities, such as
specific generative learning strategies or CFT-based computer learning
environments? This study was designed to address these questions by assigning
single and paired students to work on two qualitatively different CFT-based computer
programs (a base program and a program enriched with additional generative learning
strategies).
Research Hypotheses
The major research question involved investigating the effects of two generative
learning activities (generative learning theory and collaborative learning) on the
performance of advanced learners in a cognitive flexibility-based computer
microworld. Two important questions emerge concerning the integration of cognitive
flexibility theory, generative learning theory, and collaborative learning:
41


(1) Would the inclusion of embedded explanation and summary cues
(generative learning strategies) result in differences in learning in a
CFT-based computer microworld?
(2) How would the addition of collaborative learning affect learning in this
environment?
Based on the literature, it was hypothesized that:
(1) Students assigned to the generative program would out perform students
assigned to the base program on the posttest.
(2) Students assigned to the collaborative condition would out perform
students assigned to the individual condition on the posttest.
(3) There would be an interaction effect between the version of program and
method of instruction treatment variables as measured by scores on the
posttest. A synergy would develop between the two treatment variables
such that students assigned to the pair/generative treatment would perform
significantly better on the posttest relative to the other treatment groups.
42


CHAPTER 3
METHODOLOGY
Introduction
This study used a computer-based learning program to provide instruction in the
complex domain of transfusion medicine. This program integrated multiple cases
with access to relevant information sources to encourage medical students to
diagnose, assess, and manage infectious and non-infectious adverse events of
recipients, and donor-related blood transfusion problems. The present study
investigated the relative effects of complementary generative activities (generative
learning strategies and collaboration) on the knowledge acquired by second-year
medical students using a cognitive flexibility theory-based computer microworld.
The remainder of this chapter discusses the materials, participants, research design,
instrumentation, and instructional procedures used in this study.
Materials
Cognitive flexibility theory addresses knowledge acquisition in complex and ill-
defined subject domains. The broad field of medicine is considered such a domain, as
are the specialty fields within medicine (Spiro et al., 1988). Transfusion medicine is:
43


That area of medicine which includes the collection, production, storage and
administration of lymphohematopoietic progenitors, mature blood cells,
plasma and plasma components, as well as associated services to be used in
treatment of patients with specific diseases. (D. Ambruso, personal
communication, June 15, 1998)
Transfusion medicine is a multidisciplinary area encompassing a number of basic
sciences (e.g., biochemistry, cell biology, transplant biology, and immunology),
clinical sciences (e.g., hematology, medicine, surgery), and some applied sciences
(D. Ambruso, personal communication, June 15,1998). Due to the large number of
interrelated medical and science domains, no prototypic cases exist. Causes and
effects interact and produce a variety of symptoms or require a number of different
medical interventions that depend on the clinical status of the patient, the purpose of
the transfusion, and from whose perspective the case is viewed (Jonassen et al.,
1992). Because of the complexity and ill-structuredness of this knowledge domain, a
medical school professor responsible for teaching this curriculum, led a team of
physicians, educational consultants, and instructional designers (Transfusion
Medicine Team) to develop Handling Transfusion Hazards, a computer program
based upon cognitive flexibility theory principles.
The Program: Handling Transfusion Hazards
Diagnosing and treating transfusion-related problems is a dynamic, problem-
solving activity. Clinicians collect information, order and interpret lab tests, and
assess and manage clinical problems in their patients, including defining the
44


indication for use of blood transfusions. Handling Transfusion Hazards instantiates
this process by providing medical students with realistic cases to solve, as well as the
tools necessary to solve the cases.
The program consists of four distinct parts: (a) program introduction,
(b) pretest, (c) learning phase, and (d) posttest (see Instruments this chapter for
descriptions of the pretest and posttest). The introduction provides information to
help the students successfully complete the program. This orientation (a) discusses
the program content and learning objectives, (b) provides a concept map (i.e.,
diagram) that illustrates and explains each part of the program, (c) and informs the
students of the requirement to complete the program in its entirety by sequencing
through the pretest, practice cases, and posttest.
The learning phase contains six practice cases (scenarios) that require the
subjects to order laboratory tests (if appropriate) to help them form a diagnosis and
then select suitable assessment and management options. The cases require the
students to access relevant factual and conceptual information which is then applied
to solve the transfusion problems at hand. Each case contains (a) a description of the
case, (b) a research section, (c) an actions section, and (d) a help section (Figure 3.1
presents a sample case screen). These sections are described below.
45


Figure 3.1. Sample Case Screen from Practice Case Three
| PRACTICE CASES: Introduction
............S* "h------------------------------
The Case of Fainting Freddie
t s Saturday morning, July 5, and you receive a frantic
page from the drawing center at your hospital blood bank.
Yoix hospital routinely draws yolunteerblood donors for
processing by the local community blood center and the
Intern on duty provides medical coverage.
A donor has Just fainted and your he£ is required
immediately. You breathlessly approach the donor room
to find a pale, swealy donor lying on a cot.
Research:
[ HidorylPhysMl )
[ Ptrepectjyw )
c stwc y -
Actions:
[ OrderLabTeats ^
[ AsaeaaCaae ^
f Manage Caaa ^
f CasesMenu]
B3 o
Map Textbook Quit
Case summary: Upon selecting a case, students are presented with the
case summary which provides basic contextual information (see Fig. 3.1).
Research: The research section contains thematic information sources that
provide structural information relevant to the case.
46


History/physical: This section provides the basic patient
background information necessary for the learners to begin the
diagnostic process. Figure 3.2 presents a sample history/physical
screen.
Figure 3.2. Sample History Screen from Practice Case Three
PRACTICE CASES: History
Research:
[ Physical Exam
Actions:
Return
B3
Map TtxtbooJc Ouit
47


Perspectives: Each case contains opinions from several significant
operatives, such as the attending physician, resident, phlebotomist,
blood bank director, surgeon, pediatrician, internist, fellow, donor,
recipient, gastroenterologist, and patient. Usually four or five
perspectives are available based on the relevance of the
information to the case content. Figure 3.3 presents a sample
perspectives screen.
Figure 3.3. Sample Perspectives Screen from Practice Case Three
PRACTICE CASES: Perspective of Phlebotomist
. T-:
This donor had a normal Interview.
He is a first time do norand came to the blood bank this
morning because his mother had an operation a week ago
and he wanted to he£ replace the blood she had used.
He appeared nervous and said he has been afraid of
needles since he was a child. We did a normal draw and l
went well, but when he 9c* up from donating, he said he felt
weak, collapsed and lost consciousness He regained
consciousness alter two or three minutes and then seemed
fine. He didn't have any problems breathing or anything
else.
Research: >
; t f P rn e b or onus*
' an ana Physician}
'. ; ( Patient )
^BjoodBenkDjrectoJ
Actions:
Return
dS O
Map Textbook Quit
48


Similar cases: The program contains a bank of 24 cases that are
less detailed than the primary learning cases. These cases provide
information pertinent to the selected practice case. The similar
cases menu provides two choices: closely related similar cases and
other suggested similar cases. Closely related similar cases share
the following attributes with the practice case: group involved,
type of risk, pathophysiology, symptoms, and screening tests;
whereas, the other suggested similar cases share only one or two of
the attributes above. Figure 3.4 shows a sample similar cases
screen.
Figure 3.4. Sample Similar Cases screen from Practice Case Three
jS imitar Cases: Transient Seizure Introduction
r ^
The Case Of The Transient Seizure
N.G. is a 56-year-old white female autologous blood donor
who sustained a severe reaction during donation
Research:
Case Introduction
f Case Hotory ^
[ Physical Exam j
c
Lab Data
D
C
Comment
T
Actions:
. ( NertCase
r Return _J

6§& Map B3 Textbook o Quit
49


Help: The help section provides three choices: textbook, quit, and
map.
Textbook: The program contains a transfusion medicine textbook
that provides detailed information on donor and recipient
transfusion issues, as well as access to a glossary of medical
terminology. Figure 3.5 presents the textbook index screen.
Figure 3.5. Textbook Index Screen
TEXTBOOK: Index

Donor Issues : Recipient Issues
DONOR PROCESSING RECIPIENT PROCESSING
Assessment "Look-tack"
Registration \\ REO PI ENT RISKS
I Requirements for donation Infectious
Donor protection .Acute waf hepatitis
Recipient protection Hepatitis A
Otherrestpct/ons Hepatitis B
Mandatorvtests Hepatitis C
Post'phtebotomvcare Other causes ofhepatitis
DONOR RISKS Surrogate Testing
| General reactions to donation Retroviruses
! MSdreactions HY-t
Moderate reactions HtY-2
Severe reactions HTL n
Otherreacdons htl m
Nature of vofunteer Svphths
.Autofogous donations HemoMic Transfusion Reactions
Directed donations .Acute HTR DeiayedHTR tmmunotogKnsks :: GLOSS/RYOF TERMS

Click right
on the index
topic y.ha\.
you wish to
see
Actions:
f Return
Map
Quit: The quit function allows the students to exit the program. If
selected this option saves the students work up to, but not
including the current case.
50


Map: This function takes the students to the conceptual map
contained in the program orientation.
The research resources and textbook contain the declarative (factual) and
conceptual knowledge required to support the students problem-solving processes; as
such, they remain available throughout the diagnosis and treatment phases (labs,
assessment, and management) which are described below.
Actions: Each case requires the students to order lab tests (if appropriate),
assess, and manage a transfusion medicine problem.
Laboratory tests: Students have the option of ordering many
different types of laboratory tests to help them develop and
confirm their diagnosis. Upon selecting the lab test portion of the
program, students are given the option to choose from five
different types of lab tests: chemistry, hematology/coagulation,
blood bank/serology, microbiology/urinalysis, and diagnostic
procedures/radiology. Each of these test types contain between 4
and 20 tests and procedures. Upon selecting a test, students
receive immediate test results (a test value) and feedback
concerning the appropriateness of the test. After ordering all of the
tests for a given type (e.g., chemistry), the students have the option
to view a list of suggested tests.
51


Assessment and Management: Assessment and management
actions proceed in the same fashion. Students choose their
courses of action and receive immediate feedback concerning the
advisability of each action taken. Then they have the option to
view the correct actions. After the students complete the
management actions, each case ends with a summary of the
learning outcomes specific to that case.
Handling Transfusion Hazards facilitates knowledge acquisition in the
complex domain of transfusion medicine by giving students practice solving actual
clinical problems. This program achieves its objectives by combining case-based
instruction with the structural support of thematic information resources (patients
history/physical, a transfusion medicine textbook, perspectives of several case-
relevant operatives, and other prototypically similar cases). Specifically, this
program:
1. avoids oversimplifying instruction since no single case is prototypic,
2. provides multiple representations through the varied nature of the cases
and different sources of information available for processing the cases,
3. supports context-dependent knowledge acquisition by providing real life
cases, as well as realistic information sources, and
4. emphasizes knowledge construction through the analytical processes built
into the program. Students must access different sources of declarative
52


and conceptual knowledge to solve the cases. In the process the students
build complex schemata consisting of procedural (how to) knowledge
rather than an assortment of unrelated facts (Jonassen et al., 1992).
In summary, Handling Transfusion Hazards fosters contextualized reasoning
(Resnick, 1987). The program provides medical students with the opportunity to
scientifically reason and solve real-world medical problems just as practicing
clinicians would, and therefore, is clearly a generative activity (see Appendix A for a
copy of a practice case).
Program Development and Evaluation
An extensive development and formative evaluation process was used to develop
Handling Transfusion Hazards. This process took almost two years and is detailed
below.
First, the content topics in transfusion medicine were identified by the
physicians responsible for teaching this curriculum and organized under
the theme of risk management to donors and recipients. Selection of these
topics was guided by learning objectives established for the transfusion
medicine curriculum.
Second, this information was organized into a database by the Transfusion
Medicine Team according to adverse events (for example, Hepatitis A,
Hepatitis B, HIV, etc.). Cognitive flexibility theory was selected as the
53


theoretical framework for presenting this information in a computerized,
case-based format for two reasons: (a) the learning objectives involve
application and transfer of information and (b) cognitive flexibility theory
provides the means to create an authentic environment that promotes
critical thinking and problem solving (additional educational outcomes the
Transfusion Medicine Team wished to facilitate).
Third, the practice and test cases were developed based upon actual
clinical cases. Issues considered at this point included the number of cases
to use, the appropriate resources (textbook, etc.) necessary for the students
to solve the cases, and the relationships between the cases.
Fourth, a complete working program was developed leading to formative
evaluation of the program by users (medical students, residents, and
fellows) and expert review of content. Formative evaluation was
conducted on individual practice cases and the total program to check for
proper operation of the program. Content review was conducted by two
transfusion medicine experts (not connected with the project) who
checked the content and operation of the program for accuracy and
completeness.
Finally, changes suggested by the expert reviewers were incorporated.
This study used two versions of this program: the version described above (base)
and a generative version. The generative program includes embedded generative
54


learning strategies (explanation and summarization cues) that require the students to
explain their decisions and to summarize the main concepts of each transfusion
medicine case. Providing explanations enables students to reorganize and clarify
their ideas which helps them recognize gaps in understanding and increase learning
(Webb & Lewis, 1988). Summarization involves paraphrasing the information to be
learned and through further encoding, helps students to consolidate and strengthen
what was learned and provide feedback concerning the degree of understanding
achieved (Hooper, 1992; Yager, et al., 1985). These strategies should help the
students make the connections between the multiple contexts in which concepts apply
leading to greater knowledge acquisition.
The generative program version requires students to justify (explain) their
decisions when ordering lab tests, and assessing and managing each case. After
explaining their actions, the students can view the results, corrective feedback, and
the correct actions just as they would in the base version. Finally, before proceeding
to the next case, the program prompts the students to summarize the main concepts
contained in the case, and to highlight any similarities or differences with other cases
(Appendix B contains examples of the generative version screens).
The base program was modified to produce the generative program version by
the author with the help of a computer programmer affiliated with the Transfusion
Medicine Team. The introductory information, instructional content, and lesson
sequencing remained the same. The only changes involved embedding the
55


explanation and summary cues and providing the screen space necessary to respond
to the cues. Since the changes encompassed how the program functions, and not
program content, the author conducted an operational evaluation to ensure the reliable
operation of the generative program.
Participants
One hundred and thirty-two second-year medical students, 70 men and 62
women, participated in this study. These subjects were enrolled in Pathophysiology
of Disease, a required course for all students attending a major medical university
located in a large western city. The subjects completed Handling Transfusion
Hazards during the three-week hematology rotation of the Pathophysiology of
Disease course. All subjects completed the program; however, thirty-one students
were excluded from the final analysis due to missing or inaccurate data and
noncompliance with study conditions (see Chapter Four, Data Collection and
Analysis Screening Procedures). Thus, one hundred and one students were included
in final statistical analysis.
Assignment to Treatment
Effective interaction stimulates mindful learning (the employment of
nonautomatic volitional and metacognitively guided processes of learning) resulting
in greater cognitive effort and deeper processing of information (Salomon, 1985;
56


Salomon & Globerson, 1987; 1988). Interdependence must be established for
students to interact effectively (Johnson & Johnson, 1992); however, simply placing
students together and asking that they work as a team may not establish the level of
interdependence required for a good collaborative effort. Salomon and Globerson
(1988) write:
A team is a social system, and as such it is a qualitatively different entity than
a few individuals working alone side-by-side. Behaviors and cognitions in the
group have two major characteristics: they become interdependent and this
interdependence develops over time in a reciprocal manner. This developing
interdependence implies that individuals cognitive processes affect and
become affected by the ones of the other team members. Communication
among the team members serves as the means for the gradually growing
interdependence of cognitions and behaviors such that efforts (or effort
avoidance) become coordinated and shared, (p. 93)
Interdependence is likely to be low during short research experiments when
subjects have little previous interpersonal knowledge or interaction with one another
(Hooper, Sales, & Rysavy, 1994; Salomon & Globerson, 1988). Low
interdependence may explain the variability of results concerning achievement in the
computer-based collaborative learning literature, as well as the unanticipated results
in the pilot study (described later in this chapter). Therefore, paired subjects picked
their partners. The assignment process proceeded as follows:
1. The students were previously assigned to one of twelve unit teaching labs
(UTLs) ranging from 8 to 12 students each, therefore, the method of
instruction variable (single or pairs) was randomly assigned to the UTLs.
One UTL was assigned to the single condition for every two UTLs
57


assigned to the paired condition, resulting in the assignment of 4 UTLs (44
students) to the single condition and 8 UTLs (88 students) to the paired
condition.
2. A program version was then randomly assigned to each individual
student or pair.
This process created four treatment groups (see Figure 3.6). Groups 1 and 2
were each comprised of 22 students working individually. These groups differed by
the version of program. Groups 3 and 4 were each comprised of 22 collaborative
pairs. Again these groups differed by the program version. The breakdown of
students and treatment groups for the final analysis consisted of 31 single students (18
students assigned to Group 1 and 13 students assigned to Group 2) and 35
collaborative pairs (20 pairs assigned to Group 3 and 15 pairs assigned Group 4).
Design
This study used a 2 X 2 factorial design (see Figure 3.6) to address the question
of how embedded generative learning strategies and collaboration affect knowledge
acquisition among advanced learners (medical students) in a CFT-based computer
microworld.
58


Figure 3.6. Experimental Design
Program Version
Instructional Method Base Generative
Single Group 1 Group 2
Pair Group 3 Group 4
The experimental design employed the pretest-posttest control group design espoused
by Campbell & Stanley (1963) and is graphically depicted below:
R O Xj 0
R O X2 O
ROX3O
R O X4 0.
Variables
Independent Variables
Two independent variables were studied (see Figure 3.6). The first independent
variable was the version of computer program. As previously discussed, a base
computer program was developed according to cognitive flexibility theory principles.
A generative version was then developed by embedding two generative learning
strategies into the base version. Creating two similar, yet qualitatively different
59


programs permitted testing the effect generative learning strategies exert on learning
in a CFT-based computer microworld. The second independent variable consisted of
the instructional method (individual versus collaborative learning), which permitted
testing the effect of collaboration on the learning among advanced learners in a
cognitive flexibility environment.
Dependent Variable
The dependent variable was the knowledge acquired (constructed) by the
subjects as measured by a posttest taken at the conclusion of the learning phase of the
computer program. The dependent measure consisted of three cases that required the
subjects to diagnose and treat transfusion related problems similar to the problems
encountered in the practice cases. Each posttest case measured the students mastery
of the facts and ability to think through and solve the transfusion medicine problems
encountered. The test cases were similar in structure to the practice cases except that
the information resources (e.g., perspectives, similar cases, and textbook) were
removed (see Figure 3.7).
60


Figure 3.7. Sample Screen from Test Case B
| TEST CASE B: Introqucdon
............ ^ ...................................-
The Case of the Dilutional Dilemma
The head nurse from the premature nursery calls you and
says she is sending a blood sample from a baby who has
had a severe hemolytic reaction while undergoing an
exchange transfusion.
You are the physician covering the transfusion medicine
consultation service and the neonatology service needs your
help.
Research: .
f HbtorytPhywcd ^
Actions:
[ OrderLabTeats )
f AaaesaCaae ^
[ ManaQeCASB ^
o
Map Quit
As discussed in Chapter One, collaborative learning studies typically require
subjects assigned to the group condition to test independently to measure the
individual knowledge acquired as a result of collaborating. Collaborative learning
is premised on the notion that two heads are better than one, where the level of
group generated knowledge is greater than the knowledge of the individual
participants (Whipple, 1987). Also, the research demonstrates that group-to-
individual transfer occurs at least to the same degree between subjects assigned to
collaborate and subjects assigned to the single condition. Therefore, in this study, the
subjects assigned to the collaborative condition tested with their partners. This
61


permitted examining individual versus socially constructed knowledge. The
instruments section provides a more detailed description of the posttest.
Instruments
Pretest and Posttest
The pretest and posttest measured the subjects understanding of the transfusion
medicine knowledge domain. The students scores on the pretest and posttest were
based on the lab tests they ordered, and how they assessed and managed three cases
conceptually similar to the cases presented during the learning phase of the program.
The point system used to score the learners performance is outlined below:
Lab tests: Each lab test ordered was scored on a scale from -1 to +2
depending on how essential it was for establishing a diagnosis for the case,
its affect on the patient, and cost considerations.
Assessment: Each action selected for assessing a case received a score of
-1 or +1 depending on the validity and appropriateness of the selection.
Management: Each management action received a score of -1 or +1
depending on the appropriateness of the selection for the case.
The three test cases contained a total of 172 lab tests, 19 assessment, and 16
management choices (see Appendix C for an example test case). There were 30 lab
test points, 9 assessment points, and 8 management points for a total of 47 points.
62


The same cases were used for the pretest and posttest (see the pilot study for
justification).
The test cases were developed in conjunction with the learning cases. Two
transfusion medicine experts reviewed the test cases for content validity,
comprehensiveness, and accuracy as part of the content review process. They were
asked to judge the match between the learning objectives, the practice cases, and the
test cases, as well as to evaluate the rationale used to develop the grading scales. The
experts were asked to arrive at consensus regarding these issues. Changes were made
to the test cases based upon the experts feedback.
Procedure
Pilot Study
The base computer program was piloted during the spring of 1997 to determine:
1. If differences in learning exist between single and paired users,
2. If medical students using a computer-based learning program would
effectively collaborate, and
3. If the test cases were parallel forms.
The participants in the pilot study were second-year medical students, one year
further advanced in medical school than the current subjects. All students
participated, resulting in a subject population of one hundred and forty-two.
63


To answer these questions the base program was modified slightly. Since the
program consisted of ten cases, the decision was made to drop one non-infectious
disease case to produce a module containing two pretest cases, five learning cases,
and two posttest cases. The pretest and posttest each contained one infectious disease
case and one blood donor case, while the learning phase contained infectious, non-
infectious, and donor cases. Two versions of the base program were developed,
differing by alternate forms of the pretest and posttest. In other words, the pretest for
program version A was the posttest for program version B and vice versa.
The pilot study was conducted between April 2, 1997 and May 22, 1997.
Pretest/posttest results indicated that the test versions were not parallel. The means
for one infectious disease test case (test case 2) were higher than the means for the
second infectious disease test case (test case 1). A series of 2 (Grouping) X 2
(Version) ANOVAs were run to determine if a lack of parallelism existed. The
ANOVAs resulted in several significant main effects and interactions by program
version for the recipient cases. The data suggest that test case 2 was easier, which
corresponded with the higher mean scores on this test.
Although the tests were not equivalent versions, it was still possible to determine
if differences in learning existed between singles and pairs. In this case two separate
ANCOVAs were run (by program version) comparing the means for singles and pairs
for program version A and comparing the means for program version B. This
analysis indicated that no significant differences existed between the single and
64


paired students using the same versions of the program. Although not anticipated,
this result does not deviate from the collaborative, computer-based learning literature,
where some studies demonstrate higher achievement for collaborative groups versus
individuals, while others find no significant differences.
Finally, to determine if program users would effectively collaborate, five
volunteer pairs were videotaped using the program. A coding scheme containing two
main interaction types (on-task and off-task) and four verbal interaction categories
(questions and statements for both interaction types) was developed as an observation
checklist. The four interaction categories were further subdivided into two
categories: (a) suggestions, opinions, directions, and (b) explanations, evaluations.
The coding scheme also contained a provision for categorizing an interaction as either
cognitive conflict or co-construction.
Verbal interactions were coded in terms of their frequency of occurrence by two
independent coders. To determine if cognitive conflict or co-construction existed, the
coders were asked to document cases of cognitive conflict; if cognitive conflict was
not present, it was assumed that co-construction existed. The coders documented the
verbal interactions for the practice and posttest phases of the program. Average inter-
coder agreement was .95. Although the sample was small, the data suggest that the
medical students remained highly task focused during the two program phases
observed. Furthermore, the interaction patterns demonstrate that the students
concentrated their efforts in giving and receiving explanations, which as previously
65


discussed, are positive collaborative behaviors. Finally, the coders failed to document
any cases of cognitive conflict, suggesting that these pairs created a co-constructive,
collaborative environment.
The results of the pilot study led to the questions of interest in the present study,
as well as the following procedural changes:
1. Creation of a second program (i.e., the generative version).
2. Since the test cases were demonstrated not to be parallel forms of the
measure, the decision was made to drop one test case and use the same
cases for both the pretest and posttest.
3. Adding the dropped test case to the bank of learning cases to provide
comprehensive coverage of the transfusion medicine domain.
4. Assigning the instructional method condition to the UTLs rather than to
each individual student.
Instructional Intervention
The Pathophysiology of Disease course was broken into three sub-courses. The
transfusion medicine curriculum was taught during the hematology block of
instruction. During a 30-minute lecture on the first day of the hematology course, the
students were introduced to the computer module. The transfusion medicine
curriculum manager discussed the program content and learning objectives, the
logistics involved to obtain a computer disk, and the requirement to complete the
66


program and grading criteria. The students were also told that they must complete the
program as assigned (i.e., if assigned to work individually, they must not consult with
any other students; and if assigned to work collaboratively, they must complete the
program only with their partner). Finally, the students were directed not to discuss
program content or operation with any other students enrolled in Pathophysiology of
Disease. Appendix D contains a copy of these instructions.
The program was loaded on six computers in the medical school's Learning
Resources Center (LRC). Three computers were designated as the primary computers
for running the program; these computers were physically separated from one
another. The other three computers containing the program were co-located and were
considered backups. Each individual student and collaborative pair were issued a
diskette that provided access to the program and recorded their actions. Due to the
medical students tight schedules and the limited number of computers, the students
were given approximately three weeks (March 23, 1998 to April 10, 1998) to
complete the program. The medical school relies heavily on the honor system,
however to monitor compliance with the curriculum managers instructions to work
individually or in pairs as assigned, the author was present while 22 single students
and 31 pairs worked through the program. After completing the program the students
turned their disks into the LRC staff for collection.
67


CHAPTER 4
RESULTS
Introduction
This study examined the relative effects of embedded generative strategies and
collaboration on knowledge acquired by advanced learners in a cognitive flexibility
theory-based computer microworld. The independent variables were program version
(base or generative) and method of instruction (single or pairs). The dependent
variable was the score achieved on a posttest consisting of three transfusion medicine
cases. Analysis of covariance (ANCOVA) was used to test the experimental
hypotheses. In addition, multivariate analysis of variance (MANOVA), analysis of
variance (ANOVA), and t-test statistical techniques were used in a secondary analysis
of the data. This chapter presents the results of these analyses and is organized as
follows:
1. Data collection and screening procedures.
2. Primary data analyses.
(1) Compliance with the assumptions for ANCOVA,
(2) Summary and descriptive statistics, and
(3) Analysis of covariance.
68


3. Secondary data analyses.
(1) Multivariate analysis of variance,
(2) Program effectiveness,
(3) Time spent on instruction,
(4) Analysis of explanations and summaries, and
(5) Analysis of student comments.
4. Summary of results.
Data Collection and Analysis Screening Procedures
The students disks recorded pretest and posttest results (selections and scores),
as well as other interactional data (e.g., time information, student explanations and
summaries, and program critiques). These data were extracted from the disks,
checked for accuracy, and hand entered into the Statistical Package for the Social
Sciences (SPSS 7.5 for Windows, 1996) data editor. This database served as the
master file for the primary and secondary statistical analyses.
As previously discussed, medical students enrolled in the hematology rotation of
the Pathophysiology of Disease course were required to complete this program. One
hundred and thirty-two students completed the program; however, thirty-one students
were eliminated from the final analysis due to inaccurate data or noncompliance.
Three students were dropped because of extreme scores or missing data. Two
students assigned to the single condition were excluded because they violated the
69


requirement to work independently. The remaining twenty-six students were
eliminated due to lack of effort. These students either failed to provide the
explanations and summaries required of the generative version of the program, or
they chose to spend less than thirty-six minutes (the established minimal time
necessary to complete the program) on the six practice cases. Thus, the final analysis
included one hundred and one students (31 singles and 35 pairs).
Primary Data Analyses
This study used a 2 X 2 factorial design to test for the main effects and
interaction predicted in Chapter Two. The students were randomly assigned to the
different levels of the independent variables. An analysis of variance on the subjects
pretest scores suggests that randomization was achieved (see discussion of random
assignment in the next section); however, analysis of covariance (using the pretest as
the covariate) was selected to conduct the primary data analysis to increase the power
of the study. This section discusses the assumptions of analysis of covariance
(ANCOVA), provides summary and descriptive data, and concludes with a
presentation of the ANCOVA results.
Assumptions for Analysis of Covariance
Random and Independent Errors. Each subject was randomly assigned to one
treatment group according to the process described in Chapter Three. An analysis of
70


variance conducted on the pretest for the four treatment groups suggests that random
assignment was achieved: F(3, 63)=.413, p=.744. Additionally, the researcher
observed 84 students running the computer program and noticed only two students
not complying with the requirement to work as assigned (i.e., individually or pairs).
These students were excluded from the statistical analyses. The students disks also
recorded the dates and times the students accessed the program. Analysis of the dates
and times of the students not observed suggest compliance with the requirement to
work as assigned. To gauge the level of compliance with the condition to refrain
from discussing the program, the researcher randomly asked students what they had
heard about the program prior to arriving at the LRC. Most of the students reported
hearing that the program was long or frustrating, but none of the students
reported any discussion related to the programs content. An analysis of variance
conducted on pretest data (F(2, 63)=.217, p=.805) and posttest data
(F(2, 63)=.204, p=.816), by the week the program was completed, suggests this
noncompliance had a negligible effect on the students scores. Finally, inspection of
a residual plot of posttest scores (see Appendix F) suggests that the assumption of
random and independent errors was met.
Normality. Analysis of covariance is robust to departures of the sample from
normality (Lomax, 1992); nevertheless, the subjects posttest scores were analyzed to
determine if the requirement of normality was met. Mean posttest scores were 22.9
with a standard deviation of 5.18. The skewness statistic (-.270) indicates that the
71


distribution was slightly skewed to the left. The kurtosis statistic (-.469) indicates
that the center and tails of the distribution were slightly shorter than that of a normal
distribution. The ratio of each of these statistics to their standard error can be used as
a test of normality (SPSS, 1996). The ratios of -.92 (skewness) and -.81 (kurtosis)
fall between the values -2 and +2, and the shape of the histogram is fairly symmetric,
therefore the posttest scores approximate a normal distribution curve. Stem and leaf
plot analysis of posttest scores and the Kolmogorov-Smimov test of normality
(F (66)=. 102, p=.083) confirm that the posttest scores approximate a normal
distribution (see Appendix F).
Homogeneity of Variance. Levenes test of equal variances, and Cochrans C,
and Bartlett-Box F tests of homogeneity of variance were conducted on the posttest
data to determine if the cell variances were equal. Levenes F(3, 62)=.697, p=.558,
Cochrans C(16,4)=.282, p=1.000, and Bartlett-Box F(3, 6475)=.069, p=.977 all
support the null hypothesis that the population cell variances were equal.
Linearity and Homogeneity of Regression Slopes. Inspection of the
scattergrams showing the relationship between the pretest and posttest scores for the
entire sample, as well as each treatment group, support the assumption of linearity. In
each case a straight-line best fit the data (see Appendix F).
The assumption of homogeneity of regression slopes requires the slopes of the
regression lines to be the same for each group, which permits testing for group
intercept differences (Lomax, 1992). This assumption was supported by positive
72


Pearson correlation coefficients (see Table 4.1). However, as the table shows, the
Pearson coefficient for treatment group 3 was different than the Pearson coefficients
for the other three groups. To ensure compliance with this assumption, an analysis of
covariance was conducted in which a three-way interaction was specified between the
covariate (pretest total score) and the two independent variables (SPSS 7.5,1996).
This interaction term, PROGRAM*SINGLE/PAIRS*PRETEST TOTAL SCORE,
showed no evidence of violation of the equal slopes assumption:
F(l, 59)=.690, p=.410. Thus the null hypothesis of equal slopes was accepted.
Table 4.1
Pearson Correlation Coefficients
Treatment Group Pearson Correlation Coefficient
Total r = .490
Treatment Group 1 (single/base) r = .593
Treatment Group 2 (single/generative) r = .568
Treatment Group 3 (pair/base) r = .309
Treatment Group 4 (pair/generative) r = .660
73


Covariate Measured Without Error. As discussed in Chapter Three, the same
three cases were used for the pretest and posttest. These cases required the subjects to
evaluate, synthesize, and apply content material while making multiple decisions
regarding the appropriateness of lab test, assessment, and management actions. This
testing format precluded the ability to obtain a reliability estimate since the test cases
were not objective, multiple-choice tests. However, these test cases were developed
in conjunction with the practice cases and reviewed by transfusion medicine experts
to establish content validity and accuracy. Therefore, not meeting this assumption did
not adversely affect the results.
Fixed Independent Variables. This assumption states that the researcher fixes
the levels of the independent variables (Lomax, 1992). The studys design, which
assigned subjects to one treatment group only, ensured compliance with this
assumption.
Independence of the Covariate and Independent Variable. Although not an
assumption of the ANCOVA model, the covariate should not be influenced by the
independent variable (Lomax, 1992). There were two independent variables in this
study program version and method of instruction. This condition was met for
version of program since the pretest was taken before the students entered the practice
portion of the computer program. Although the students failed to comply with the
request to refrain from discussing the program, there appeared to be no significant
effect on the pretest scores. An analysis of variance investigating the differences in
74


pretest scores over time suggests that there were no significant differences in pretest
scores during the course of the study: F(2, 65)=.217, p=. 805. When asked, none of
the students admitted to knowing about the content of the pretest before running the
program. In fact, several students commented that they wish they had known the
pretest and the posttest were the same.
Method of instruction, the second independent variable, was randomly assigned
to the unit teaching labs instead of each individual student. The studys design
established the pair as the unit of analysis (rather than two individuals); therefore,
students assigned to the collaborative condition tested together rather than separately.
An analysis of variance conducted on the pretest scores demonstrated no significant
difference between the single and paired treatments: F(l, 64)=.479, p=.491. This
result suggests that method of instruction was independent of, and did not influence,
the pretest scores.
In summary, the following assumptions were fully met for this study: random
and independent errors, normality, homogeneity of variance, linearity and
homogeneity of regression slopes, fixed independent variables, and independence of
the covariate and the independent variable. The assumption pertaining to the
measurement of the covariate without error could not be tested due to the nature of
the measure.
75


Summary and Descriptive Statistics
Table 4.2 presents summary and descriptive statistics for this investigation. The
pretest scores ranged from 13.7 for the single/base treatment to 15.9 for the pair/base
treatment, with an overall mean of 15.1 (SD = 6.4). The pretest means ranged from
14.8 to 15.5 and from 14.5 to 15.6 for the two independent variables respectively.
The overall mean for posttest performance was 22.9 (SD = 5.2). Posttest scores
ranged from 21.1 for the single/generative treatment to 24.2 for the pair/base
treatment. For the two independent variables, the posttest scores were 24.1 versus
21.4 favoring the base version of the program, and 23.1 compared with 22.7 favoring
the collaborative condition.
Table 4.2 also shows that the mean time for completing the practice portion of
the program was 72.2 minutes (SD = 30.8). The mean completion time ranged from
63 minutes to 88.8 minutes. For the two independent variables the mean completion
times were: (a) 63.2 minutes for the base program versus 84.4 minutes for the
generative version of the program and (b) 70.5 minutes for the pairs compared with
74.1 minutes for the single students.
76


Table 4.2
Summary Statistics
Program Single or Pairs N Pretest Posttest Practice Case Time
Base Single 18 M 13.7 23.9 63.4
SD 8.6 5.0 18.6
Pairs 20 M 15.9 24.2 63.0
SD 5.5 4.9 19.1
Total 38 M 14.8 24.1 63.2
SD 7.1 4.9 18.7
Generative Single 13 M 15.7 21.1 88.8
SD 6.5 5.2 41.7
Pairs 15 M 15.3 21.6 80.6
SD 4.9 5.5 38.0
Total 28 M 15.5 21.4 84.4
SD 5.5 5.3 39.2
Total Single 31 M 14.5 22.7 74.1
SD 7.7 5.2 32.5
Pairs 35 M 15.6 23.1 70.5
SD 5.2 5.2 29.6
Total 66 M 15.1 22.9 72.2
SD 6.4 5.2 30.8
Analysis of Covariance
It was hypothesized that the embedded generative learning strategies and
collaboration would lead to deeper processing of content material and result in higher
levels of learning as measured by performance on the posttest. Furthermore, it was
predicted that the two independent variables would interact to influence learning and
posttest performance. Analysis of covariance (ANCOVA) was selected to test for the
77


presence of these main effects and interaction. Table 4.3 presents the results of the
ANCOVA. Table 4.4 presents the adjusted cell and marginal means, by treatment
group.
Table 4.3
Analysis of Covariance Summary
Source SS df MS F P
Program 144.48 1 144.48 7.49* .008*
Singles/Pairs 0.10 1 0.10 .01 .943
Program X Singles/Pairs 6.79 1 6.79 .35 .555
Pretest Total Score3 418.83 1 418.83 21.72 .000
Error 1176.40 61 19.29
* P < .05
a Covariate
Note. The posttest total score is the dependent variable. Effect size and
power at the .05 level are .61 and .78 respectively.
Table 4.4
Adjusted Cell and Marginal Means
Program Single or Pairs Adjusted Cell Means Adjusted Marginal Means
Base Single 24.53 24.20
Pairs 23.90
Generative Single 20.84 21.20
Pairs 21.51
Single 22.97
Pairs 22.89
78


Main Effect: Version of Program. The results of the ANCOVA suggest the
presence of a significant main effect for the version of program:
F(l, 61)=7.49, p=.008. Inspection of the adjusted means indicates that subjects
assigned to the base version of the program scored significantly higher on the posttest
than subjects assigned to the generative program.
Main Effect: Method of Instruction. The preceding data suggest that there was
not a significant main effect for method of instruction: F(l, 61)=.01, p=.943. The
adjusted means were virtually identical for the single and collaborative treatments.
Interaction: Version of Program X Method of Instruction. The analysis of
covariance results suggest that the two independent variables did not interact:
F(l, 61)=.35, p=.555. This indicates that the version of program and method of
instruction variables operated independently to affect learning.
Summary of ANCOVA Results. The preceding data suggests that there was a
main effect for the version of program variable, but not in the direction hypothesized.
Furthermore, there was no main effect for the method of instruction variable, nor was
there an interaction between the two independent variables as hypothesized. Before
addressing the implications of these results, a series of secondary analyses were
conducted to provide additional insight concerning the dynamics of the study.
79


Secondary Data Analyses
Multivariate Analysis of Variance
The ANCOVA demonstrated a significant main effect for the version of
program. Since the posttest consisted of three different cases, a multivariate analysis
of variance was conducted to determine where these differences occurred. Before
computing the MANOVA it was necessary to again test for assumptions. The
additional assumptions include composite observations that are normally distributed,
equally variable in the population sampled, and independent.
Analysis of the histograms for the three posttest case scores indicates that the
scores for test cases B and C appear to violate the assumption that composite
observations were normally distributed. Comparing the ratios of the skewness and
kurtosis statistics with the respective standard errors for each test case confirmed the
violation. The ratios for test case A (-1.02 and -.24 respectively) were between the
-2 to +2 parameter for normality; however, for test cases B and C these ratios were
outside the acceptable range for normality (for case B, skewness ratio was -.2.36; for
case C, skewness ratio was -4.1 and kurtosis ratio was 3.75). These ratios indicate
that the distributions for test cases B and C were more highly skewed to the left than
normal and that the tails for case C were longer than normal. Levenes test of equal
variance, Cochrans C, and Bartlett-Box F analyses of each posttest case suggests that
the assumption of equal variance was met (see Table 4.5). Finally, Boxs M=17.683,
80


F(18, 10791)=.892, p=.589, indicates compliance with the test for homogeneity of
dispersion (i.e., the observed covariance matrices of the dependent variables were
equal across groups). In sum, the assumptions of equal variability and independence
were met, whereas the assumption of normality was not. However, multivariate
analysis of variance is relatively robust to moderate departures from normality
(Lomax, 1992), therefore, this violation had minimal effect.
Table 4.5
Tests for Equal Cell Variance
A B C
Levenes Test F=1.454, p=.24 F=. 147, p=.93 F=1.695, p=. 18
Cochrans C C= .33177, p=.57 C=.33644, p=.53 C= .40654, p=.ll
Bartlett- Box F F= .84663, x) ii L- F=.41605, p=74 F=1.8231, p=. 14
The results of the MANOVA demonstrate a significant main effect for the
version of program variable on the posttest: Wilks F(3, 57)=4.46, p=.007. Follow up
univariate tests found significant differences for test case A (F(l, 59)=4.17, p=.046)
and test case C (F(l, 59)=4.64, p=.035) (see Table 4.6). Mean scores for subjects
using the base program were 4.5 and 3.6 (for test cases A and C respectively),
whereas mean scores for subjects using the generative program were 3.0 and 2.9 for
the same test cases (see Table 4.7). These results suggest that subjects using the base
81


program significantly outscored subjects using the generative program on these two
test cases.
Table 4.6
Univariate Tests of Significance
Dependent Variable Sum of Squares Df Mean Square F Sig.
Posttest Case A 34.88 1,59 34.88 4.17 .046*
Posttest Case B 17.05 1,59 17.05 1.39 .243
Posttest Case C 7.97 1,59 7.97 4.64 .035*
* P < .05
Table 4.7
Posttest Case Scores for Version of Program
Dependent Variable Program Mean
Posttest Case A Base 4.50
Generative 3.01
Posttest Case B Base 16.25
Generative 15.21
Posttest Case C Base 3.57
Generative 2.86
82


Program Effectiveness
The results of the ANCOVA suggest that subjects using the base program
significantly outperformed subjects using the generative version of the program. The
results of the MANOVA indicate that the differences in performance occurred on test
cases A and C respectively. This section seeks to determine if the programs were
effective teaching principles of transfusion medicine. To address the question of
effectiveness, a series of one-tailed t-tests were computed to (a) compare total
pretest posttest scores, (b) determine which components of the test cases appeared
to stimulate learning, and (c) specifically address the effectiveness of the generative
program, since students using this version scored significantly lower on the posttest.
Total Pretest Posttest Gain. The first analysis consisted of comparing the
pretest and posttest mean scores for the total group of subjects. Alpha was set at .05.
This analysis indicates that the total posttest scores were significantly higher than
pretest scores (t=l 0.582, p<.0001).
Test Case Component Analysis. The pretest and posttest were broken into their
component parts to determine in which portions of the program posttest scores were
significantly higher than pretest scores. This analysis consisted of running fifteen
separate t-tests, therefore, to control alpha from growing unacceptably large, alpha
was set at .005 for each t-test. Table 4.8 presents the results of these t-tests.
These data suggest that posttest scores were significantly higher than pretest
scores for the three decision making areas (tubs=7.817, p<-005; tASSess=4.524, p<.005;
83


and tManage=5.795, p<.005), as well as for all three test cases (tA=5.34, p<.005;
te=6.97, p<.005; tc=4.87, p<.005). However, learning was not uniform across the
three test cases. Lab scores were significantly higher on the posttest than the pretest
for all three cases (tAL=4.814, p<.005; tBL=4-408, p<.005; and tcL=3.725, p<.005),
whereas assessment scores were significant in the test case B only (tBA=4.065,
p<.005), and management scores were significant in test cases B and C (tsM=2.862,
p<.005; tcM=5.744, p<.005).
84


Table 4.8
Aggregate Pretest/Posttest T-test Results
Component Mean SD Df T
Total Score- Labs 5.67 5.89 65 7.817*
Total Score Assessment 1.09 1.96 65 4.524*
Total Score Management 1.10 1.55 65 5.795*
Test Case A Total 2.74 4.17 65 5.337*
Labs 2.35 3.96 65 4.814*
Assessment .27 1.34 65 1.651
Management .13 .72 65 1.537
Test Case B Total 3.29 3.83 65 6.974*
Labs 2.09 3.85 65 4.408*
Assessment .82 1.64 65 4.065*
Management .37 1.03 65 2.862*
Test Case C Total 1.83 3.06 65 4.865*
Labs 1.23 2.67 65 3.725*
Assessment .00 .68 65 .363
Management .61 .86 65 5.744*
* P < .005
85


Pretest Posttest Gain for the Generative Program. The final analysis compared
pretest and posttest mean scores for the students using the generative version of the
program. A one-tailed t-test (alpha = .05) indicated that posttest scores were
significantly higher than pretest scores (t=6.48, p<.0001). The pretest and posttest
were again broken into their component parts for analysis. Alpha was set at .005.
The component analysis t-tests indicate that lab scores were significantly higher
on the posttest than the pretest (t=4.84, p<.005), but there were no significant
differences between pretest and posttest scores for assessment and management.
Also, the posttest scores were significantly higher than pretest scores for test cases B
and C (te=3.85, p<.005; Tc=2.87, p<.005). Finally, management scores were
significant for test case C only (tcM=3.057, p<.005).
In summary, total posttest scores for the generative program were significantly
higher than pretest scores; however the difference appeared significant in only one
decision making area (labs), two test cases (B and C), and one part of case C
(management). In contrast, when the analysis included both program versions,
posttest scores were significantly higher than pretest scores for all three decision
areas, all three test cases, labs in all three cases, assessment in one case, and
management in two cases. Thus, there appeared to be a significant fall off in learning
effectiveness for the generative program when compared to the aggregate analysis
that included both program versions (Note: t-test analyses performed on the base
version of the program mirror the aggregate results and are presented in Appendix F).
86


Full Text

PAGE 1

THE EFFECTS OF EMBEDDED GENERATIVE LEARNING STRATEGIES AND COLLABORATION ON KNOWLEDGE ACQUISITION IN A COGNITIVE FLEXffiiLITY -BASED COMPUTER LEARNING ENVIRONMENT by John Robert Higgs, Jr. B.S., University of Maryland, 1980 M.A., State University ofNew York, Plattsburgh, 1991 A thesis submitted to the University of Colorado at Denver in partial fulfillment of the requirements for the degree of Doctor of Philosophy Educational Leadership and Innovation 1998

PAGE 2

1998 by John Robert Higgs, Jr. All rights reserved.

PAGE 3

This thesis for the Doctor of Philosophy degree by John Robert Higgs, Jr. has been approved by R. Scott Grabinger Laura D. Goodwin Judith A. Duffield Michael P. Marlow 6.. 19'13' Daniel R. Ambruso Date

PAGE 4

Higgs, John Robert, Jr. (Ph.D., Educational Leadership and Innovation) The Effects of Embedded Generative Learning Strategies and Collaboration on Knowledge Acquisition in a Cognitive Flexibility-Based Computer Learning Environment Thesis directed by Associate Professor R. Scott Grabinger ABSTRACT Inert knowledge is typically produced in learning environments that simplify content and context. Several theories and methods exist to establish learning environments that overcome the negative effects of inert knowledge. This study examined the effects of combining three instructional approaches on knowledge acquisition by advanced learners in a computer-based learning environment. This study used a 2 X 2 experimental design. The two independent variables were program version and method of instruction. Two qualitatively different computer programs were used: a base program developed according to cognitive flexibility theory (CFT) and a generative program containing embedded generative learning strategies (explanation and summary cues). Students completed these programs either individually or collaboratively. One hundred and thirty-two second-year medical students participated in the study. The two treatment variables were randomly assigned to the students. Students received computer diskettes that permitted access to the computer module, "Handling Transfusion Hazards," and were given instructions on how, when, and where to complete the program. The computer program consisted of an orientation section, a IV

PAGE 5

pretest, a learning phase, and a posttest. The pretest and posttest consisted of the same three clinical transfusion cases; the learning phase contained six cases. One hundred and one students were included in the final analysis. Analysis of covariance was used to test for the presence of main effects or an interaction. The ANCOVA yielded a significant main effect (p=.008) on posttest performance for the version of program treatment variable. No significant differences in achievement were found for method of instruction, nor was an interaction present between the two treatment variables. This research suggests that advanced learners did not benefit equally from the two computer programs. Learners using the base computer program performed significantly better than students using the generative version on the posttest. Also, although the students felt there was an advantage to working with a peer, no significant advantage materialized. Recommendations for future research based on these findings are presented. This abstract accurately reflects the content of the candidate's thesis. I recommend its publication. Signed R. Scott Grabing r v

PAGE 6

DEDICATION To my family, Kathy, Stephanie, and Katie. Thank you for your unyielding support and understanding during this doctoral process. Also a special thanks to Lieutenant Colonel Michael Whyte, who supported my doctoral pursuit.

PAGE 7

ACKNOWLEDGEMENTS I would like to acknowledge Scott Heath who provided the programming expertise to develop the generative version of the program used in this study. Additionally, I would like to thank Lisa Traditi and her staff (Kathei, David, and Jim) at the Dennison Library Learning Resources Center for the support they provided during the data collection period. To my dissertation committee members-Mike Marlow, Judy Duffield, Laura Goodwin, and Dan Ambruso thank you for your active involvement during the dissertation process. I would particularly like to thank Dan Ambruso for the opportunity to work with the Transfusion Medicine Team. Laura Goodwin deserves special recognition for the significant time she spent guiding the research design and data analysis used in this investigation. Finally, Scott Grabinger served as program advisor and dissertation chair during my doctoral program. He invested three years as advisor and mentor, and provided the support, encouragement, and friendship necessary for me to successfully complete this study. Thank you Scott for your wisdom and guidance.

PAGE 8

CONTENTS Figures ............................................................................................................ xiii Tables ............................................................................................................. xiv CHAPTER 1. STATEMENT OF THE PROBLEM ............................................................ 1 Introduction ........................................................................................... 1 Purpose of the Study .............................................................................. 4 Problem Statement .................................................................... 4 Problem Solution ....................................................................... 5 Cognitive Flexibility Theory ................................................................. 6 Generative Learning Theory .................................................................. 6 Collaborative Learning .......................................................................... 7 Research Issues ...................................................................................... 8 Research Questions ............................................................................. 1 0 Research Methodology ........................................................................ 11 Experimental Intervention ....................................................... 11 Dependent Variable ................................................................. 12 Definition of Terms ............................................................................. 14 Vlll

PAGE 9

2. REVIEW OF LITERATURE ...................................................................... 18 Introduction .......................................................................................... 18 Cognitive Flexibility Theory: Assumptions and Theory ...................... 21 Research on Cognitive Flexibility Theory ............................... 27 Generative Learning Theory ................................................................ 30 Compatibility of Cognitive Flexibility Theory and Generative Learning Theory ................................................................ 32 Individual Student Focus ......................................................... 34 Collaborative Learning ........................................................................ 34 Research on Collaborative Learning ........................................ } 7 Research Questions .............................................................................. 40 Research Hypotheses ........................................................................... 41 3. METHODOLOGY ....................................................................................... 43 Introduction .......................................................................................... 43 Materials .............................................................................................. 43 The Program: Handling Transfusion Hazards ........................ .44 Program Development and Evaluation .................................... 53 Participants ........................................................................................... 56 Assignment to Treatment ..................................................................... 56 Design .................................................................................................. 58 IX

PAGE 10

Variables ........................................................................ ..................... 59 Independent Variables .............................................................. 59 Dependent Variable .................................................................. 60 Instruments ........................................................................................... 62 Pretest and Posttest .................................................................. 62 Procedure ............................................................................................. 63 Pilot Study ................................................................................ 63 Instructional Intervention ..................................................................... 66 4. RESULTS .................................................................................................... 68 Introduction .......................................................................................... 68 Data Collection and Analysis Screening Procedures ........................... 69 Primary Data Analysis ......................................................................... 70 Assumptions for Analysis of Covariance ................................. 70 Summary and Descriptive Statistics ........................................ 76 Analysis of Covariance ............................................................ 77 Secondary Data Analyses ..................................................................... 80 Multivariate Analysis of Variance ........................................... 80 Program Effectiveness ............................................................. 83 Time Spent on Instruction ........................................................ 87 Analysis ofExplanations and Summaries ................................ 88 X

PAGE 11

Analysis of Student Comments ................................................ 89 Summary of Results ............................................................................. 91 5. DISCUSSION AND RECOMMENDATIONS ........................................... 93 Introduction .......................................................................................... 93 Discussion ............................................................................................ 94 Effect of Embedded Generative Learning Strategies ............... 94 Effect of Collaboration ............................................................ 98 Interaction of Generative Learning Strategies and Collaboration ................................................................... 1 01 Other Consideration: Influence of the Overall Instructional Environment. ..................................................... 1 02 Summary ................................................................................ 1 06 Recommendations for Future Research ............................................ .1 06 Study Limitations ............................................................................... 1 09 XI

PAGE 12

APPENDIX A. SAMPLE PRACTICE CASE ....................................................... 112 B. SAMPLE GENERATIVE SCREENS ......................................... .121 C. SAMPLE TEST CASE ................................................................. 124 D. STUDENT INSTRUCTIONAL HANDOUT ............................... 132 E. PRETEST/POSTTEST DATA SUMMARY ................................ 135 F. STATISTICAL INFORMATION ................................................. 137 G. SAMPLE STUDENT COMMENTS ............................................ 142 REFERENCES .............................................................................................. 144 Xll

PAGE 13

FIGURES Figure 3.1 Sample Case Screen from Practice Case Three ............................................... 46 3.2 Sample History Screen from Practice Case Three .......................................... 4 7 3.3 Sample Perspectives Screen from Practice Case Three .................................. 48 3.4 Sample Similar Cases Screen from Practice Case Three ................................ 49 3.5 Textbook Index Screen .................................................................................... 50 3.6 Experimental Design ....................................................................................... 59 3. 7 Sample Screen from Test Case B .................................................................... 61 X Ill

PAGE 14

TABLES Table 4.1 Pearson Correlation Coefficients .................................................................... 73 4.2 Summary Statistics .......................................................................................... 77 4.3 Analysis of Covariance Summary ................................................................... 78 4.4 Adjusted Cell and Marginal Means ................................................................. 78 4. 5 Tests for Equal Cell Variance ....... ." ................................................................. 81 4.6 Univariate Tests of Significance .................. ................................................... 82 4.7 Posttest Case Scores for Version of Program .................................................. 82 4.8 Aggregate Pretest!Posttest T-test Results ........................................................ 85 4. 9 Mean Time Spent on Instruction ..................................................................... 87 4.10 Summary Statistics for Explanations and Summaries ..................................... 89 4.11 Student Comments .......................................................................................... 90 XIV

PAGE 15

CHAPTER 1 STATEMENT OF THE PROBLEM Introduction A common criticism of the traditional model of teaching and learning involves the lack of effective instructional methods to facilitate critical thinking and higher levels of learning (Bransford, Franks, Vye, & Sherwood, 1989; Grabinger, 1996; Resnick, 1987; Scardamalia & Bereiter, 1994; Spiro, Coulson, Feltovich, & Anderson, 1988; Weinstein, 1978). The goal of instruction in the traditional model of education involves efficiently and effectively transferring knowledge to learners by breaking instruction down into simple, basic units devoid of content and context (Bednar, Cunningham, Duffy, & Perry, 1992). This form of teaching produces students who "treat information as facts to be memorized and recited rather than as tools to solve problems ... (Grabinger, 1996, p. 666), which results in a use-of knowledge gap (Perkins, 1992). The traditional model of teaching, therefore, produces learners unable to recall, use, or transfer knowledge and skills to new and novel situations. Whitehead (1929) calls this "inert knowledge" since it is information students in fact possess, but because it is memorized devoid of context, is not transferable, even to relevant situations (Bransford et al., 1989; Bransford & Vye, 1989; Cognition and Technology Group [CTG], 1992; Grabinger, 1996; Perkins, 1

PAGE 16

1992; Whitehead, 1929). Thus, due to the absence of context, inert knowledge maybe the predominant form of knowledge acquired by students in the traditional model of education. In contrast to the traditional model of education, many educators and scholars believe that today's complex world requires the ability to use tools and knowledge in a variety of domains and situations. These critical-thinking skills allow people to fully participate in our modem, information-age society by giving them the ability "to analyze and synthesize information to solve technical, social, economic, political, and scientific problems" (Grabinger, 1996, p. 665). The question then becomes how to encourage critical-thinking skills development in students rather than the formation of inert knowledge. Resnick (1987) suggests that the key to developing critical-thinking ability involves using instructional strategies that (a) emphasize socially shared intellectual work organized around joint task accomplishment, (b) encourage student observation and commentary which makes usually covert cognitive processes overt, and (c) tailor the treatment of the subject matter to engage students in the processes of meaning construction and interpretation. Such student-centered strategies increase time on task, increase the amount and relevance of feedback to each student, use a performance-based system to assess student achievement (rather than a system where all students spend the same amount of time studying instructional material), and incorporate self-paced instruction geared to student capabilities rather than class average (Reiser & Salisbury, 1995). This approach to learning promotes achievement 2

PAGE 17

of the basic goals of education-retention, understanding, and active use of the knowledge and skills (Perkins, 1992). Perkins (1992) writes: Surely we want what is taught retained, else why teach it? Unless knowledge is understood, to what purposes can it be put? Finally, having and understanding knowledge and skills come to naught unless the learner actually makes active use of them later in life. (p. 18) A student-centered approach considers learning to be an experiential process (Dewey, 1938) that promotes the social structures and dynamics required for knowledge building to occur among students (Scardamalia & Bereiter, 1994). Rather than focusing on the teacher-centered model of education, which is based on the assumption that students will learn because they are asked or told to learn (Weinstein, 1978), a student-centered approach focuses on students sharing perspectives with the teacher and other students and then modifying internal representations of knowledge in response to this sharing experience. In a student-centered environment, learning becomes a process of constructing, interpreting, and modifying representations of reality based on experience (Jonassen, 1994). Wilson (1996) defines such a studentcentered learning environment as "a place where learners may work together and support each other as they use a variety of tools and information resources in their guided pursuit of learning goals and problem-solving activities" (p. 5). 3

PAGE 18

Purpose of the Study Problem Statement For many years now, research (Feltovich, Spiro, & Coulson, 1989; Spiro et al., 1988; Spiro, Vispoel, Schmitz, Samarapungavan, & Boerger, 1987) has been conducted on learning at the stage of advanced knowledge acquisition (i.e., the post introductory or intermediate stage oflearning in a subject domain). The goals of advanced knowledge acquisition shift from recognition and recall to attaining a deep understanding and flexible application of content material (Spiro et al., 1988); however these goals have been hard to reach. This has been especially true of advanced students trying to learn in ill-structured knowledge domains (e.g., medicine). For example, F eltovich, Spiro, and Coulson ( 1989) found that medical students had problems transferring knowledge previously learned in one context, such as medical school coursework, to new situations, such as clinical problem solving. Spiro and his colleagues attribute the deficiencies in learning outcomes (i.e., inert knowledge formation) of students at this advanced stage of learning to oversimplification (reductive bias) of complexity. A predominant share of the misconceptions (and networks of misconception) that we have identified reflect one or another kind of oversimplification of complex material. ... Misconceptions of advanced material result from both interference from earlier, simplified treatments ofthat material and from a prevailing mode of approaching the learning process in general that fosters simplificational strategies and leaves learners without an appropriate cognitive repertoire for the processing of complexity. (Spiro et al., 1988, p. 3 76) 4

PAGE 19

Thus, ill-structuredness (the combination of breadth, complexity, and irregularity of a content domain) and the goals of advanced knowledge acquisition unite to create a difficult problem for both teachers and students who tend to rely on simple strategies to present and learn complex subject matter. Problem Solution In response to the problem of reductive bias, Spiro and his colleagues ( 1988) proposed cognitive flexibility theory as a means to create learning environments that provide the complexity and multidimensionality of content required for advanced knowledge acquisition in ill-structured domains. This study investigated the effects of a computer-based microworld on knowledge acquisition among advanced learners. The computer program was developed according to principles of learning, instruction, and knowledge representation espoused by cognitive flexibility theory. The program was further enriched with embedded generative learning strategies (explanation and summary cues) and collaborative activity to produce a computer-based microworld that presents information, stimulates exploration, facilitates high level thought processes, and promotes collaboration (Grabinger, 1996). The remainder of this chapter briefly outlines cognitive flexibility theory, generative learning theory, and collaborative learning, frames the research questions, and provides an overview of the research methodology. 5

PAGE 20

Cognitive Flexibility Theory Cognitive flexibility theory (CFT) targets the stage of advanced knowledge acquisition, and, as such, provides a framework for addressing the difficulties inherent in complex and ill-structured knowledge domains (Spiro et al., 1988; Spiro, Feltovich, Coulson, & Anderson, 1989; Spiro & Jehng, 1990). Evidence suggests that learners develop conceptual misunderstandings during the advanced phase of learning, which seriously affects their ability to transfer knowledge to new situations (Jonassen, Ambruso, & Olesen, 1992; Spiro et al., 1988). Cognitive flexibility theory emphasizes case-based instruction with multiple representations of content in ill structured knowledge domains to avoid oversimplifying instruction (Spiro & Jehng, 1990). This enables learners to investigate the multiple perspectives inherent in a knowledge domain in an exploratory manner (Jonassen et al., 1992), which promotes knowledge acquisition and transfer. Generative Learning Theory Generative learning theory is based on the notion of an active learner (Wittrock, 1974a; 1974b; 1985; 1990; 1992). In this case, the learner works to create meaning by drawing relationships between existing information and experiences stored in cognitive structures and information presented in the learning environment (Grabowski, 1997; Wittrock, 1985). Wittrock (1974b) demonstrates the importance of active learner participation in the learning process by writing: "Although a student 6

PAGE 21

may not understand sentences spoken to him by his teacher, it is highly likely that a student understands sentences that he generates himself" (p. 182). Thus, understanding cannot be directly transferred to students, but must arise from the interactions between the learner and the environment (Wittrock, 1974a; 1974b; 1985). The new meaning constructed from the interactions between the learner and the environment results in learning with understanding (Wittrock, 1974a; 1974b; 1985), characterized by the creation of meaningful cognitive structures, better retention and retrieval of information, and better explanations of the information (DiVesta. 1989; Grabowski, 1997). Collaborative Learning Collaborative learning actively involves learners in the construction of their knowledge, but, in contrast to most applications of cognitive flexibility theory and generative learning theory which focus primarily on the individual student's cognitive development, collaborative learning promotes socially shared intellectual development. Collaborative learning involves the instructional use of small groups where students work together to maximize each other's learning (Johnson & Johnson, 1992). To accomplish the goal of maximizing learning, the students are responsible for learning the assigned material themselves and ensuring that all other members of the group learn the material as well (Johnson & Johnson, 1992). Such peer groupwork allows the students to actively engage in interactive higher-level thinking 7

PAGE 22

and problem-solving activities (Damon & Phelps, 1989) leading to enhanced academic achievement and cognitive growth, better attitudes and motivation toward learning, and a host of positive social-emotional benefits (Johnson & Johnson, 1992; Nastasi & Clements, 1991; Slavin, 1996). Research Issues What makes learning so hard and what can be done about it? The answers are teacher-centered education and student-centered education respectively. Learning requires intellectual effort on the part of the learner (Osborne & Wittrock, 1985), yet with its emphasis on passive absorption of information, the teacher-centered model of education devalues the effort required. In contrast, the student-centered approach to learning activates the learners' information-processing strategies and stores of relevant specific memories related to the information to be learned (Wittrock, 1978). This results in "mindful" (Salomon, 1985) processing. Thus, the student-centered approach values "effortful learning" and uses strategies designed to promote generative learning (i.e., knowledge construction). Cognitive flexibility theory is a student-centered approach specifically targeted toward knowledge construction and acquisition by advanced learners in ill-structured knowledge domains. Cognitive flexibility theory advocates creating case-based computer learning environments that emphasize the conceptual complexities and interrelatedness of subject matter content (Spiro et al., 1988). The use of authentic, 8

PAGE 23

multiple cases permits students to see many contexts in which concepts occur or apply, and index meaning accordingly. Students, therefore, develop the ability to adaptively re-assemble diverse elements of knowledge in response to changing situational demands (i.e., the needs of a given problem-solving or knowledge application situation) (Spiro & Jehng, J 990). Thus, cognitive flexibility theory-based (CFT -based) computer learning environments stimulate mindful processing which, in tum, facilitates the generation of fluid, flexible, and usable knowledge structures. The research to date (Hartman & Spiro, 1989; Jacobson, 1990; Jacobson, Maouri, Mishra, & Kolar, 1996; Spiro et al., 1987) supports this premise; computer programs based upon cognitive flexibility theory principles do indeed promote higher levels of knowledge acquisition and transfer (this research is discussed more fully in Chapter Two). However, given the multitude of student-centered instructional strategies available that also promote generative learning, might advanced learners benefit from the simultaneous integration of compatible learning strategies in a CFT based environment? Might a synergy develop that results in a greater degree of mindfulness and consequently, deeper processing of the content material? If so, would greater knowledge acquisition result? 9

PAGE 24

Research Questions This study examined the effectiveness of a computer-based microworld (see definition on p. 15) designed to integrate Resnick's (1987) keys for developing critical thinking and increasing learning. Specifically, this study addressed the question: Would knowledge acquisition among advanced learners increase if a cognitive flexibility theory-based computer program was enriched with (a) additional generative learning strategies designed to facilitate comprehension and understanding and (b) collaborative activity to attend to the social nature of learning and the synergistic development and application ofknowledge among groups of students? In this context two main questions were explored: 1. Would the inclusion of embedded explanation and summary cues (generative learning strategies) result in differences in learning in a CFT -based computer microworld? 2. How would the addition of collaborative learning affect learning in this environment? 10

PAGE 25

Research Methodology Experimental Intervention This study used a 2 X 2 factorial design consisting of two independent variables: (a) method of instruction (single learners versus collaborative learners) and (b) program version (CFT base program versus CFT generative program). This design resulted in four treatments consisting of: 1. Single student, base program (Single/base). Subjects in this treatment used the CFT base computer program (i.e., the computer program without the embedded generative learning strategies). 2. Single student, generative program (Single/generative). Subjects in this treatment used the CFT computer program with the embedded generative learning strategies. These subjects were asked to explain the choices they made during the program, as well as to summarize the main ideas and concepts for each case presented. 3. Paired students, base program (Pair/base). Subjects in this treatment collaborated with a partner while using the CFT base computer program. 4. Paired students, generative program (Pair/generative). Subjects in this treatment collaborated with a partner while using the CFT generative computer program. These students were asked to explain and summarize in the same manner as the single/generative subjects. 11

PAGE 26

Dependent Variable The dependent variable for this study was the knowledge constructed (i.e., acquired) by advanced learners. Subjects took a posttest consisting of three transfusion medicine cases similar in structure and content to the six cases contained in the practice segment of the computer program. However, the present study departed from the traditional practice of requiring the students assigned to the collaborative condition to test individually based on the following rationale: 1. Collaborative learning is presumed to provide positive group-to-individual transfer. "Group-to-individual transfer occurs when students ... demonstrate mastery of the material being studied on a subsequent test taken individually" (Yager, Johnson, & Johnson, 1985, p. 61). Research on collaborative, computer-based instruction generally supports this assertion; students working collaboratively in computer-based instruction perform as well as students working alone (Carrier & Sales, 1987; Makuch, Robillard, & Yoder, 1992; Webb, 1985; 1987; 1989) and often perform better (Dalton, Hannafin, & Hooper, 1989; Hooper, Temiyakarn, & Williams, 1993; Johnson, Johnson, & Stanne, 1985; 1986). 2. Collaborative learning is also premised on the notion that collaborative activity leads to emergent knowledge-a level of knowledge that results from the interaction between the understandings of the group rather than a summation of the knowledge ofthe individual participants (Whipple, 1987). 12

PAGE 27

Scaffolding enables the group to solve a problem or achieve a goal which is beyond the capabilities of the individuals involved. If, in fact, collaboration leads to higher performance of the group, why test collaborating students individually?. After all, research demonstrates that group-to-individual transfer occurs; therefore, if the learning experience is positive for students working individually, the learning experience should also be positive for the students working in groups. 3. The computer program used in this study was entirely self-contained. The learning and testing phases of the program were developed and delivered as one complete module, with the pretest and posttest immediately preceding and following the learning phase respectively. Factor in the medical students' tight schedules and it would have been awkward and inexpedient to require separate testing. Furthermore, sentiment exists among some researchers that promoting shared cognitive activity but assessing learning individually in the same study leads to inconsistencies in results (Forman & Kraker, 1985). This study, therefore, accepted the assumptions of a positive learning experience for the students working individually and that transfer would occur for the collaborative pairs. This provided the opportunity to compare individual versus socially-mediated knowledge acquisition. 13

PAGE 28

Definition of Terms Advanced knowledge"learning beyond the introductory stage for a subject area, but before the achievement of practiced expertise that comes with massive experience" (Spiro et al., 1988, p. 375). Case"a contextualized piece of knowledge representing an experience that teaches a lesson fundamental to achieving the goals of the reasoner" (Kolodner, 1993, p. 13). Case-based learning involves problem solving, understanding, and learning. Case-based learning "can mean adapting old solutions to meet new demands, using old cases to explain new situations, using old cases to critique new solutions, or reasoning from precedents to interpret a new situation (much as lawyers do) or create an equitable solution to a new problem" (Kolodner, 1993, p. 4). Cognitive flexibility"the ability to spontaneously restructure one's knowledge, in many ways, in adaptive response to radically changing situational demands (both within and across knowledge application situations)" (Spiro & Jehng, 1990, p. 165). Cognitive flexibility theory -"a conceptual model for instruction that is based upon cognitive learning theory. Its intention is to facilitate the advanced acquisition of knowledge to serve as the basis for expertise in complex and ill structured knowledge domains" (Jonassen et al., 1992, p. 312). 14

PAGE 29

Collaborative learning-"Collaboration is a coordinated, synchronous activity that is the result of a continued attempt to construct and maintain a shared conception of a problem" (Teasley & Roschelle, 1993, p. 235). Collaboration involves a coordinated and mutual effort to problem solve (Teasley & Roschelle, 1993). Comprehension -results from fonnulating connections, rather than from placing infonnation or transfonning infonnation in memory. "The subtle difference lies in the creation of new understanding of the infonnation by the learner, rather than changing the presented infonnation" (Grabowski, 1996, p. 898). Computer-based microworlds -computer programs that are "a small but complete subset of reality in which one can go to learn about a specific domain through personal discovery and exploration" (Rieber, 1992, p. 94). Computer based microworlds pennit students to explore, experience phenomena, and fonnulate hypotheses (Hannafin, Hannafin, Hooper, Rieber, & Kini, 1996). Computer-based microworlds present infonnation, stimulate exploration, facilitate high level thought processes, and promote collaboration (Grabinger, 1996). Cooperative learning -cooperative learning involves placing students in small groups to facilitate working toward common goals (Johnson & Johnson, 1992; Nastasi & Clements, 1991 ). Cooperative work involves an activity where, through division of labor, each participant is responsible for a portion of the problem solving (Teasley & Roschelle, 1993). 15

PAGE 30

Generative learning activities -require students to work with, manipulate, and change information, relate information to existing knowledge structures, and use information to support problem solving (Dunlap & Grabinger, 1996). Generative learning theory-is based on the assumption that the learner is an active participant in the learning process where "comprehension and understanding result from the processes of generating relations both among and between experience or prior learning and new information" (Wittrock, 1992, p. 532). Ill-structured knowledge domains -knowledge domains in which "many concepts (interacting contextually) are pertinent in the typical case of knowledge acquisition, and their patterns of combination are inconsistent across case applications ofthe same nominal type" (Spiro et al., 1988, p. 375). Examples include history, biomedicine, and literary interpretation (Jacobson & Spiro, 1994). Inert knowledge"is knowledge that can usually be recalled when people are explicitly asked to do so but that is not used spontaneously in problem-solving contexts even though it is relevant" (CTG, 1992, p. 136). Inert knowledge is passively learned by students and available for tests, but not available for application in new situations (Bransford et al., 1989). Learning with understanding-Wittrock (1974a) writes: "Learning with understanding, which is defined by long-term memory plus transfer to 16

PAGE 31

conceptually related problems, is a process of generating semantic and distinctive idiosyncratic associations between stimuli and stored information" (p. 89). Mindfulness"the voluntary, controlled employment of nonautomatic processing operations ... .It is manifested in attention to details, in the careful examination of a problem's given conditions, in consideration of alternatives, in the generation of hypotheses and inferences, in the reading of text for deeper meanings, in linking new information to remote knowledge structures, in processes of abstraction and decontextualization, and the like" (Salomon, 1985, p. 213). Reductive bias -the tendency to oversimplify important aspects of complexity resulting in misconceptions of advanced material. Reductive bias "leaves learners without an appropriate cognitive repertoire for the processing of complexity" (Spiro et al., 1988). Spiro and his colleagues identify the following forms of bias: (a) oversimplification of complex and irregular structure, (b) overreliance on a single basis for mental representation, (c) overreliance on top down processing, (d) overreliance on precompiled knowledge structures, (e) context-independent conceptual representation, (f) rigid compartmentalization ofknowledge structures, and (g) passive transmission of knowledge. 17

PAGE 32

CHAPTER2 REVIEW OF LITERATURE Introduction Since the 1930s, competitive and individualistic learning has dominated all levels of education (Johnson & Johnson, 1992). This view of education (commonly called the traditional model of education) assumes learning is a passive, individual activity (Brown & Campione, 1996), where the teacher simply transmits knowledge to students as efficiently as possible (Bednar et al., 1992). To accomplish this, educators simplify instruction by neatly defining and prescribing subject matter domains in order to teach only the critical attributes of the domains (Duffy & Knuth, 1990). The traditional model of education, therefore, assumes (a) that learning involves forming simple associations based upon external reinforcement (Brown & Campione, 1996) and (b) that learning occurs most efficiently if the excess baggage of irrelevant content and context are eliminated (Bednar, Cunningham, Duffy, & Perry, 1995). Many researchers believe the traditional approach to education is obsolete (Bednar et al., 1992; 1995; Bransford et al., 1989; Bransford & Vye, 1989; CTG, 1992; Grabinger, 1996; Perkins, 1992; Reigeluth, 1995; Resnick, 1987; 1989; Scardamalia & Bereiter, 1994). A major criticism focuses on the assumption that 18

PAGE 33

learning occurs devoid of context. Whitehead (1929) believed the lack of relevant context provided by the traditional model of education produces inert knowledge. Inert knowledge is knowledge that students possess, but not used in problem-solving contexts, even when relevant, unless the students are specifically asked to do so (Bransford et al., 1989; Bransford & Vye, 1989; CTG, 1992). Inert knowledge exists as an "island of information," which while retrievable, provides little value to the learner for interpreting, modifying, or influencing performance (Hannafin, 1992). John Dewey similarly criticized the traditional approach to education because teachers present dry problems of little relevance to students (Phillips & Soltis, 1991 ). Dewey believed information presented this way is committed to memory as "static, cold-storage knowledge" and unless students have the opportunity to use this information in meaningful problem-solving activities, the information remains "sterile" (Phillips & Soltis, 1991 ). Removing the context, therefore, limits students' abilities to learn and apply concepts (i. e., successful learning). Successful learning requires more than literal encoding of formal instruction in a context-free environment (Hannafin, 1992). Successful learning occurs by engaging in activities that encompass the concepts students are trying to learn (Bransford et al., 1989; Brown, Collins, & Duguid, 1989; CTG, 1992; Spiro et al., 1988). Successful learning occurs when students retain, understand, and actively use the knowledge and skills learned-Perkins' goals of education-in the world beyond the school walls. Perkins (1992) defines these goals as generative knowledge; "knowledge that does 19

PAGE 34

not just sit there but functions richly in people's lives to help them understand and deal with the world" (p. 5). Generative knowledge requires existing knowledge to be evaluated concurrently with new knowledge and reconstructed accordingly, that knowledge be assimilated, and that perceptions of meaning, value, and importance be derived (Hannafin, 1992; Hannafin & Rieber, 1989). Generative knowledge therefore, is the antithesis of inert knowledge. Thus, how can educators promote the development of generative knowledge? What theories or instructional methods exist to promote the development of generative knowledge? Current cognitive theory (Bednar et al., 1992; 1995; Bransford et al., 1989; Bransford & Vye, 1989; Brown et al., 1989; CTG, 1992; Duffy & Knuth, 1990; Grabinger, 1996; Perkins, 1992; Jonassen, 1991; Resnick 1987; 1989; Spiro et al., 1988) emphasizes three essential and interrelated aspects of learning: (a) learning involves knowledge construction rather than knowledge absorption and recording, (b) learners build on current knowledge to construct new knowledge (i.e., learning is knowledge dependent), and (c) learning depends on the context or situation in which it occurs. Many researchers (Bransford et al., 1989; Brown & Campione, 1996; Grabinger, 1996; Hannafin, 1992; Jonassen, 1991; Spiro et al., 1988; Wilson, 1996) advocate creating student-centered learning environments that embody the three principles of learning just discussed. This chapter presents a selective review of the research and theory ofthree strategies designed to capture the complexities of learning and promote generative learning. 20

PAGE 35

The chapter begins by examining cognitive flexibility theory (CFT), a conceptual model for instruction that facilitates advanced knowledge acquisition (Spiro et al., 1988) by activating generative learning processes. The chapter then moves to a discussion of generative learning theory, which assumes that learners actively participate in the learning process by constructing meaningful understanding ofinformation found in the learning environment (Wittrock, 1974a; 1974b ). Next, research on collaborative learning is reviewed. The reviews of literature provide the rationale for the development of the student-centered computer programs used in this study. The chapter concludes by restating the research questions and offering the hypotheses of interest. Cognitive Flexibility Theory: Assumptions and Theory Spiro et al. ( 1988) believe that schools simplify representations of knowledge to allow students to see the main point. Simplifying the content removes the context, which rather than simplifying the concept, results in teaching a different concept. Spiro and his colleagues (1988) argue that case-based learning captures the complexity of real experiences. Students work through several cases examining the concept from different points of view in different contexts. This allows the students to see multiple contexts in which the concept occurs and applies, and thus index its meanmg. 21

PAGE 36

Cognitive flexibility theory is based on two elements: (a) that learning occurs along a continuum from novice to expert and (b) that knowledge domains can be defined by two conceptual characteristics increased complexity and increased ill structuredness (Jacobson & Spiro, 1994; Spiro et al., 1988; Spiro, & Jehng, 1990). Cognitive flexibility theory posits that knowledge acquisition follows a continuum from beginner to expert, but focuses on the intennediate or advanced learning stage. Introductory learning typically involves learners with very little transferable knowledge in a content area (Jonassen & Grabinger, 1993). The goal of instruction for beginning learners involves providing a general orientation and exposure to content followed by objective assessment measuring recognition and recall of facts (Spiro et al., 1988). In contrast, during the intennediate stage of learning, learners acquire advanced knowledge in order to solve complex domain-oriented problems (Jonassen & Grabinger, 1993). During the advanced learning phase the learning goals shift to (a) mastery of content complexity (i.e., acquisition of the conceptual complexity necessary for understanding important concepts) and (b) knowledge applicability (i.e., the ability to adaptively transfer knowledge to novel and realistic situations) (Spiro et al., 1989). Advanced knowledge acquisition, therefore, involves attaining a deeper, richer understanding of content material, as well as the ability to intelligently reason with and apply it in diverse contexts (Jacobson & Spiro, 1994; Spiro et al., 1988). 22

PAGE 37

The second distinguishing element of cognitive flexibility theory involves the conceptual characteristics of increased domain complexity and increased ill structuredness (Jacobson & Spiro, 1994; Jonassen et al., 1992; Spiro et al., 1987; 1988; 1989; Spiro & Jehng, 1990). Complexity occurs when the domain content places unusual demands on cognition as compared to the cognitive loading in the introductory phase of learning (Feltovich et al., 1989). Complexity can result from (a) task multidimensionality, which places unusual or excessive demands on working memory, (b) the abstractness of the content or the semantic difference between concepts and their formal symbolic representation, (c) heavy reliance on the learner's prior knowledge (which may be faulty or at odds with the concepts), or (d) irregular, inconsistent concepts or the interaction of many concepts (Feltovich et al., 1989; Jonassen et al., 1992). Related to domain complexity is the notion of ill-structuredness (Jacobson & Spiro, 1994; Jonassen et al., 1992; Spiro et al., 1987; 1988; 1989; Spiro & Jehng, 1990). "By ill-structuredness we mean that many concepts (interacting contextually) are pertinent in the typical case of knowledge acquisition, and that their patterns of combination are inconsistent across case applications of the same nominal type" (Spiro et al., 1988, p. 3 75). Characteristics of an ill-structured knowledge domain include the following: no general rules or defining characteristics exist that cover most cases, inconsistent hierarchies of relationships exist among cases, 23

PAGE 38

the same features of cases assume different levels of importance in different contexts, no prototypic cases exist or they are misleading, and interactions among features cause case novelty. (Spiro et al., 1987; Jonassen et al., 1992). Ill-structured knowledge domains place demands on learners that are at odds with the cognitive modes and instructional practices appropriate for introductory learning (Feltovich et al., 1989). The notion that advanced learning differs from introductory learning with respect to goals, complexity, and ill-structuredness suggests different approaches to learning are necessary. However, all too often the same teaching methods and tactics used for initial knowledge acquisition are also used for intermediate learning. Learning and instruction for mastery of complexity and application in a complex and ill-structured domain cannot be compartmentalized, linear, uniperspectival, neatly hierarchical, simply analogical, or rigidly prepacked. Yet it much too often is, and the result is the development of widespread and serious misconceptions and difficulties in knowledge application. (Spiro & Jehng, 1990, p. 168) Teaching methods geared toward introductory learning lead to oversimplification (reductive bias) of complex material (Spiro et al., 1988) which contributes significantly to examples of learning failure (Spiro et al., 1988; Jacobson & Spiro, 1995). Cognitive flexibility theory offers several principles designed to overcome 24

PAGE 39

reductive bias and promote the goals of advanced knowledge acquisition (Spiro et al., 1988): 1. Avoid oversimplification and overregulation. Advanced knowledge acquisition requires that learners be made aware that knowledge is not simple and orderly, but complex and irregular. "It is important to lay bare the limitations of initial, first pass understandings, to highlight exceptions, to show how the superficially similar is dissimilar and superficial unities are broken" (Spiro et al., 1988, p. 377). Additionally, rather than decomposing and reassembling information, instruction should reflect the intricate patterns of component interactions. 2. Provide multiple representations of content. Single depictions of complex and ill-structured knowledge will miss or misrepresent important aspects of complex concepts. Cognitive flexibility requires the use of multiple themes, concepts, analogies, and points of view (Jacobson & Spiro, 1995) to provide a diversified repertoire ofways of thinking about a conceptual topic (Spiro et al., 1988). 3. Use and emphasize cases. In complex, ill-structured knowledge domains, great variability may exist between cases concerning the applicability of relevant concepts. Cases may not be linked together by general principles, therefore, multiple cases are necessary to illustrate abstract concepts associated with ill-structured knowledge domains. "The more variegated 25

PAGE 40

these cases are, the broader the conceptual basis that they are likely to support" (Jonassen et al., 1992, p. 312). 4. Emphasize conceptual knowledge as knowledge in use. In an ill-structured knowledge domain the way a concept is used or applied may vary greatly across cases, which makes it more difficult to extrapolate the concepts from the features of the cases. Therefore, "if a concept's meaning cannot be determined universally across cases, then one must pay attention to the details of how the concept is used-knowledge in practice, rather than in the abstract" (Spiro et al., 1988, p. 380). 5. Emphasize knowledge construction rather than knowledge transmission. Given the emphasis placed on case-based learning and knowledge in use, learners must be able to assemble (construct) meaningful knowledge representations in order to adaptively fit the situation at hand. In other words, the complexity and irregularity inherent in ill-structured domains requires flexible, recombinable knowledge structures. Thus the "storage of fixed knowledge is devalued in favor of the mobilization of potential knowledge" (Spiro et al., 1987, p. 181 ). Knowledge construction implies active learner involvement in knowledge acquisition accompanied by expert guidance and cognitive support. 6. Emphasize noncompartmentalization of cases and concepts. The complexity and irregularity inherent in the cases and examples in ill-structured 26

PAGE 41

knowledge domains precludes compartmentalizing knowledge. "Rather than mapping knowledge onto the learner, the learner must map contexts onto his/her own knowledge as it is being acquired in order to support the transferability of knowledge" (Jonassen et al., 1992, p. 313 ). Multiple interconnectedness and differing thematic perspectives among different cases and concepts enable the situation-dependent and adaptive schema assembly that promote knowledge transfer (Spiro et al., 1988). Cognitive flexibility theory prescribes using a case-based instructional approach that provides access to thematic information structures to create learning environments that stimulate the development and application of flexible knowledge structures. Such environments stimulate the mindful processing of information (Salomon, 1985) necessary for generative learning to occur. In sum, by focusing on students generating flexible, usable knowledge in an information-rich learning environment, cognitive flexibility theory is a generative activity. Research on Cognitive Flexibility Theory Evidence validating the effectiveness of CFT is generally positive in the area of higher level thinking skills. For example, Spiro et al. (1987) conducted two studies with high school students in which control groups studied the same cases as the experimental groups, but the cases were presented linearly, while the cases for the 27

PAGE 42

experimental groups contained case-to-case linkages. Control group learners outscored experimental group learners on reproductive tests of memory, but the experimental groups outscored the control groups on six different tests of application and transfer. Spiro and his colleagues (1987) concluded that conventional methods produce better results on traditional tests stressing rote memorization, but if the goal of education is generative knowledge, "then it would seem that methods like ours are far preferable to the conventional ones" (p. 191). Hartman and Spiro (1989) expanded on this research effort by studying the effects of multiple perspectives, flexible representation, and assessment of transfer. They found no differences in reproductive memory, but in knowledge application and transfer, the flexibility group significantly outperformed the control group. Finally, Jacobson (1990) compared a linear, computer-based drill treatment with an experimental treatment emphasizing multiple representations, linking abstract ideas to case examples, and the interrelationships between surface and structural knowledge. Again the drill group recalled more facts, but the experimental group attained higher scores on all transfer tests ofknowledge. More recent investigations concentrate on expanding our understanding of cognitive flexibility theory and computer-based microworlds. For example, Jacobson & Spiro ( 1995) investigated the effects of learning from a minimal hypertext/drill condition versus a cognitive flexibility hypertext. The study consisted of two main parts. The first part was a reading stage where both groups read the same 28

PAGE 43

instructional content. The second stage consisted of a study stage, in which the control group completed a computer-based drill program based on the facts and concepts of the reading stage, while the experimental group received a hypertext treatment stressing knowledge interrelationships and knowledge assembly. The results revealed that the control group performed higher on measures of memory for factual knowledge, while the experimental group demonstrated greater knowledge transfer. These research efforts appear to validate the efficacy of cognitive flexibility theory; instruction based upon CFT principles promotes development of certain types of generative knowledge. The studies demonstrate that CFT -based learning environments require students to manipulate and relate information to existing knowledge structures which supports the problem-solving and application processes endemic to advanced knowledge acquisition. However, opportunities for deep learning are not always fully taken advantage of, even when presented as unique learning environments (Salomon, 1985), such as a cognitive flexibility learning environment. In other words, although a learning environment may be based upon cognitive flexibility theory (or some other student-centered strategy) students may not employ the mindful processes required of generative learning. Given the necessity to activate mindful processes for generative learning to occur, the question becomes: Would the integration of compatible generative learning activities into a CFT -based learning environment promote deeper processing of material, and hence, increased 29

PAGE 44

knowledge acquisition by advanced learners? The next section of this chapter discusses generative learning theory and collaborative learning, two generative activities that, in combination with cognitive flexibility theory, should increase learning by advanced learners. Generative Learning Theory Generative learning theory provides an understanding of the process of comprehension (Wittrock, 1992). Comprehension requires learners to formulate connections between the different parts of the information being perceived and between that information and what exists in memory (Wittrock, 1985). This causes learners to reorganize, elaborate, and/or reconceptualize information rather than stuff more information into memory which results in meaningful learning and comprehension (Grabowski, 1996). The important pedagogical point is that knowledge cannot be transferred directly from the teacher to students, but has to be created within each student (Harlen & Osborne, 1985). Learning with understanding, therefore, requires intellectual effort on the part of the learner to generate relationships between stimuli and stored information (Osborne & Wittrock, 1985). Wittrock (1992) sums up generative learning theory in the following manner: 30

PAGE 45

At the essence of this functional model are generative learning processes people use actively and dynamically to (a) selectively attend to events and (b) generate meaning for events by constructing relations between new or incoming information and previously acquired information, conceptions, and background knowledge. These active and dynamic generations lead to reorganizations and reconceptualiztions and to elaborations and relations that increase understanding. (p. 532) Wittrock (1990) believes that there are two categories of activities that stimulate generative processes. The first category, designed to stimulate processing between the different parts of the information being received, includes activities such as composing titles and headings, writing questions, stating objectives, writing summaries, drawing graphs and tables, and constructing main ideas. The second category consists of demonstrations, metaphors, analogies, examples, problem solving, explanations, paraphrases, and inferences which generate relationships between the external stimulus (instruction) and prior knowledge. These categories can be used in either a teacher-provided or learner-generated format (Grabowski, 1996; Wittrock, 1990). For example, the teacher can provide summaries or ask the learners to summarize. However, Wittrock (1974a) cautions that understanding cannot be given directly to students; if understanding is to occur, students must make the connections themselves. Di Vesta ( 1989) concurs: What is learned is not necessarily that which is stated in the title of the course, what is described in the curriculum or syllabus, or the behavior described in a behavioral objective or the content of the course defined by texts, assignments, delivery system, and curricular materials. What is learned depends on processing requirements that are actually carried out by the learner. (p. 55) 31

PAGE 46

In summary, generative learning theory posits that learners put forth intellectual effort to actively construct knowledge by generating relationships (a) between the different parts of the incoming information and (b) between the new information and prior knowledge and other memory components (Grabowski, 1996; Wittrock, 1974a; 1974b; 1985; 1990; 1992). Unfortunately, most of the research on generative learning theory concentrates on reading comprehension and individual students. Fortunately, the model emphasizes transforming static information (text) into flexible, usable knowledge (Grabinger, 1996), thus there may be transferability to other student-centered strategies designed to increase learning. Compatibility of Cognitive Flexibility Theory and Generative Learning Theory Cognitive flexibility theory advocates constructing/applying knowledge in a multiple case format to encourage development of flexible representations of content (Spiro et al., 1988). Learners manipulate the content information to make the multiple connections between and among the information and prior learning. Thus cognitive flexibility theory is a generative activity and conceptually, very similar to generative learning theory. However, the two models differ with respect to practice. First, there are obvious contextual differences between the two theories. Cognitive flexibility theory advocates using multiple cases in a complex learning environment to increase learning, whereas, generative learning theory prescribes specific strategies designed to increase student reading comprehension and understanding of text. 32

PAGE 47

Second, Spiro and his colleagues specifically target the advanced stage of learning for CFT application, while generative learning theory application appears unrestricted regarding the stages of learning. Given the conceptual similarities, but practical differences, how might cognitive flexibility theory and generative learning theory interact to affect learning? The limited cognitive flexibility theory research discussed earlier demonstrates the effectiveness of cognitive flexibility theory in promoting advanced knowledge acquisition. Since generative learning theory offers strategies designed to make the learning processes explicit (i.e., stimulates mindful processes) and increase comprehension and understanding, it is reasonable to expect that generative learning strategies would enrich a cognitive flexibility learning environment. In other words, using strategies to stimulate one or both of the generative processes should help advanced learners develop the rich knowledge structures cognitive flexibility strives to promote. The strategies of interest in this study are summarization, which facilitates making organizational connections between the different parts of the new information, and explanation, which supports making connections between prior knowledge and the new information. 33

PAGE 48

Individual Student Focus One interesting observation concerning the theories of learning just reviewed-especially cognitive flexibility theory-is the focus on the individual student. Given the proclivity to use computers to deliver CFT -based instruction, the focus on students interacting individually with computers to learn the respective subject domains is especially intriguing since much, if not most, research on computer-based learning focuses on collaborative learning. The concentration on the individual learner within the cognitive flexibility theory literature may reflect the initial empirical evaluation of cognitive flexibility theory, or the assumption that advanced learners (the stage of learning hypothesized to benefit from CFT-based instruction) learn best individually, or perhaps it is based on the belief that computers represent the ideal medium to individualize instruction. Whatever the reason, the reality is that students often work in small groups at computers (Hooper, 1992). Therefore, an investigation into the effects of collaborative learning on cognitive flexibility theory is warranted, especially since social interaction is such an integral part of a student-centered learning environment (Resnick, 1987; Wilson, 1996). Collaborative Learning At the outset it is important to make the distinction between the terms "cooperative learning" and "collaborative learning" because they differ with respect to how they encourage student interaction. Cooperative learning refers to group 34

PAGE 49

learning approaches where a division of labor among participants requires individual students to take responsibility for different portions of the task (Crook, 1994; Damon & Phelps, 1989; Stodolsky, 1984; Teasley & Roschelle, 1993). Collaborative learning, on the other hand, involves students working jointly in a coordinated effort on the same problem (Damon & Phelps, 1989; Teasley & Roschelle, 1993). In both cases, the grouped students share a common goal, interact, and contribute to the group activity (Stodolsky, 1984), but the differences in task structure result in differences in equality and mutuality (two indexes of peer engagement). "Equality means that both parties ... take direction from one another rather than one party submitting to a unilateral flow of direction from the other; and mutuality means that the discourse in the engagement is extensive, intimate, and connected" (Damon & Phelps, 1989, p. 1 0). According to Damon and Phelps ( 1989), cooperative learning is high on equality, but variable on mutuality due to the task subdivision that requires learners to accomplish much of their work individually. In contrast, students work jointly on the same problem in a collaborative environment creating "an engagement rich in mutual discovery, reciprocal feedback, and frequent sharing of ideas" (Damon & Phelps, 1989, p. 13). The higher degrees of mutuality and equality present in a collaborative environment results in interaction between (not summation of) the understandings of the students which leads to emergent knowledge (Whipple, 1987). Collaborative learning is rooted in the assumption that knowledge is socially constructed and that social interaction is necessary for learning to occur (Vygotsky, 35

PAGE 50

1978). Two different theoretical perspectives offer possible explanations for the cognitive growth associated with collaborative learning. The first, socio-cognitive theory, posits that interindividual conflict facilitates cognitive growth (Nastasi & Clements, 1992; 1993). Development occurs as students enter into a conversational process of negotiation and justification in an attempt to resolve differences of agreement (Crook, 1994; Nastasi & Clements, 1992; 1993). As the students reevaluate their positions and seek new information to resolve conflict, they learn the material better (Webb, 1987) and restructure old knowledge structures into new knowledge structures (Hanafin, 1992). Co-construction is the second theoretical explanation for improved achievement. In this case interpersonal conflict is absent and partners integrate their differing task conceptualizations into a mutual plan for solving a problem neither could solve alone (Nastasi & Clements, 1993). Thus collaborating students scaffold or guide and correct each other and build on each other's ideas until they reach the solution. In either case, the process of collaborating requires students to make their thinking public and explicit (Crook, 1994), which facilitates intellectual growth through a process of declaring and justifying ideas, opinions, and interpretations (Damon & Phelps, 1989). Collaborating students, therefore, "retrieve prior knowledge, seek new information, evaluate their own and others' answers, ideas, and opinions, confront their own misunderstandings and lack of knowledge, and as a consequence, restructure their thinking" (Webb & Lewis, 1988, p. 181) which leads to higher levels of understanding. 36

PAGE 51

Research on Collaborative Learning Literally hundreds of studies and research reviews exist and most demonstrate the value of collaborative learning. For example, meta-analyses comparing the effects of small group and individual learning environments on achievement indicate effect sizes of .75 (Johnson, Maruyama, Johnson, Nelson, & Skon, 1981) and .63 (Johnson & Johnson, 1989) favoring small group learning. In addition to improved learning, numerous other studies demonstrate that collaborative learning positively benefits the social-emotional (positive relationships and psychological health) aspects of instruction (Johnson & Johnson, 1992). The cognitive and social benefits documented for traditional classroom use of collaborative learning appear to transfer to computer-based learning environments as well. Some studies demonstrate significantly higher achievement for collaborative groups (Dalton et al., 1989; Hooper et al., 1993; Johnson et al., 1985; 1986), while others find no significant differences between individuals and small groups (Carrier & Sales, 1987; Makuch et al., 1992; Webb, 1985). Webb (1987) reviewed 14 studies comparing achievement between group and individual computer work and found similar results; only five studies found differences favoring the group activity. However, Webb concludes: The important result is that no study found greater learning among students working alone than students working in groups. This suggests that, averaging over all students, group work is not detrimental to students' learning, and may be beneficial. (p. 195) 37

PAGE 52

For example, groups spend more time interacting on task (Johnson et al., 1985) and express more positive attitudes toward working at computers (Hooper et al., 1993). Although variable, these results mirror the positive results found for collaborative learning in the non-computer classroom. Thus, computer-based collaborative learning appears to be an enriching experience and may increase achievement. Given the volume of studies investigating the effects of computer-based collaborative learning, as well as the large number of factors involved (age, experience with computers, the computer activity, achievement measures, etc.) it is no wonder that results vary. This disparity of results notwithstanding, the general consensus is that the collaborative use of classroom computers is the preferable implementation strategy (Watson, 1990). One reason for grouping students at computers involves solving logistical problems (Hooper, 1992); in many instances, there simply are more students than computers. A second and more educationally sound reason for grouping students, rests on the assumption that learning occurs in a social context where learners interact and draw on one another's expertise (Krajcik, Blumenfeld, Marx, & Soloway, 1994). In the latter case, grouping students at the computer is based on the roles that communication and interaction play in creating meaning, while in the former case, grouping students may be based solely on classroom expediency. In both cases, however, teachers expect some educational benefit which presumably results from the quality of communications and interactions that occur between the grouped students. 38

PAGE 53

In summary, considerable evidence exists demonstrating the efficacy of collaborative learning, both in a computer-based learning environment and the non computer classroom. The benefits of collaborative learning result from the active engagement of students in an equal and mutual discourse of conversation and communication where learners make connections between incoming information and between the incoming information and prior knowledge. Clearly collaborative learning is a generative activity. Despite the considerable literature devoted to computer-based collaborative learning, most of this research is conducted with courseware designed for individual use (Jonassen, 1988). Additionally, computer-based learning is dominated by strategies that facilitate only superficial levels of processing (Hooper & Hannafin, 1991 ). These types of software may be inappropriate for group use (Hooper, 1992) and may explain the variability of results found for computer-based collaborative learning. Several researchers (Hooper, 1992; King, 1989; Nastasi & Clements, 1992; Webb, 1987) call for research efforts to investigate how to facilitate and strengthen the use of effective collaborative processes. It is my contention that combining strategies designed to engage students in generative learning activities should facilitate student interaction and increase achievement. 39

PAGE 54

Research Questions Several themes emerge from the literature review just concluded. First, although limited at this point, research supports the central assertion of cognitive flexibility theory that case-based learning produces greater knowledge acquisition and transfer than a linear, computer-based approach to learning the same material. Could student achievement be improved by modifying a CFT-based computer program with additional generative strategies? Additionally, the cognitive flexibility theory literature only evaluates students working individually in the CFT -based environment. What effect, if any, would pairing students have on the effectiveness of a CFT -based computer microworld? Second, research demonstrates that generative learning strategies are effective for increasing comprehension (Di Vesta & Peverley, 1984; Doctorow, Wittrock, & Marks, 1978; Grabowski, 1996; Johnsey, Morrison, & Ross, 1992; Linden & Wittrock, 1981; Stein & Bransford, 1979; Wittrock, 1990; 1992; Wittrock & Alesandrini, 1990). However, most generative learning studies focus on learners working alone. Given the evidence demonstrating the effectiveness of collaborative learning, might collaborating pairs generate more effective explanations and summaries and therefore, reach higher levels of knowledge acquisition than single students? 40

PAGE 55

Finally, a common theme running through the vast literature on computer-based group work is the intense focus on children. Energetic computer-based group work has been most thoroughly documented within the earliest years of education ... and in the middle and secondary years .... There is very little relevant research on this topic that considers further and higher education or work in training communities. (Crook, 1994, p. 124) Given the relative paucity of research regarding adult-oriented, computer-based collaborative learning, is collaborative learning effective for advanced learners? Also, can the collaborative process, and hence achievement, be improved by integrating the opportunity for collaboration with other generative activities, such as specific generative learning strategies or CFT -based computer learning environments? This study was designed to address these questions by assigning single and paired students to work on two qualitatively different CFT -based computer programs (a base program and a program enriched with additional generative learning strategies). Research Hypotheses The major research question involved investigating the effects of two generative learning activities (generative learning theory and collaborative learning) on the performance of advanced learners in a cognitive flexibility-based computer microworld. Two important questions emerge concerning the integration of cognitive flexibility theory, generative learning theory, and collaborative learning: 41

PAGE 56

(1) Would the inclusion of embedded explanation and summary cues (generative learning strategies) result in differences in learning in a CFT-based computer microworld? (2) How would the addition of collaborative learning affect learning in this environment? Based on the literature, it was hypothesized that: (1) Students assigned to the generative program would out perform students assigned to the base program on the posttest. (2) Students assigned to the collaborative condition would out perform students assigned to the individual condition on the posttest. (3) There would be an interaction effect between the version of program and method of instruction treatment variables as measured by scores on the posttest. A synergy would develop between the two treatment variables such that students assigned to the pair/generative treatment would perform significantly better on the posttest relative to the other treatment groups. 42

PAGE 57

CHAPTER3 METHODOLOGY Introduction This study used a computer-based learning program to provide instruction in the complex domain of transfusion medicine. This program integrated multiple cases with access to relevant information sources to encourage medical students to diagnose, assess, and manage infectious and non-infectious adverse events of recipients, and donor-related blood transfusion problems. The present study investigated the relative effects of complementary generative activities (generative learning strategies and collaboration) on the knowledge acquired by second-year medical students using a cognitive flexibility theory-based computer microworld. The remainder of this chapter discusses the materials, participants, research design, instrumentation, and instructional procedures used in this study. Materials Cognitive flexibility theory addresses knowledge acquisition in complex and ill defined subject domains. The broad field of medicine is considered such a domain, as are the specialty fields within medicine (Spiro et al., 1988). Transfusion medicine is: 43

PAGE 58

That area of medicine which includes the collection, production, storage and administration of lymphohematopoietic progenitors, mature blood cells, plasma and plasma components, as well as associated services to be used in treatment of patients with specific diseases. (D. Ambruso, personal communication, June 15, 1998) Transfusion medicine is a multidisciplinary area encompassing a number of basic sciences (e.g., biochemistry, cell biology, transplant biology, and immunology), clinical sciences (e.g., hematology, medicine, surgery), and some applied sciences (D. Ambruso, personal communication, June 15, 1998). Due to the large number of interrelated medical and science domains, no prototypic cases exist. Causes and effects interact and produce a variety of symptoms or require a number of different medical interventions that depend on the clinical status of the patient, the purpose of the transfusion, and from whose perspective the case is viewed (Jonassen et al., 1992). Because of the complexity and ill-structuredness of this knowledge domain, a medical school professor responsible for teaching this curriculum, led a team of physicians, educational consultants, and instructional designers (Transfusion Medicine Team) to develop "Handling Transfusion Hazards," a computer program based upon cognitive flexibility theory principles. The Program: Handling Transfusion Hazards Diagnosing and treating transfusion-related problems is a dynamic, problemsolving activity. Clinicians collect information, order and interpret lab tests, and assess and manage clinical problems in their patients, including defining the 44

PAGE 59

indication for use of blood transfusions. "Handling Transfusion Hazards" instantiates this process by providing medical students with realistic cases to solve, as well as the tools necessary to solve the cases. The program consists of four distinct parts: (a) program introduction, (b) pretest, (c) learning phase, and (d) posttest (see "Instruments" this chapter for descriptions of the pretest and posttest). The introduction provides information to help the students successfully complete the program. This orientation (a) discusses the program content and learning objectives, (b) provides a concept map (i.e., diagram) that illustrates and explains each part of the program, (c) and informs the students of the requirement to complete the program in its entirety by sequencing through the pretest, practice cases, and posttest. The learning phase contains six practice cases (scenarios) that require the subjects to order laboratory tests (if appropriate) to help them form a diagnosis and then select suitable assessment and management options. The cases require the students to access relevant factual and conceptual information which is then applied to solve the transfusion problems at hand. Each case contains (a) a description of the case, (b) a research section, (c) an actions section, and (d) a help section (Figure 3.1 presents a sample case screen). These sections are described below. 45

PAGE 60

Figure 3.1. Sample Case Screen from Practice Case Three l's Sattrday morning, July 5, and you r&c&iv& a frartic pag& rrom th& drawing c&nt&ratyour hospital blood bari c Your hospital rG>I.tlnety draws volunte&r blood donors r or processing by the local cOil"fTTunity blood certer and the lnt&rn on duty provides medical coverage. A donor has just rairt&d and your h&p is r&quir&d irrrn&diat&ty. You br&athl&ssty approach th& donor room to rind a pale, SW&aty donor tying on a cot. Actions: ( Order Lab Tests ) ( Assess Case ) ( MenageCase ) ..... ""' ) ' \ Cases Case summary: Upon selecting a case, students are presented with the case summary which provides basic contextual information (see Fig. 3.1 ). Research: The research section contains thematic information sources that provide structural information relevant to the case. 46

PAGE 61

History/physical: This section provides the basic patient background infonnation necessary for the learners to begin the diagnostic process. Figure 3.2 presents a sample history/physical screen. Figure 3.2. Sample History Screen from Practice Case Three .. 47

PAGE 62

Perspectives: Each case contains opinions from several significant operatives, such as the attending physician, resident, phlebotomist, blood bank director, surgeon, pediatrician, internist, fellow, donor, recipient, gastroenterologist, and patient. Usually four or five perspectives are available based on the relevance of the information to the case content. Figure 3.3 presents a sample perspectives screen. Figure 3.3. Sample Perspectives Screen from Practice Case Three I PRACTICE CASES: Perspective or Phlebotomist This donor had a normallrtervlew He Is a l'i'sttine donor and came to the blood bank this morning because his mother had an operation a week ago and he warted to replace the blood she had used He appeared nervous and said he has been arrald or needles since he was a child We did a normal <:taw and t wert well, blt When he got up from donating, he said he ret weak, collapsed and lost consciousness. He regained consciousness
PAGE 63

Similar cases: The program contains a bank of 24 cases that are less detailed than the primary learning cases. These cases provide information pertinent to the selected practice case. The similar cases menu provides two choices: closely related similar cases and other suggested similar cases. Closely related similar cases share the following attributes with the practice case: group involved, type of risk, pathophysiology, symptoms, and screening tests; whereas, the other suggested similar cases share only one or two of the attributes above. Figure 3.4 shows a sample similar cases screen. Figure 3.4. Sample Similar Cases screen from Practice Case Three N G is a 56-year-old female altolo9us blood donor who sustained a severe reaction during donation 49 c .. e Hiol ory ( Phy$icel Le.b Dete.

PAGE 64

Help: The help section provides three choices: textbook, quit, and map. Textbook: The program contains a transfusion medicine textbook that provides detailed information on donor and recipient transfusion issues, as well as access to a glossary of medical terminology. Figure 3.5 presents the textbook index screen. Figure 3.5. Textbook Index Screen I TEXTBOOK: Index A$ses,.,..ent Repis(nlicn Rlf/WI/I'Imlsltdomlion Ocnt()/fflicn Ret:ipiml !Yfflion Pl>pl/llb()/cm,;wrl DONOR RISKS Generel reections to donation A#druclians Al4dlfl!llr14l:titn$ Se>'ef'UIIJclitJn$ ""'"' 41>11111>/ur domliom Clrffletldomliom Recipient Issues REOPIENT PROCESSING '1.4CI:b4Jt:t" REOPIENT RISKS Wectious He{)41m8 He,..isC allw"uus clllep41Kis Sur4pite Tlsli19 RetrC>Wuses H;'t Hl''-.t,. HTL;'l HTL ;'lf 5)-pl/ils Hernot,ijc Reections O&,;rdHTR lmlll1ic"list s GLOSSAAYOFTERMS Quit: The quit function allows the students to exit the program. If selected this option saves the student's work up to, but not including the current case. 50

PAGE 65

Map: This function takes the students to the conceptual map contained in the program orientation. The research resources and textbook contain the declarative (factual) and conceptual knowledge required to support the students' problem-solving processes; as such, they remain available throughout the diagnosis and treatment phases (labs, assessment, and management) which are described below. Actions: Each case requires the students to order lab tests (if appropriate), assess, and manage a transfusion medicine problem. Laboratory tests: Students have the option of ordering many different types of laboratory tests to help them develop and confirm their diagnosis. Upon selecting the lab test portion of the program, students are given the option to choose from five different types of lab tests: chemistry, hematology/coagulation, blood bank/serology, microbiology/urinalysis, and diagnostic procedures/radiology. Each of these test types contain between 4 and 20 tests and procedures. Upon selecting a test, students receive immediate test results (a test value) and feedback concerning the appropriateness of the test. After ordering all ofthe tests for a given type (e.g., chemistry), the students have the option to view a list of suggested tests. 51

PAGE 66

Assessment and Management: Assessment and management actions proceed in the same fashion. Students choose their courses of action and receive immediate feedback concerning the advisability of each action taken. Then they have the option to view the correct actions. After the students complete the management actions, each case ends with a summary of the learning outcomes specific to that case. "Handling Transfusion Hazards" facilitates knowledge acquisition in the complex domain of transfusion medicine by giving students practice solving actual clinical problems. This program achieves its objectives by combining case-based instruction with the structural support of thematic information resources (patient's history/physical, a transfusion medicine textbook, perspectives of several case relevant operatives, and other prototypically similar cases). Specifically, this program: 1. avoids oversimplifying instruction since no single case is prototypic, 2. provides multiple representations through the varied nature of the cases and different sources of information available for processing the cases, 3. supports context-dependent knowledge acquisition by providing real life cases, as well as realistic information sources, and 4. emphasizes knowledge construction through the analytical processes built into the program. Students must access different sources of declarative 52

PAGE 67

and conceptual knowledge to solve the cases. In the process the students build complex schemata consisting of procedural (how to) knowledge rather than an assortment of unrelated facts (Jonassen et al., 1992). In summary, "Handling Transfusion Hazards" fosters contextualized reasoning (Resnick, 1987). The program provides medical students with the opportunity to scientifically reason and solve real-world medical problems just as practicing clinicians would, and therefore, is clearly a generative activity (see Appendix A for a copy of a practice case) Program Development and Evaluation An extensive development and formative evaluation process was used to develop "Handling Transfusion Hazards." This process took almost two years and is detailed below. First, the content topics in transfusion medicine were identified by the physicians responsible for teaching this curriculum and organized under the theme of risk management to donors and recipients. Selection of these topics was guided by learning objectives established for the transfusion medicine curriculum. Second, this information was organized into a database by the Transfusion Medicine Team according to adverse events (for example, Hepatitis A, Hepatitis B, HIV, etc.) Cognitive flexibility theory was selected as the 53

PAGE 68

theoretical framework for presenting this information in a computerized, case-based format for two reasons: (a) the learning objectives involve application and transfer of information and (b) cognitive flexibility theory provides the means to create an authentic environment that promotes critical thinking and problem solving (additional educational outcomes the Transfusion Medicine Team wished to facilitate). Third, the practice and test cases were developed based upon actual clinical cases. Issues considered at this point included the number of cases to use, the appropriate resources (textbook, etc.) necessary for the students to solve the cases, and the relationships between the cases. Fourth, a complete working program was developed leading to formative evaluation of the program by users (medical students, residents, and fellows) and expert review of content. Formative evaluation was conducted on individual practice cases and the total program to check for proper operation of the program. Content review was conducted by two transfusion medicine experts (not connected with the project) who checked the content and operation of the program for accuracy and completeness. Finally, changes suggested by the expert reviewers were incorporated. This study used two versions of this program: the version described above (base) and a generative version. The generative program includes embedded generative 54

PAGE 69

learning strategies (explanation and summarization cues) that require the students to explain their decisions and to summarize the main concepts of each transfusion medicine case. Providing explanations enables students to reorganize and clarify their ideas which helps them recognize gaps in understanding and increase learning (Webb & Lewis, 1988). Summarization involves paraphrasing the information to be learned and through further encoding, helps students to consolidate and strengthen what was learned and provide feedback concerning the degree of understanding achieved (Hooper, 1992; Yager, et al., 1985). These strategies should help the students make the connections between the multiple contexts in which concepts apply leading to greater knowledge acquisition. The generative program version requires students to justify (explain) their decisions when ordering lab tests, and assessing and managing each case. After explaining their actions, the students can view the results, corrective feedback, and the correct actions just as they would in the base version. Finally, before proceeding to the next case, the program prompts the students to summarize the main concepts contained in the case, and to highlight any similarities or differences with other cases (Appendix B contains examples ofthe generative version screens). The base program was modified to produce the generative program version by the author with the help of a computer programmer affiliated with the Transfusion Medicine Team. The introductory information, instructional content, and lesson sequencing remained the same. The only changes involved embedding the 55

PAGE 70

explanation and summary cues and providing the screen space necessary to respond to the cues. Since the changes encompassed how the program functions, and not program content, the author conducted an operational evaluation to ensure the reliable operation of the generative program. Participants One hundred and thirty-two second-year medical students, 70 men and 62 women, participated in this study. These subjects were enrolled in Pathophysiology of Disease, a required course for all students attending a major medical university located in a large western city. The subjects completed "Handling Transfusion Hazards" during the three-week hematology rotation of the Pathophysiology of Disease course. All subjects completed the program; however, thirty-one students were excluded from the final analysis due to missing or inaccurate data and noncompliance with study conditions (see Chapter Four, Data Collection and Analysis Screening Procedures). Thus, one hundred and one students were included in final statistical analysis. Assignment to Treatment Effective interaction stimulates "mindful learning" (the employment of nonautomatic volitional and metacognitively guided processes ofleaming) resulting in greater cognitive effort and deeper processing of information (Salomon, 1985; 56

PAGE 71

Salomon & Globerson, 1987; 1988). Interdependence must be established for students to interact effectively (Johnson & Johnson, 1992); however, simply placing students together and asking that they work as a team may not establish the level of interdependence required for a good collaborative effort. Salomon and Globerson (1988) write: A team is a social system, and as such it is a qualitatively different entity than a few individuals working alone side-by-side. Behaviors and cognitions in the group have two major characteristics: they become interdependent and this interdependence develops over time in a reciprocal manner. This developing interdependence implies that individuals' cognitive processes affect and become affected by the ones of the other team members. Communication among the team members serves as the means for the gradually growing interdependence of cognitions and behaviors such that efforts (or effort avoidance) become coordinated and shared. (p. 93) Interdependence is likely to be low during short research experiments when subjects have little previous interpersonal knowledge or interaction with one another (Hooper, Sales, & Rysavy, 1994; Salomon & Globerson, 1988). Low interdependence may explain the variability of results concerning achievement in the computer-based collaborative learning literature, as well as the unanticipated results in the pilot study (described later in this chapter). Therefore, paired subjects picked their partners. The assignment process proceeded as follows: 1. The students were previously assigned to one of twelve unit teaching labs (UTLs) ranging from 8 to 12 students each, therefore, the method of instruction variable (single or pairs) was randomly assigned to the UTLs. One UTL was assigned to the single condition for every two UTLs 57

PAGE 72

assigned to the paired condition, resulting in the assignment of 4 UTLs (44 students) to the single condition and 8 UTLs (88 students) to the paired condition. 2. A program version was then randomly assigned to each individual student or pair. This process created four treatment groups (see Figure 3.6). Groups 1 and 2 were each comprised of22 students working individually. These groups differed by the version of program. Groups 3 and 4 were each comprised of 22 collaborative pairs. Again these groups differed by the program version. The breakdown of students and treatment groups for the final analysis consisted of 31 single students ( 18 students assigned to Group 1 and 13 students assigned to Group 2) and 35 collaborative pairs (20 pairs assigned to Group 3 and 15 pairs assigned Group 4). Design This study used a 2 X 2 factorial design (see Figure 3.6) to address the question ofhow embedded generative learning strategies and collaboration affect knowledge acquisition among advanced learners (medical students) in a CFT-based computer microworld. 58

PAGE 73

Figure 3.6. Experimental Design Program Version Instructional Method Base Generative Single Group 1 Group 2 Pair Group 3 Group 4 The experimental design employed the pretest-posttest control group design espoused by Campbell & Stanley (1963) and is graphically depicted below: Independent Variables R o x1 o R o x2 o R o x3 o R o x4 o. Variables Two independent variables were studied (see Figure 3.6). The first independent variable was the version of computer program. As previously discussed, a base computer program was developed according to cognitive flexibility theory principles. A generative version was then developed by embedding two generative learning strategies into the base version. Creating two similar, yet qualitatively different 59

PAGE 74

programs pennitted testing the effect generative learning strategies exert on learning in a CFT-based computer microworld. The second independent variable consisted of the instructional method (individual versus collaborative learning), which pennitted testing the effect of collaboration on the learning among advanced learners in a cognitive flexibility environment. Dependent Variable The dependent variable was the knowledge acquired (constructed) by the subjects as measured by a posttest taken at the conclusion of the learning phase of the computer program. The dependent measure consisted of three cases that required the subjects to diagnose and treat transfusion related problems similar to the problems encountered in the practice cases. Each posttest case measured the students' mastery of the facts and ability to think through and solve the transfusion medicine problems encountered. The test cases were similar in structure to the practice cases except that the infonnation resources (e.g., perspectives, similar cases, and textbook) were removed (see Figure 3.7). 60

PAGE 75

Figure 3.7. Sample Screen from Test Case B I TEST CASE 8 : Introduction Too head nurse rrom too premattre mrsery calls you and says soo is sending a blood sarrple rrom a baby who has had a severe hemolytic r e action while undergoing an exchange transfusion. You are too physician covering the transfusion medicine consu tation service and the neonatology service needs yotr help. Research: ( ) : Actions: ( OrderlabTe$1s ),. '-"es.sCase ) As discussed in Chapter One, collaborative learning studies typically require subjects assigned to the group condition to test independently to measure the "individual" knowledge acquired as a result of collaborating. Collaborative learning is premised on the notion that "two heads are better than one," where the level of group generated knowledge is greater than the knowledge of the individual participants (Whipple, 1987). Also, the research demonstrates that group-toindividual transfer occurs at least to the same degree between subjects assigned to collaborate and subjects assigned to the single condition. Therefore, in this study, the subjects assigned to the collaborative condition tested with their partners. This 61

PAGE 76

permitted examining individual versus socially constructed knowledge. The instruments section provides a more detailed description of the posttest. Instruments Pretest and Posttest The pretest and posttest measured the subjects' understanding of the transfusion medicine knowledge domain. The students' scores on the pretest and posttest were based on the lab tests they ordered, and how they assessed and managed three cases conceptually similar to the cases presented during the learning phase of the program. The point system used to score the learners' performance is outlined below: Lab tests: Each lab test ordered was scored on a scale from -1 to + 2 depending on how essential it was for establishing a diagnosis for the case, its affect on the patient, and cost considerations. Assessment: Each action selected for assessing a case received a score of -1 or+ 1 depending on the validity and appropriateness of the selection. Management: Each management action received a score of -1 or + 1 depending on the appropriateness of the selection for the case. The three test cases contained a total of 172 lab tests, 19 assessment, and 16 management choices (see Appendix C for an example test case). There were 30 lab test points, 9 assessment points, and 8 management points for a total of 4 7 points. 62

PAGE 77

The same cases were used for the pretest and posttest (see the pilot study for justification). The test cases were developed in conjunction with the learning cases. Two transfusion medicine experts reviewed the test cases for content validity, comprehensiveness, and accuracy as part of the content review process. They were asked to judge the match between the learning objectives, the practice cases, and the test cases, as well as to evaluate the rationale used to develop the grading scales. The experts were asked to arrive at consensus regarding these issues. Changes were made to the test cases based upon the experts' feedback. Procedure Pilot Study The base computer program was piloted during the spring of 1997 to determine: 1. If differences in learning exist between single and paired users, 2. If medical students using a computer-based learning program would effectively collaborate, and 3. If the test cases were parallel forms. The participants in the pilot study were second-year medical students, one year further advanced in medical school than the current subjects. All students participated, resulting in a subject population of one hundred and forty-two. 63

PAGE 78

To answer these questions the base program was modified slightly. Since the program consisted often cases, the decision was made to drop one non-infectious disease case to produce a module containing two pretest cases, five learning cases, and two posttest cases. The pretest and posttest each contained one infectious disease case and one blood donor case, while the learning phase contained infectious, non infectious, and donor cases. Two versions ofthe base program were developed, differing by alternate forms of the pretest and posttest. In other words, the pretest for program version A was the posttest for program version B and vice versa. The pilot study was conducted between April 2, 1997 and May 22, 1997. Pretest/posttest results indicated that the test versions were not parallel. The means for one infectious disease test case (test case 2) were higher than the means for the second infectious disease test case (test case 1). A series of2 (Grouping) X 2 (Version) ANOVAs were run to determine if a lack ofparallelism existed. The ANOV As resulted in several significant main effects and interactions by program version for the recipient cases. The data suggest that test case 2 was easier, which corresponded with the higher mean scores on this test. Although the tests were not equivalent versions, it was still possible to determine if differences in learning existed between singles and pairs. In this case two separate ANCOV As were run (by program version) comparing the means for singles and pairs for program version A and comparing the means for program version B. This analysis indicated that no significant differences existed between the single and 64

PAGE 79

paired students using the same versions of the program. Although not anticipated, this result does not deviate from the collaborative, computer-based learning literature, where some studies demonstrate higher achievement for collaborative groups versus individuals, while others find no significant differences. Finally, to determine if program users would effectively collaborate, five volunteer pairs were videotaped using the program. A coding scheme containing two main interaction types (on-task and off-task) and four verbal interaction categories (questions and statements for both interaction types) was developed as an observation checklist. The four interaction categories were further subdivided into two categories: (a) suggestions, opinions, directions, and (b) explanations, evaluations. The coding scheme also contained a provision for categorizing an interaction as either cognitive conflict or co-construction. Verbal interactions were coded in terms of their frequency of occurrence by two independent coders. To determine if cognitive conflict or co-construction existed, the coders were asked to document cases of cognitive conflict; if cognitive conflict was not present, it was assumed that co-construction existed. The coders documented the verbal interactions for the practice and posttest phases of the program. Average inter coder agreement was .95. Although the sample was small, the data suggest that the medical students remained highly task focused during the two program phases observed. Furthermore, the interaction patterns demonstrate that the students concentrated their efforts in giving and receiving explanations, which as previously 65

PAGE 80

discussed, are positive collaborative behaviors. Finally, the coders failed to document any cases of cognitive conflict, suggesting that these pairs created a co-constructive, collaborative environment. The results of the pilot study led to the questions of interest in the present study, as well as the following procedural changes: 1. Creation of a second program (i.e., the generative version). 2. Since the test cases were demonstrated not to be parallel forms of the measure, the decision was made to drop one test case and use the same cases for both the pretest and posttest. 3. Adding the dropped test case to the bank ofleaming cases to provide comprehensive coverage of the transfusion medicine domain. 4. Assigning the instructional method condition to the UTLs rather than to each individual student. Instructional Intervention The Pathophysiology ofDisease course was broken into three sub-courses. The transfusion medicine curriculum was taught during the hematology block of instruction. During a 30-minute lecture on the first day of the hematology course, the students were introduced to the computer module. The transfusion medicine curriculum manager discussed the program content and learning objectives, the logistics involved to obtain a computer disk, and the requirement to complete the 66

PAGE 81

program and grading criteria. The students were also told that they must complete the program as assigned (i.e., if assigned to work individually, they must not consult with any other students; and if assigned to work collaboratively, they must complete the program only with their partner). Finally, the students were directed not to discuss program content or operation with any other students enrolled in Pathophysiology of Disease. Appendix D contains a copy of these instructions. The program was loaded on six computers in the medical school's Learning Resources Center (LRC). Three computers were designated as the primary computers for running the program; these computers were physically separated from one another. The other three computers containing the program were co-located and were considered backups. Each individual student and collaborative pair were issued a diskette that provided access to the program and recorded their actions. Due to the medical students' tight schedules and the limited number of computers, the students were given approximately three weeks (March 23, 1998 to April 10, 1998) to complete the program. The medical school relies heavily on the honor system, however to monitor compliance with the curriculum manager's instructions to work individually or in pairs as assigned, the author was present while 22 single students and 31 pairs worked through the program. After completing the program the students turned their disks into the LRC staff for collection. 67

PAGE 82

CHAPTER4 RESULTS Introduction This study examined the relative effects of embedded generative strategies and collaboration on knowledge acquired by advanced learners in a cognitive flexibility theory-based computer microworld. The independent variables were program version (base or generative) and method of instruction (single or pairs). The dependent variable was the score achieved on a posttest consisting of three transfusion medicine cases. Analysis of covariance (ANCOV A) was used to test the experimental hypotheses. In addition, multivariate analysis ofvariance (MANOVA), analysis of variance (ANOV A), and t-test statistical techniques were used in a secondary analysis of the data. This chapter presents the results of these analyses and is organized as follows: 1. Data collection and screening procedures. 2. Primary data analyses. (1) Compliance with the assumptions for ANCOVA, (2) Summary and descriptive statistics, and (3) Analysis of covariance. 68

PAGE 83

3. Secondary data analyses. ( 1) Multivariate analysis of variance, (2) Program effectiveness, (3) Time spent on instruction, (4) Analysis of explanations and summaries, and (5) Analysis of student comments. 4. Summary of results. Data Collection and Analysis Screening Procedures The students' disks recorded pretest and posttest results (selections and scores), as well as other interactional data (e.g., time information, student explanations and summaries, and program critiques). These data were extracted from the disks, checked for accuracy, and hand entered into the Statistical Package for the Social Sciences (SPSS 7.5 for Windows, 1996) data editor. This database served as the master file for the primary and secondary statistical analyses. As previously discussed, medical students enrolled in the hematology rotation of the Pathophysiology of Disease course were required to complete this program. One hundred and thirty-two students completed the program; however, thirty-one students were eliminated from the final analysis due to inaccurate data or noncompliance. Three students were dropped because of extreme scores or missing data. Two students assigned to the single condition were excluded because they violated the 69

PAGE 84

requirement to work independently. The remaining twenty-six students were eliminated due to lack of effort. These students either failed to provide the explanations and summaries required of the generative version of the program, or they chose to spend less than thirty-six minutes (the established minimal time necessary to complete the program) on the six practice cases. Thus, the final analysis included one hundred and one students (31 singles and 35 pairs). Primary Data Analyses This study used a 2 X 2 factorial design to test for the main effects and interaction predicted in Chapter Two. The students were randomly assigned to the different levels of the independent variables. An analysis ofvariance on the subjects' pretest scores suggests that randomization was achieved (see discussion of random assignment in the next section); however, analysis of covariance (using the pretest as the covariate) was selected to conduct the primary data analysis to increase the power of the study. This section discusses the assumptions of analysis of covariance (AN COY A), provides summary and descriptive data, and concludes with a presentation of the ANCOV A results. Assumptions for Analysis of Covariance Random and Independent Errors. Each subject was randomly assigned to one treatment group according to the process described in Chapter Three. An analysis of 70

PAGE 85

variance conducted on the pretest for the four treatment groups suggests that random assignment was achieved: F(3, 63)=.413, p=.744. Additionally, the researcher observed 84 students running the computer program and noticed only two students not complying with the requirement to work as assigned (i.e., individually or pairs). These students were excluded from the statistical analyses. The students' disks also recorded the dates and times the students accessed the program. Analysis ofthe dates and times of the students not observed suggest compliance with the requirement to work as assigned. To gauge the level of compliance with the condition to refrain from discussing the program, the researcher randomly asked students what they had heard about the program prior to arriving at the LRC. Most of the students reported hearing that the program was "long" or "frustrating," but none of the students reported any discussion related to the program's content. An analysis of variance conducted on pretest data (F(2, 63)=.217, p=.805) and posttest data (F(2, 63)=.204, p=.816), by the week the program was completed, suggests this noncompliance had a negligible effect on the students' scores. Finally, inspection of a residual plot ofposttest scores (see Appendix F) suggests that the assumption of random and independent errors was met. Normality. Analysis of covariance is robust to departures of the sample from normality (Lomax, 1992); nevertheless, the subjects' posttest scores were analyzed to determine if the requirement of normality was met. Mean posttest scores were 22.9 with a standard deviation of 5.18. The skewness statistic ( -.270) indicates that the 71

PAGE 86

distribution was slightly skewed to the left. The kurtosis statistic ( -.469) indicates that the center and tails of the distribution were slightly shorter than that of a normal distribution. The ratio of each of these statistics to their standard error can be used as a test of normality (SPSS, 1996). The ratios of -.92 (skewness) and -.81 (kurtosis) fall between the values -2 and +2, and the shape of the histogram is fairly symmetric, therefore the posttest scores approximate a normal distribution curve. Stem and leaf plot analysis ofposttest scores and the Kolmogorov-Smirnov test of normality (F (66)=.102, p=.083) confirm that the posttest scores approximate a normal distribution (see Appendix F). Homogeneity of Variance. Levene's test of equal variances, and Cochran's C, and Bartlett-Box F tests of homogeneity of variance were conducted on the posttest data to determine ifthe cell variances were equal. Levene's F(3, 62)=.697, p=.558, Cochran's C(l6,4)=.282, p=l.OOO, and Bartlett-Box F(3, 6475)=.069, p=.977 all support the null hypothesis that the population cell variances were equal. Linearity and Homogeneity ofRegression Slopes. Inspection of the scattergrams showing the relationship between the pretest and posttest scores for the entire sample, as well as each treatment group, support the assumption of linearity. In each case a straight-line best fit the data (see Appendix F). The assumption of homogeneity of regression slopes requires the slopes of the regression lines to be the same for each group, which permits testing for group intercept differences (Lomax, 1992). This assumption was supported by positive 72

PAGE 87

Pearson correlation coefficients (see Table 4.1). However, as the table shows, the Pearson coefficient for treatment group 3 was different than the Pearson coefficients for the other three groups. To ensure compliance with this assumption, an analysis of covariance was conducted in which a three-way interaction was specified between the covariate (pretest total score) and the two independent variables (SPSS 7.5, 1996). This interaction term, PROGRAM*SINGLEIPAIRS*PRETEST TOTAL SCORE, showed no evidence of violation of the equal slopes assumption: F(1, 59)=.690, p=.410. Thus the null hypothesis of equal slopes was accepted. Table 4.1 Pearson Correlation Coefficients Treatment Group Pearson Correlation Coefficient Total r = .490 Treatment Group 1 (single/base) r = .593 Treatment Group 2 (single/generative) r = .568 Treatment Group 3 (pair/base) r = .309 Treatment Group 4 (pair/generative) r = .660 73

PAGE 88

Covariate Measured Without Error. As discussed in Chapter Three, the same three cases were used for the pretest and posttest. These cases required the subjects to evaluate, synthesize, and apply content material while making multiple decisions regarding the appropriateness of lab test, assessment, and management actions. This testing format precluded the ability to obtain a reliability estimate since the test cases were not objective, multiple-choice tests. However, these test cases were developed in conjunction with the practice cases and reviewed by transfusion medicine experts to establish content validity and accuracy. Therefore, not meeting this assumption did not adversely affect the results. Fixed Independent Variables. This assumption states that the researcher fixes the levels of the independent variables (Lomax, 1992). The study's design, which assigned subjects to one treatment group only, ensured compliance with this assumption. Independence of the Covariate and Independent Variable. Although not an assumption of the ANCOV A model, the covariate should not be influenced by the independent variable (Lomax, 1992). There were two independent variables in this study program version and method of instruction. This condition was met for version of program since the pretest was taken before the students entered the practice portion of the computer program. Although the students failed to comply with the request to refrain from discussing the program, there appeared to be no significant effect on the pretest scores. An analysis of variance investigating the differences in 74

PAGE 89

pretest scores over time suggests that there were no significant differences in pretest scores during the course of the study: F(2, 65)=.217, p=.805. When asked, none of the students admitted to knowing about the content of the pretest before running the program. In fact, several students commented that they wish they had known the pretest and the posttest were the same. Method of instruction, the second independent variable, was randomly assigned to the unit teaching labs instead of each individual student. The study's design established the "pair" as the unit of analysis (rather than two individuals); therefore, students assigned to the collaborative condition tested together rather than separately. An analysis of variance conducted on the pretest scores demonstrated no significant difference between the single and paired treatments: F{l, 64)=.479, p=.491. This result suggests that method of instruction was independent of, and did not influence, the pretest scores. In summary, the following assumptions were fully met for this study: random and independent errors, normality, homogeneity ofvariance, linearity and homogeneity of regression slopes, fixed independent variables, and independence of the covariate and the independent variable. The assumption pertaining to the measurement of the covariate without error could not be tested due to the nature of the measure. 75

PAGE 90

Summary and Descriptive Statistics Table 4.2 presents summary and descriptive statistics for this investigation. The pretest scores ranged from 13.7 for the single/base treatment to 15.9 for the pair/base treatment, with an overall mean of 15.1 (SD = 6.4). The pretest means ranged from 14.8 to 15.5 and from 14.5 to 15.6 for the two independent variables respectively. The overall mean for posttest performance was 22.9 (SD = 5.2). Posttest scores ranged from 21.1 for the single/generative treatment to 24.2 for the pair/base treatment. For the two independent variables, the posttest scores were 24.1 versus 21.4 favoring the base version of the program, and 23.1 compared with 22.7 favoring the collaborative condition. Table 4.2 also shows that the mean time for completing the practice portion of the program was 72.2 minutes (SD = 30.8). The mean completion time ranged from 63 minutes to 88.8 minutes. For the two independent variables the mean completion times were: (a) 63.2 minutes for the base program versus 84.4 minutes for the generative version of the program and (b) 70.5 minutes for the pairs compared with 74.1 minutes for the single students. 76

PAGE 91

Table 4.2 Summary Statistics Single or N Pretest Posttest Practice Program Pairs Case Time Base Single 18 M 13.7 23.9 63.4 SD 8.6 5.0 18.6 Pairs 20 M 15.9 24.2 63.0 SD 5.5 4.9 19.1 Total 38 M 14.8 24.1 63.2 SD 7.1 4.9 18.7 Generative Single 13 M 15.7 21.1 88.8 SD 6.5 5.2 41.7 Pairs 15 M 15.3 21.6 80.6 SD 4.9 5.5 38.0 Total 28 M 15.5 21.4 84.4 SD 5.5 5.3 39.2 Total Single 31 M 14.5 22.7 74.1 SD 7.7 5.2 32.5 Pairs 35 M 15.6 23.1 70.5 SD 5.2 5.2 29.6 Total 66 M 15.1 22.9 72.2 SD 6.4 5.2 30.8 Analysis of Covariance It was hypothesized that the embedded generative learning strategies and collaboration would lead to deeper processing of content material and result in higher levels of learning as measured by performance on the posttest. Furthermore, it was predicted that the two independent variables would interact to influence learning and posttest performance. Analysis of covariance (ANCOV A) was selected to test for the 77

PAGE 92

presence of these main effects and interaction. Table 4.3 presents the results of the ANCOV A. Table 4.4 presents the adjusted cell and marginal means, by treatment group. Table 4.3 Analysis of Covariance Summary Source Program Singles/Pairs Program X Singles/Pairs Pretest Total Score3 Error p < .05 3Covariate ss 144.48 0.10 6.79 418.83 1176.40 df 1 1 1 1 61 MS F p 144.48 7.49* .008* 0.10 .01 .943 6.79 .35 .555 418.83 21.72 .000 19.29 Note. The posttest total score is the dependent variable. Effect size and power at the .05 level are .61 and .78 respectively. Table 4.4 Adjusted Cell and Marginal Means Program Single or Pairs Adjusted Cell Adjusted Means Marginal Means Base Single 24.53 24.20 Pairs 23.90 Generative Single 20.84 21.20 Pairs 21.51 Single 22.97 Pairs 22.89 78

PAGE 93

Main Effect: Version of Program. The results of the ANCOV A suggest the presence of a significant main effect for the version of program: F(1, 61)=7.49, p=.008. Inspection ofthe adjusted means indicates that subjects assigned to the base version of the program scored significantly higher on the posttest than subjects assigned to the generative program. Main Effect: Method oflnstruction. The preceding data suggest that there was not a significant main effect for method of instruction: F(1, 61)=.01, p=.943. The adjusted means were virtually identical for the single and collaborative treatments. Interaction: Version of Program X Method of Instruction. The analysis of covariance results suggest that the two independent variables did not interact: F(l, 61)=.35, p=.555. This indicates that the version of program and method of instruction variables operated independently to affect learning. Summary of ANCOV A Results. The preceding data suggests that there was a main effect for the version of program variable, but not in the direction hypothesized. Furthermore, there was no main effect for the method of instruction variable, nor was there an interaction between the two independent variables as hypothesized. Before addressing the implications of these results, a series of secondary analyses were conducted to provide additional insight concerning the dynamics ofthe study. 79

PAGE 94

Secondary Data Analyses Multivariate Analysis ofVariance The ANCOV A demonstrated a significant main effect for the version of program. Since the posttest consisted of three different cases, a multivariate analysis of variance was conducted to determine where these differences occurred. Before computing the MANOV A it was necessary to again test for assumptions. The additional assumptions include composite observations that are normally distributed, equally variable in the population sampled, and independent. Analysis of the histograms for the three posttest case scores indicates that the scores for test cases B and C appear to violate the assumption that composite observations were normally distributed. Comparing the ratios of the skewness and kurtosis statistics with the respective standard errors for each test case confirmed the violation. The ratios for test case A ( -1.02 and -.24 respectively) were between the -2 to +2 parameter for normality; however, for test cases Band C these ratios were outside the acceptable range for normality (for case B, skewness ratio was -.2.36; for case C, skewness ratio was -4.1 and kurtosis ratio was 3. 75). These ratios indicate that the distributions for test cases B and C were more highly skewed to the left than normal and that the tails for case C were longer than normal. Levene's test of equal variance, Cochran's C, and Bartlett-Box F analyses of each posttest case suggests that the assumption of equal variance was met (see Table 4.5). Finally, Box's M=17.683, 80

PAGE 95

F(18, 10791)=.892, p=.589, indicates compliance with the test for homogeneity of dispersion (i.e., the observed covariance matrices of the dependent variables were equal across groups). In sum, the assumptions of equal variability and independence were met, whereas the assumption of normality was not. However, multivariate analysis of variance is relatively robust to moderate departures from normality (Lomax, 1992), therefore, this violation had minimal effect. Table 4.5 Tests for Equal Cell Variance A B c Levene's Test F=1.454, p=.24 F=.147, p=.93 F=1.695, p=.18 Cochran's c C=.33177, p=.57 C=.33644, p=.53 C= .40654, p=.11 Bartlett-BoxF F= .84663, p=.47 F=.41605, p=.74 F=l.8231, p=.14 The results of the MANOVA demonstrate a significant main effect for the version of program variable on the posttest: Wilk's F(3, 57)=4.46, p=.007. Follow up univariate tests found significant differences for test case A (F( 1, 59)=4.17, p=.046) and test case C (F(1, 59)=4.64, p=.035) (see Table 4.6). Mean scores for subjects using the base program were 4.5 and 3.6 (for test cases A and C respectively), whereas mean scores for subjects using the generative program were 3.0 and 2.9 for the same test cases (see Table 4.7). These results suggest that subjects using the base 81

PAGE 96

program significantly outscored subjects using the generative program on these two test cases. Table 4.6 Univariate Tests of Significance Dependent Sum of Squares Mean Square Variable Df F Sig. Posttest Case A 34.88 1, 59 34.88 4.17 .046* Posttest CaseB 17.05 1, 59 17.05 1.39 .243 Posttest CaseC 7.97 1,59 7.97 4.64 .035* p < .05 Table 4.7 Posttest Case Scores for Version of Program Dependent Program Mean Variable Posttest Case A Base 4.50 Generative 3.01 Posttest Case B Base 16.25 Generative 15.21 Posttest Case C Base 3.57 Generative 2.86 82

PAGE 97

Program Effectiveness The results ofthe ANCOVA suggest that subjects using the base program significantly outperformed subjects using the generative version of the program. The results of the MANOV A indicate that the differences in performance occurred on test cases A and C respectively. This section seeks to determine if the programs were effective teaching principles of transfusion medicine. To address the question of effectiveness, a series of one-tailed t-tests were computed to (a) compare total pretest posttest scores, (b) determine which components of the test cases appeared to stimulate learning, and (c) specifically address the effectiveness of the generative program, since students using this version scored significantly lower on the posttest. Total Pretest-Posttest Gain. The first analysis consisted of comparing the pretest and posttest mean scores for the total group of subjects. Alpha was set at .05. This analysis indicates that the total posttest scores were significantly higher than pretest scores (t=l 0.582, p<.OOOI ). Test Case Component Analysis. The pretest and posttest were broken into their component parts to determine in which portions of the program posttest scores were significantly higher than pretest scores. This analysis consisted of running fifteen separate t-tests, therefore, to control alpha from growing unacceptably large, alpha was set at .005 for each t-test. Table 4.8 presents the results of these t-tests. These data suggest that posttest scores were significantly higher than pretest scores for the three decision making areas (tLabs=7.817, p<.005; tAssess=4.524, p<.005; 83

PAGE 98

and tManage=5.795, p<.005), as well as for all three test cases (tA=5.34, p<.005; ts=6.97, p<.005; tc=4.87, p<.005). However, learning was not uniform across the three test cases. Lab scores were significantly higher on the posttest than the pretest for all three cases (tAL=4.814, p<.005; tsL=4.408, p<.005; and tcL=3.725, p<.005), whereas assessment scores were significant in the test case B only (t8A=4.065, p<.005), and management scores were significant in test cases B and C (tsM=2.862, p<.005; tcM=5.744, p<.005). 84

PAGE 99

Table 4.8 Aggregate Pretest/Posttest T -test Results Component Mean SD Df T Total Score-Labs 5.67 5.89 65 7.817* Total Score-Assessment 1.09 1.96 65 4.524* Total Score-Management 1.10 1.55 65 5.795* Test Case A Total 2.74 4.17 65 5.337* Labs 2.35 3.96 65 4.814* Assessment .27 1.34 65 1.651 Management .13 .72 65 1.537 Test Case B Total 3.29 3.83 65 6.974* Labs 2.09 3.85 65 4.408* Assessment .82 1.64 65 4.065* Management .37 1.03 65 2.862* Test Case C Total 1.83 3.06 65 4.865* Labs 1.23 2.67 65 3.725* Assessment .00 .68 65 .363 Management .61 .86 65 5.744* p < .005 85

PAGE 100

Pretest-Posttest Gain for the Generative Program. The final analysis compared pretest and posttest mean scores for the students using the generative version of the program. A one-tailed t-test (alpha= .05) indicated that posttest scores were significantly higher than pretest scores (t=6.48, p<.OOOI). The pretest and posttest were again broken into their component parts for analysis. Alpha was set at .005. The component analysis t-tests indicate that lab scores were significantly higher on the posttest than the pretest (t=4.84, p<.005), but there were no significant differences between pretest and posttest scores for assessment and management. Also, the posttest scores were significantly higher than pretest scores for test cases B and C (ts=3.85, p<.005; Tc=2.87, p<.005). Finally, management scores were significant for test case Conly (tcM=3.057, p<.005). In summary, total posttest scores for the generative program were significantly higher than pretest scores; however the difference appeared significant in only one decision making area (labs), two test cases (Band C), and one part of case C (management). In contrast, when the analysis included both program versions, posttest scores were significantly higher than pretest scores for all three decision areas, all three test cases, labs in all three cases, assessment in one case, and management in two cases. Thus, there appeared to be a significant fall off in learning effectiveness for the generative program when compared to the aggregate analysis that included both program versions (Note: t-test analyses performed on the base version of the program mirror the aggregate results and are presented in Appendix F). 86

PAGE 101

Time Spent on Instruction Table 4.2 reports the amount of time subjects spent in the practice portion of the program. Analysis of variance (ANOVA) indicated a significant main effect for the version of program (F(l, 62)=8.383, p=.005), but no main effect for method of instruction (F(l, 62)=.265, p=.608). These results indicate that subjects using the generative version of the program spent significantly more time working on the six practice cases than those who used the base program. A second ANOVA was conducted to determine ifthe time spent on the practice cases varied significantly over the course of the study. Time spent in the practice portion of the program is reported in Table 4.9. These data reveal that there was an eleven minute decline after week one; however, an ANOV A indicated that there were no significant differences in time spent completing the practice cases over the duration ofthe study: F(2, 63)=.874, p=.422. Table 4.9 Mean Time Spent on Instruction Week Mean Time Spent N Program On Practice Completed Cases (minutes) Week One 79.8 20 Week Two 68.7 18 Week Three 69.0 28 Total 72.2 66 87

PAGE 102

Analysis of Explanations and Summaries Each explanation generated by the subjects was coded relative to lesson content as "irrelevant," "partially relevant," or "relevant." If the explanation was an incorrect synthesis and application of the content material (i.e., obviously not related or very superficially related to the case) or not attempted, it was classified as "irrelevant." When an explanation involved a substantially correct synthesis and application of the content material, it was coded as "relevant." If an explanation involved synthesis and application of information, but was not substantially correct, it was categorized as "partially relevant." The same scale was used to classify the summaries, but relative to the case objectives. Coding was accomplished by a physician with expertise in transfusion medicine. Table 4.10 reports the means and standard deviations for the subjects' explanations and summaries. The mean explanation score was 24.9 (out of a possible total of 44 points) and the mean summary score was 9.2 (out of a possible score of 12 points). The mean for total elaborations (i.e., the combination of explanations and summaries) was 34.1. Mean scores for each elaboration category were divided by the corresponding total possible points to provide an indication of relevancy (i.e., quality). This procedure produced relevancy scores of 56% for explanations, 77% for summaries, and 61% for total elaborations. 88

PAGE 103

Table 4.10 Summary Statistics for Explanations and Summaries Singles or Generative Generative Total Pairs Explanations Summaries Singles Mean 23.2 8.5 31.8 N=13 SD 7.4 2.4 9.1 Pairs Mean 26.3 9.8 36.1 N= 15 SD 7.0 1.6 8.2 Total Mean 24.9 9.2 34.1 N=28 SD 7.2 2.1 8.8 An ANOVA was conducted to determine if the mean differences between the explanations, summaries, and total elaborations generated by the single students and pairs were significant. Using an alpha of .05, there were no significant differences [FE(l, 26)=1.228, p=.278; Fs(1, 26)=2.781, p=.l07; and FE+s(1,26)=1.721, p=.201]. This result suggests that collaborating pairs were no more successful at generating explanations and summaries than the single program users. Analysis of Student Comments Subjects were given the opportunity to provide constructive feedback after finishing the program. These comments were independently coded by two raters. Where disagreement occurred, the raters discussed and revised their codes to achieve consensus. Fourteen categories were initially established based on the subjects' comments. These categories were then collapsed for categories of responses that were similar. 89

PAGE 104

For example, some subjects directly critiqued the time required to run the program, while others criticized the speed of the computers-an indication of dissatisfaction with time (Appendix G contains example student comments). Table 4.11 presents the students' feedback. The subjects' responses were negative relative to the time required to run the program, as well as the program as an instructional method. However, the subjects' responses were positive regarding the opportunity and benefits of collaborative learning. Table 4.11 Student Comments Category of Feedback Time Too much time Computers too slow Not enough time Instructional Method Do not like computer-based instruction Prefer lecture Did not learn from program Posttest cases differ from practice cases Program is a good educational tool Program Specific Feedback Lack of immediate feedback on test cases Inconvenient access to information sources Inability to change answers Collaboration Preferred Not preferred 90 Number of Responses 15 8 2 8 11 10 15 8 14 4 9 13 1

PAGE 105

Summary of Results Results ofthe preceding statistical analyses are summarized as follows: 1. The ANCOV A found a significant main effect for the base version, compared to the generative version, of "Handling Transfusion Hazards." There was no significant difference in mean posttest scores by method of instruction, nor was there an interaction between the two treatment variables. 2. A MANOV A yielded significant differences for test cases A and C for the program treatment variable. Subjects using the base program outperformed students using the generative program on both of these test cases. 3. A series oft-tests found (a) total posttest scores significantly higher than aggregate pretest scores and (b) significant posttest scores for all three decision making areas, all three test cases, and most of the component parts of the test cases. An identical analysis performed on the generative version found total posttest scores significantly higher than total pretest scores, but failed to duplicate the results of the component analysis. 4. An ANOVA on the time data indicated that subjects using the generative program spent significantly more time in the practice portion of the program than subjects using the base program. There was no significant difference in time for the method of instruction treatment. Also, although 91

PAGE 106

the mean time declined after week one, there was no significant difference in time spent on the practice cases over the duration of the study. 5. Subjects using the generative program provided "partially relevant" explanations and "borderline relevant" summaries. There was no significant difference between the quality of explanations and summaries provided by the single and paired students. 6. Analysis of student feedback demonstrated a generally negative disposition toward the program as an instructional method, as well as the time required to complete the program. The students were positive about the opportunity and effects of collaboration. 92

PAGE 107

CHAPTERS DISCUSSION AND RECOMMENDATIONS Introduction Cognitive flexibility theory (CFT) has been advanced as a means to overcome the problem of inert knowledge formation by advanced learners in complex and ill structured knowledge domains (Spiro et al., 1988; 1989; Spiro & Jehng, 1990). The research to date supports the assertion that CFT -based environments support and promote greater knowledge acquisition and transfer than linear treatments of the same subject content (Hartman & Spiro, 1989; Jacobson, 1990; Jacobson & Spiro, 1995; Spiro et al., 1987). This study accepted this premise and addressed the following questions: 1. Would the inclusion of embedded explanation and summary cues (generative learning strategies) result in differences in learning in a CFT -based computer microworld? 2. How would the addition of collaborative learning affect learning in this environment? Based on the literature reviewed in Chapter Two, three hypotheses were generated. First, it was hypothesized that students assigned to the generative program would 93

PAGE 108

perfonn better on the posttest than students assigned to the base program. Second, it was predicted that students assigned to the collaborative treatment would out perfonn students assigned to the individual condition on the posttest. Third, it was hypothesized that the treatment variables would interact to increase learning. Discussion This section relates the study's findings to relevant research and theory. Each hypothesis is addressed relative to the literature reviewed in Chapter Two. Additionally, other variables that might account for the study's results are addressed. Effect ofEmbedded Generative Learning Strategies Counter to the first hypothesis, findings indicate that students learned more from the base version of the program. The analysis of covariance indicated that students using the base program significantly outperfonned students using the generative version on the posttest. The multivariate analysis of variance found that the perfonnance differences occurred on test cases A and C. The t-test analyses confinn that the base program was superior to the generative program in promoting learning. This difference in posttest perfonnance occurred despite generative program users spending a significantly greater amount of time on the practice cases. These findings suggest that the embedded generative learning strategies depressed 94

PAGE 109

knowledge acquisition (relative to the base program) by advanced learners in a cognitive flexibility theory-based learning environment. This finding deviates from studies showing the positive effects of generative learning strategies on learning (Di Vesta & Peverly, 1984; Johnsey, Morrison, & Ross, 1992; Linden & Wittrock, 1981; Slamecka & Graf, 1978; Wittrock & Alesandrini, 1990). There are several plausible explanations. First, the discrepancy in results may be due to the context of application of the embedded generative strategies. Explanation and summarization strategies have been used predominantly in linear, text-based environments that focus on retention and recall of factual information. In contrast, this cognitive flexibility theory-based microworld, while highly text-based, required the students to synthesize, evaluate, and apply information to solve transfusion medicine cases. Thus, this study focused on the effects of generative learning strategies in a complex environment requiring critical thinking and problem solving. By definition, problem solving is a generative activity (Wittrock, 1990), in which case, the thinking processes the students used to assess and manage the cases in the base program were likely "mindful" enough to support knowledge construction and acquisition. Therefore, rather than complementing the thinking processes, the explanation and summary cues may actually have been redundant. Second, the subjects in this study were high achieving, successful students. It is likely, therefore, that these students could construct the relationships between prior 95

PAGE 110

knowledge and new information required for comprehension and transfer to occur without intervention. Therefore, asking these students to provide explanations and summaries may have been viewed by the students as trivializing the learning experience (Wittrock, 1990). There is evidence to support this argument. During the course of the study, one student commented that she had seen "the smartest student in the class" running the program and he was frustrated by the necessity to type in explanations and summaries. Another student typed in "What's the point of typing in my responses" during the summary of one of the cases. These examples suggest that some students viewed the necessity to type in responses as a nuisance, which may have influenced their interactions with the program and their posttest performance. Third, the deficiency in learning for subjects using the generative program may relate to the quality of explanations and summaries generated. Elaborations must be precise and relevant to facilitate memory and learning (Di Vesta, 1989), yet the quality of the explanations and summaries produced was suspect. The coding scheme used to analyze the explanations and summaries suggests a tendency by the students to provide "partially relevant" explanations and summaries only slightly better. The students had not received training on how to generate effective explanations and summaries; therefore, this finding may indicate that the students experienced difficulty while attempting to generate precise and relevant elaborations and highlight the need for instruction. Or, as discussed above, perhaps the students questioned the relevance and necessity to type in elaborated responses. In any event, these results 96

PAGE 111

support the view that elaboration must meaningfully relate new and old information to promote generative learning (Di Vesta, 1989; Wittrock, 1974a; 1974b; 1985). Fourth, the difference in posttest performance may be related to the significantly greater amount of time students needed to complete the generative program. Table 4.11 shows that many of the students included in the final analysis objected to the amount of time required to complete the programs. The data suggest that some students may have believed the demands and value of the task were outside the reasonable range of required effort to learn the material (Salomon & Globerson, 1988). Consequently, the students may have put less effort into interacting with the programs. Since the generative program, on average, took twenty-one minutes longer to complete, it is possible that these students put even less effort into learning the program content. The reduced effort may also explain the average quality of explanations and summaries produced by the students. A fifth explanation may relate to the structure of the generative program. After typing in explanations for lab, assessment, and management choices, students could access expert feedback only by reselecting their choices. Neither version of the program was equipped to process an audit trail of student activity, but the researcher observed that some students using the generative program continuously bypassed the feedback option. Bypassing expert feedback may have reinforced the students' mistakes and negatively influenced their posttest results. This explanation, however, deviates from studies that demonstrate a significant advantage favoring subject-97

PAGE 112

generated elaborations versus instruction-provided elaborations (Slamecka & Graf, 1978) or no relative advantage either way (Johnsey et al., 1992). Effect of Collaboration The second hypothesis predicted that students given the opportunity to collaborate would outperform students working individually on the posttest. Results for posttest performance indicated no significant difference between the single and paired treatment conditions. This lack of significant advantage occurred despite researcher observation of some pairs that revealed a pattern of collaborative activity best described as task focused and co-constructive. These students concentrated their efforts on giving and receiving explanations, which are collaborative behaviors positively correlated with increased achievement (King, 1989; Webb, 1987). Also, the collaborating students chose their partners, which is thought to increase group interdependence on short-term interventions and positively affect learning (Salomon & Globerson, 1988). Even the students themselves expected collaboration to increase learning and posttest performance (see Chapter Four). For example, one student commented, "I know I would have learned much, much more had I been working with a partner." However, despite the positive collaborative interactions, the ability to choose partners, and the students' expectations, the lack of significant effect for method of instruction is not surprising given the variable results of studies investigating the effects of computer-based collaborative learning (see Chapter Two). 98

PAGE 113

There are several possible explanations for the lack of significant posttest results favoring collaboration. These explanations may be related to the subjects' academic ability, the context of use, and posttest administration. As medical students, these subjects were successful, high-achieving learners with similar academic ability. This lack of academic variability may explain why collaboration did not produce the differences in learning anticipated. Four studies comparing achievement between students learning individually and students learning collaboratively at the same academic ability level have produced mixed results. In studies of elementary grade students, Yager, Johnson, and Johnson ( 1985) found a significant difference favoring collaborating high-ability students relative to high ability students working individually; whereas, Mevarech (1993) found no significant difference in achievement between high ability students working independently or collaboratively. In a study of college students, Gokhale (1997) found that collaboration significantly enhanced critical-thinking and problem-solving skills; meanwhile, in a study of adult learners (considered high ability), Makuch, Robillard, and Yoder ( 1992) found no significant difference in achievement between collaborating pairs and individuals. Although the finding of no significant difference was unanticipated, this result is not out of line with previous research and appears to support the findings of other studies (Hooper & Hannafin, 1988; Makuch, et al., 1992; Mevarech, 1993) that suggest that high-ability students benefit the least academically from collaborative learning. 99

PAGE 114

The manner of presentation may have also affected the results. Although required to complete each portion of the program sequentially (i.e., introduction, pretest, practice cases, posttest), the students were given considerable latitude regarding which resources they could use while completing the practice cases. The collaborating students spent less time on the practice cases, which may indicate a higher degree of work efficiency, but may also indicate that the collaborating students used less of the available information resources while working on the practice cases. Since the subject domain was new to the students, this may account for the lack of significant posttest performance favoring the collaborative condition. The immediacy of the posttest may have curtailed the positive performance effects often attributed to collaborative learning. Since collaborative learning is thought to promote deeper processing of material especially for problem-solving activities -by collaborating students, it is likely that collaborating students would retain more of the information they learned (Johnson & Johnson, 1996; Makuch et al., 1992; Shlechter, 1990; Yager et al., 1985). If this is true, then an immediate posttest might have masked treatment differences reflecting deeper processing of material by the collaborative condition. Therefore, a delayed test designed to test application of retained knowledge, rather than students' short-term retention and understanding of content material, might yield different results favoring the collaborative condition. In any event, the finding of no significant difference between individuals and pairs on 100

PAGE 115

posttest performance supports Webb's (1987) position that group work is not detrimental and may be beneficial to students' learning. Interaction of Generative Learning Strategies and Collaboration The third hypothesis predicted that the two treatment variables would interact to affect performance on the posttest. The expectation was that the interaction would lead to greater performance by students assigned to the pair/generative treatment. In contrast to previous studies (Sherman & Klein, 1995; Yager et al., 1985) that demonstrate the beneficial effects of cueing students to elaborate within a computer based, collaborative learning environment, the results of this study found no such benefit (i.e., there was no interaction). Students assigned to the pair/generative treatment were no more successful at generating explanations and summaries than subjects assigned to the single/generative treatment. Furthermore, students assigned to the pair/generative treatment were outscored on the posttest by students assigned to the single/base and pair/base conditions. The interaction effect was based on the expectation that the requirement to elaborate would increase the interaction between students, which in tum, would promote the generation of higher quality explanations and summaries favoring the collaborating students relative to the students working alone. Direct observation of eight pairs assigned to the generative program did reveal extra collaborative interactions when compared to collaborating students assigned to the base program. 101

PAGE 116

These interactions generally took two forms: (a) students worked jointly on wording the explanations and summaries or (b) one student typed while the other student corrected mistakes or suggested additional information. Given the average quality of the explanations and summaries generated by the students, the quality of these interactions were most likely low level information exchanges designed to satisfy the requirements of the generative program. Therefore, the lack of an interaction effect may be attributable to the inability of the embedded cues to stimulate effective interaction among these students. Other Consideration: Influence of the Overall Instructional Environment The discussion to this point has focused on explaining the findings relative to each hypothesis. Several explanations were discussed as possible reasons for the results. It is also likely that some combination of these reasons may account for the study's findings. However, the data suggests that a broader perspective may be necessary to fully address the research question. For example, the t-test analysis suggests that "Handling Transfusion Hazards" was an effective learning environment, but was it? The mean score for the base program was only 51% of the total possible score (24 points out of a total of 47 points), and the highest score obtained was 32 points (68%). Since research has demonstrated the effectiveness of cognitive flexibility theory-based computer environments (see Chapter Two), and given the caliber of students used in the study, one might expect better performance than an 102

PAGE 117

absolute high score of 68% or mean of 51%. Additional explanations may be related to the overall instructional environment of the medical school curriculum. The first two years of education at this institution focus on disseminating information to the students and then testing recall through multiple-choice and true false examinations (D. Ambruso, personal communication, May 13, 1998). According to Ambruso, the curriculum is taught predominantly in lectures, with little opportunity for implementation of active learning strategies. Also the students are highly extrinsically (i.e., grade) motivated. Several students corroborated Ambruso's statements. One student commented: "The medical school curriculum is mostly lecture. Medical students spend a great deal of their time memorizing information." Another student stated that many of her classmates would put minimal effort into interacting with the program because it was only a pass/fail exercise, and therefore, had virtually no impact on their grade or class standing. For these students the traditional model of education prevails, punctuated by efficient transmission of information to passive learners and multiple-choice testing to evaluate what was learned (i.e., were simple associations formed among the material?). "Handling Transfusion Hazards" imposed three demands on the learners cognitive complexity, task management, and buying in (Perkins, 1992). These demands required the students to think harder, think for themselves, and accept a new instructional approach; yet many students' actions and comments suggest that they were accustomed to the traditional model of learning and were surprised at the kind of 103

PAGE 118

learning required by the program. For example, twenty-six students were dropped from the final analysis due to lack of effort when running the program. Additionally, Table 4.11, which provides feedback from those students included in the final analysis, indicates that the students were critical of(a) the time they spent running the program, (b) the case-based presentation style (i.e., they preferred lecture), and (c) the perceived content differences among the cases. These data suggest that many of the students disliked "Handling Transfusion Hazards" because it was new, different, and harder than the lecture-based learning environment they were accustomed to. Most students have been conditioned by the current educational system to let the "teacher" organize their learning experiences for them. This relieves the student of the responsibility for thinking for themselves. Thus, the important learning tasks of planning, gathering relevant resources, more planning, more self-directed learning, more decision-making, and ultimately more consequences for their own actions are placed squarely on the students and they don't like it. Simply stated, it's harder work. (J. Savery, personal communication, May 8, 1998) The data also suggest that since the students were accustomed to the traditional approach to learning, they would rather rely on simple strategies to learn the material. For example, one student commented: "Not that I am old fashioned, but give me a pencil, paper, and a book and I'm much happier." Another student wrote: "Your practice cases were easier than the pre/post cases. Also, the practice cases do not directly apply to the test cases." These comments suggest that the students would rather memorize information and rely on prototypic cases to learn; however, these 104

PAGE 119

strategies result in deficient learning outcomes in complex and ill-structured knowledge domains (Spiro et al., 1988). Thus, the preference of some students for lectures and for memorizing information and relying on a representative case to instantiate all relevant cases, may account for the somewhat disappointing learning results and may have affected the main findings of this study. The generative program may have been most affected because it required more mental effort. A second explanation may involve the inability of the students to deeply process the subject content. As previously discussed, the students had access to case specific history and physical exam information, perspectives of case-relevant operatives, a bank of similar cases, and the transfusion medicine textbook. Although not directly measured, the researcher noted a relationship between the amount of time spent in the practice cases and the amount of information accessed. This increased time would then denote information searches necessary for the students to assess and manage the clinical problems presented in the cases. Since "Handling Transfusion Hazards" was information rich, then it would be reasonable to expect a positive correlation to exist between time and posttest performance. While not reported in Chapter Four, a correlation was computed and indicated a slightly negative, but not significant, relationship (r = -.185) between the time spent on practice cases and posttest scores. This suggests that some students may have suffered from cognitive overload (mental fatigue) due to the number of information options available. Since students using the 105

PAGE 120

generative program spent, on average, twenty-one minutes longer interacting with the generative program, perhaps they suffered most from cognitive overload. Summary Would the integration of compatible generative learning activities into a cognitive flexibility theory-based learning environment promote deeper processing of material, and consequently, increased knowledge acquisition by advanced learners? The findings of this study suggest that the answer is no. Rather than stimulate the mindful processes of successful learning, the results suggest that embedded explanation and summary cues (generative learning strategies) had the opposite effect relative to the control condition (i.e., base program). Students using the base program outperformed students using the generative program on the posttest. Additionally, despite the apparent effective behavioral involvement and student expectations, collaborative activity had no significant effect on posttest performance. This suggests that there is no relative advantage or disadvantage to computer-based collaborative learning for advanced learners. Finally, since the embedded generative strategies depressed knowledge acquisition (relative to the control condition) and there was no advantage to working collaboratively, the expected interaction failed to materialize. Recommendations for Future Research The research question that framed this investigation addressed whether embedded generative strategies and collaboration would benefit advanced learners in 106

PAGE 121

a CPT -based learning environment. The findings suggest that the answer is no, but many questions remain. For example, would generative learning strategies benefit advanced learners in a different learning environment? Was the negative effect for generative learning strategies due solely to the quality of the students used in the study (i.e., advanced learners), use in a problem-solving environment, or both? In addition, the findings support the relationship between the quality of the generated elaborations and learning (Di Vesta, 1989), but contradict the relationship between learner-generated elaborations and learning relative to instruction-provided elaborations (Johnsey et al., 1992; Slamecka & Graf, 1978). Given the importance of facilitating the mindful processes of learning, more research devoted to the contextual application of generative learning strategies is necessary. With respect to collaborative learning, was the lack of significant difference between individual students and pairs due to the quality of student, the learning environment, or both? The students were permitted to select their partners, what effect did this have on collaborative activity and achievement? Research suggests that simply instructing students to work together is ineffective (Canier & Sales, 1987), but some students exhibited the constructive intra-group interactions and behaviors associated with successful group work (Damon & Phelps, 1989; King, 1989; Webb, 1987). This study was not designed to directly evaluate student interactions, but since some students (and perhaps most or all) collaborated successfully, why the finding of no difference in performance? This suggests that 107

PAGE 122

future research should focus on the quality of the interactions (i.e., the conversational content), not the quantity of specific behaviors. Cognitive flexibility theory evolved from the observed learning deficiencies of medical school students (Spiro et al., 1988), yet most ofthe relevant research has used volunteer advanced high school and college students to validate the efficacy of the theory. The present study used second-year medical students who were required to complete a computer program developed according to CFT principles. This investigation did not directly assess the effectiveness of cognitive flexibility theory, but the data and results suggest more research is warranted. For example, the results suggest that "Handling Transfusion Medicine" promoted learning, but just how effective was it? Despite providing feedback and an organizational structure to help students manage the complexity of information, many students appeared to have difficulty processing the information contained in the program. Was this due to the problem-solving context of application? Is additional scaffolding or coaching help necessary for advanced learners to function highly in a CFT -based environment? Finally, a larger question must be answered regarding the integration of cognitive flexibility theory or any other active, student-centered learning environment. The data suggest that many of the students simply did not put much time or effort into interacting with the computer program used in this study. These students were accustomed to memorizing and reproducing information and therefore, unwilling or unable to deeply process the information. Thus, this study points to the 108

PAGE 123

need to address the question of what circumstances or conditions are necessary to successfully implement an active, student-centered learning environment. Study Limitations The research reported in this study was complicated by the problems associated with conducting research at a medical school; specifically, the necessity to provide a fair degree of flexibility to accommodate the students' schedules. The students were required to complete the program as part of the hematology rotation of the Pathophysiology of Disease course; however, no time was allocated for this purpose during the students' academic day. The students had to balance competing demands from other courses, and given the pass/fail grade designation for this exercise relative to the requirements of the other courses, some students chose to minimize how they interacted with the program. Additionally, only three computers were available for the students to use. Therefore, to provide the flexibility required, the students were given approximately three weeks (March 23, 1998 to April I 0, 1998) to complete the program. The students were asked to complete the program as assigned (i.e., individually or collaboratively) and to refrain from discussing program content or operation with one another until after the hematology course ended. There was no practical way to enforce this request; however, to monitor the level of compliance, the researcher observed a large number (84) of the subjects running the program and asked many 109

PAGE 124

students what they had heard about the program from other students. Only two subjects were observed violating the requirement to work as assigned, which provided reasonable assurance that the students were abiding by this condition. However, every subject the researcher questioned, reported hearing something about the program from other students. The typical comments were that the program was "long" or "frustrating." These types of comments may have affected the students' attitudes and motivation toward the program and altered their interaction. The generative version of the program may have been especially susceptible to these influences. The final limitation involved the pretest-intervention-posttest format of the computer program. Since the program was developed to provide a comprehensive learning environment providing both content instruction and evaluation, the pretest and posttest were separated only by the time required to finish the six practice cases (intervention). Also, since the program was developed to provide a complete learning experience, it was deemed inappropriate to appreciably change the program structure, except to develop a second generative version. Therefore, the same cases were used for both the pretest and the posttest (see Chapter Three). Although not ideal due to the potential testing effect, it would have been time and cost prohibitive to develop the requisite number of different cases for the pretest and posttest and still maintain the same level of experimental rigor. Also, given the logistics of the study, it would have been extremely difficult to use a delayed posttest. Thus, while not eliminated, 110

PAGE 125

the pretest-posttest control group design reduced the effect of testing by subjecting all students to the same testing format. Analysis of variance testing indicates that there was little variation in pretest and posttest scores over the course of the data collection period. Therefore, the posttest scores of all the subjects were most likely similarly influenced by the pretest. 111

PAGE 126

APPENDIX A SAMPLE PRACTICE CASE I PRACTICE CASES : lrlroductlon l s Satlrday morning July s and you rtm tile IPh...l PonpeetiYto L ( -C&>e> )

PAGE 127

I PRACTlC E CASE S : Physical Exam K 113

PAGE 128

To read tho perspectives or various people on tho case you are currently researching, selec t a person from the list on the right. This donor had a Ho Is a rtsttrne donor and came to the blood bal"k 11115 morning because his mother had an operation a -ago and he warted b:> h&p replace the blood she had us.d. Ho appear<>d neNous and saki ht has b&&n atrakl o1 nt&dl&s since ht was a chlkl We dkla normal aaw and t went well, bt.twhtn he got 14> II'M1 ht saki ht r.t weak cotlaps<>d and lost consciousness. Ho f890in&d consciousness lltf
PAGE 129

This sounds Ike t could be ether SQmtlhlng psychological 0< son. sort ol rospo""" to blood lOSS. Maybe you d bettor decide wtllch alter reading ttltse donor risks In yourteldtlool< then you should be able to dilgll064t and lroat hin. I PRACTIC E CASES: Perspective ol Palltrt r-d I thougtt you had to last to gtvo blood, so I havenleat.n s ince last night The ont)' tiling I had tills morn ing was a olbladc coi!M 115

PAGE 130

The acldllonallrloonation you and the phlebotomist have gathered is vee rtactions may occu. tsounds lie& this donor may have had psychologic, as well as neu-ophyslologic reasons for this syncopal episode as he expressed neNousness abol.t donaling and also may be relatiVely dehy
PAGE 131

SIMILAR CASE S MENU 1-----_..;;. tlttt cbai" *''* IJU" c.aa follwl"l otlributoo : troUJ ,.....,..., bj"' ... --.... '"'""'"'*" Tlltt tt.ice coa N!ds IIDU Yltt. oue:s tt.t.,.. tllllilorla ... .,.Miofl .. lltrib-IIIW .... 117

PAGE 132

T1le case ex T1le Transient Seizure N O .Is a 55-year-<>ld wnte remale allolo9Qus blOod donor wno susmlned a severe ruction dtrtng donation I PRACTICE CASES: Lab Tests Menu Ch embtry HernalologylCoa.g.atlon Oie.IJ'I03lic: Procedwu tRe.diology 118

PAGE 133

j PRACTICE C.I.SES: l.ssessmert Select a. dl\J"'o,a lcrtti.s cue 'W'Hth mht dosetymetthu !Epiopoy 119

PAGE 134

" Management Bythetine you Mived lht d onor IM.dei' .. ltybten ptlced on hrs bed: ..th lut elr.el:edabovt his hetd ,coldcOMPf'I'$S .. hld blenappledlo tit f orehea d and errrnoriurntlngSlllsvwtadrriri3hred You htw obt.-wd the petierl: lor fHtll hel.-. ho11endtht blood preuweretnlilllt 601plipalllt ..thap We o1 52. Nwtctthee.ppropriltttltptlwtrttmo Fred de I I I I I emergencyroomrortllthwtr..tmlnl I I !Docwneriel evenbandtrelltrntnl on the donore.d I I llfolwup onth tcona.ion olthe andreltut Cong-a!ulallonsl You have COI!l>leted Tho Case o1 Fainting Fre blood OOnallon. Rt<:all tho Inc !donee o1 a vasovagal reaction I n r tsponM 11> blood OOnallon. pr9Ventlve ancllrntmentl'cr a vasovagal noacllon I n nosponse II> blood donation. 120

PAGE 135

APPENDIXB SAMPLE GENERA TNE SCREENS The Introduction, History/Physical, and Perspectives segments are identical in the basic and generative versions. The following screens are different for the generative version. l PRACTICE CASES: Hematology/Coagulation Tests Oct onllthtHernelologo,cteoe.g\Mtionltst{a)you..,.hto orO. In tltt IJK:t ,. ..... ,, hltY, ex,Wn.,.wca.ekn. Thn "'"' th rthm t e y VlwthttutrniAt bycktilg ontht iHt{') Th.,'t4Nthttuggnhdt .. ta bltort ret.,....gtolhtltbtNbm.-tll 121 Research: ( ttoior,IPhy>lcol J ."::=( :::::::.::::=::::{

PAGE 136

In tilt a,ec:t ,-ntlllelll a.ttev, expllln .,.we Ill Tllltft ,..,,, t lllt rlt.m tey. VINthecorrrnwts bycldci"'g onlheahp(s) aOWI. I Vt.SO't'lOJ' 3)0COpt Manag<>ment By the ti'neyouwrivtd t he donor had ehacty been p&ec:ed on twa blct ...th lut Mewleda.bove hla heed, coldcornpresset hld bunlf)pltdloH:s forehd .,d lfTIITionlumelng Mia Vou htvt oblervtd the petientforovert MIW'I hOII'W'Idthe blood pressll'tretr'lliu el: 601ptlpable d 52. Novuiectlhta.pproprillealeptfortruti'\g Frtdtle In I lilt I,.CI ,..Wilt II hiiY, IX,.., ytWCIIItkH. Tllltn ,.. ... I lilt ret.rn tey. VINU'II convnenta byc:lcti'IQ on the step( a) ageh 122

PAGE 137

I PRACTICE CASES: Sli1YTlal'/ In the space below. br1efy sllm'\artze the tnponant concepts covered I n this case. Where appropriate highlight the conceptual stnllaril les wHh other cases haY& completed any ) Then press the retum 123

PAGE 138

APPENDIXC SAMPLE TEST CASE Tho head trom promahre nln}' calls you and says she Is s.ndlng a blood saJT1lle trom a baby \\1'lo has had a sevtre r.adlon while undtrQOing an lranslilslon You are the physician cov.t1ng the lranstuslon medicine consulatlon s.rvlc and the neonatology s.rvlce neects your hep To read the case history and phySical exam resuns tor the case you are currently researching, click on the desired option on the r1ght. 124

PAGE 139

m!9,QA.ncv was montortd cartfu!ly and In the steortd lrtntsttr she had a do<; muted arti=P tler oC 11&1 Tile !rtart was dillvtrid bY Ctsartan stctlon at 33 wttlIn rtd cells S.!!$liZiodYTht holll !ocreastd to danotf9us ltvt!S t1 9 !IQ'dO and reauttd 1 concernOYtL :O.urologlc cojrp!qtlonutthls !ml9( bllbbln tilt houst ol!jcertd and sho btcamo !attwcar
PAGE 140

Cti onllu type ol teX )1'011\Mftl o orderlcr tN:t petienl:. Ch,.;,Qy Elood Brtl Serology 126

PAGE 141

'"' NOIIIIOI .. .. 127

PAGE 142

Blood Bark I Serologt Tests r.,t Noomtl le$lruu b . g>iU!JI .J!oo .. ..-loAl 128

PAGE 143

I TEST CASE B Diagnostic Prl N"""" J!l!!ll! "--... U...b!oosv -p......,...,, I TEST CASE B : Assssmert. part 1 WNthlld(s ) tromtNa eueil ( w t)ctUCielllotht ot a hemct,tic t."'n$$IQionrMCtion n t.-swtrt? I relltionship ollht tnnaha i onlothe dr'trNkclwlgnr. :ll.eholthei-Venl I !ld.....-IC'timolw
PAGE 144

130

PAGE 145

I TEST CASE B : SIIM1ary COngatulatlonsl You hav CO!Tl>ioted The case ol the OiiLbona I Oilerrma This test meastred yotr abllly to: Detine the risk ol hemol)tic lransluslon reaction ldentry the signs o1 a hemol)tic lranstuslon roactlon lnvestigat9 the caUS9S ol hemol)tic lranslusion ...actions InclUding non-lrrmune hemoly91s Apptf the princ"l9s for managing a hemol)tic lranstuslon roactlon 131

PAGE 146

MEMO To: From: Date: Subject: APPENDIXD STUDENT INSTRUCTIONAL HANDOUT Medicine 6000 (Pathophysiology) Students ----------'MD Monday, March 23, 1998 Transfusion Medicine Computer-Based Learning Program As students enrolled in Medicine 6000, you will complete "Handling Transfusion Hazards," a computer-based instruction program that provides you with a unique opportunity to learn about the risks associated with blood transfusions. The program consists of several authentic clinical cases that cover a broad array of transfusion medicine topics for donors (e.g., pain, infection, bleeding at the phlebotomy site, and vasovagal reactions) and recipients (e.g., transfusion transmitted viruses, immediate and delayed hemolytic reactions, and non-immune adverse events of transfusions). "Handling Transfusion Hazards" requires you to collect relevant information from several on-line resources to help you order and interpret lab tests, assess, and manage the transfusion problems presented. This program simulates the thinking and problem-solving processes used by practicing clinicians. Transfusion medicine is a burgeoning field in medicine; this may be your only formal exposure to this information in medical school. Procedures 1. The program is loaded on three Macintosh computers located in the Learning Resources Center (LRC) on the 3rd floor of the Dennison Library. These computers are prominently labeled with yellow placards. You will be given a diskette to access the program; the diskette will record your interactions with the program and must be turned into the LRC staff when you finish the program. This is a Pass/Fail exercise that you must complete by April13, 1998. If you do not complete the 132

PAGE 147

program by this date, a "failure" notation (for this exercise) will be entered into your record for the hematology rotation of Medicine 6000. Please note: the hematology final exam, scheduled for April 13, 1998, will contain transfusion medicine questions derived from the program. It is advisable, therefore, to finish the program by Friday, April10. 2. To help us evaluate the effectiveness of this program, you will complete the program either individually or in pairs. Students in the following UTLs will work individually: Room 1862B, Room 2809 (Inner), and Room 1860. All other students will work collaboratively with a partner of your choosing from your UTL. To maintain the integrity of the program, complete the program according to this condition (i.e., by yourself or collaboratively with your chosen partner) and refrain from discussing program content or operation with any other Med 6000 students. 3. Given the large number of students and the limited resources (number of computers and time) available, we have developed a schedule to alleviate potential frustration and to ensure the timely completion of the program. The schedule begins on Wednesday, March 25 and runs through April 9. The schedule allocates computer time in three hour blocks on a Monday, Wednesday, Friday, and Saturday format. Please note, the computers are available for use on Tuesday, Thursday, and Sunday on a first-come, first-served basis, but you run the risk of not gaining access to a computer right away on these days. The schedule is available for sign up immediately following the initial lecture (until1 p.m.) and then will be available in room 1862A until5 p.m. Wednesday, March 25. Copies ofthe schedule will be posted in each UTL and next to the computers in the LRC. Please respect the schedule and adhere to your time. 4. Disks are available for pick up immediately following the first lecture (until 1 p.m.) and then will available in your UTL. Please complete the information requested on the sign-up sheet when you pick up your disk. Also print your name and student number on your disk. For students working collaboratively, please be sure to sign 133

PAGE 148

for the same disk as your partner, and print both names and student numbers on the disk. Running the Program 5. The program should take approximately 2 to 2 1/2 hours to complete. You do not have to complete the program in one sitting, although it is advisable to do so. If you choose to complete the program in two or more sittings, please ensure that you pick a time that does not conflict with anyone else. 6. The program consists of an orientation section, pretest, learning phase, and posttest. The orientation discusses the learning objectives and provides valuable information about the other sections of the program--take the time to read through this section. The pretest consists ofthree cases, the learning phase contains six cases, and the posttest includes three cases. The pretest provides a baseline of your knowledge and will be used to judge the effectiveness of the program through comparison with the posttest. The on-line resources available during the learning portion of the program are not available during the pretest or posttest. Your pretest and posttest scores will be available for viewing after completing the three posttest cases. Analyses of the pretest and posttest cases will be available for pick up on Friday, April 10, in the LRC. 7. A researcher may ask to observe you work--please oblige him. He will not interfere with you, but will record observational data, such as program navigation patterns, collaborative behavior patterns, etc. This information, and preand posttest comparisons, will help us modify the program to optimize its utility as a learning exercise. 8. The program prompts you to enter your name and identification number. Paired students should enter both names and identification nwnbers. 9. If the unexpected happens and you encounter technical problems, alert the LRC staff and the on-site researcher. If the problem cannot be fixed by these individuals, exit the program and contact ____ (363-2235) or_ (363-2241) who will arrange to correct the problem. 134

PAGE 149

APPENDIX E PRETEST/POSTTEST DATA SUMMARY Subject Treatment Group Pretest Posttest 1 Pair/Generative 19 32 2 Pair/Base 19 29 3 Pair/Generative 20 27 4 Pair/Base 0 22 5 Pair/Generative 10 20 6 Pair/Base 18 25 8 Single/Base 9 14 9 Single/Generative 15 25 12 Single/Base -5 19 13 Single/Generative 17 20 15 Single/Generative 10 27 16 Single/Base 18 26 17 Single/Generative 6 13 18 Single/Base 17 16 19 Pair/Generative 24 26 20 Pair/Base 21 32 21 Pair/Generative 13 26 22 Pair/Base 13 26 24 Pair/Base 18 21 25 Pair/Generative 8 19 26 Pair/Base 9 18 27 Pair/Generative 15 21 28 Single/Base 3 25 30 Pair/Base 25 25 32 Single/Base 10 22 33 Single/Generative 30 29 34 Single/Base 12 22 37 Single/Generative 15 13 38 Single/Base 11 18 40 Single/Base 8 22 41 Single/Generative 14 20 135

PAGE 150

Subject Treatment Group Pretest Posttest 42 Pair/Base 18 23 43 Pair/Generative 11 18 44 Pair/Base 11 32 45 Pair/Generative 8 24 47 Pair/Generative 7 10 48 Pair/Base 16 21 49 Pair/Generative 14 21 50 Pair/Base 17 21 52 Pair/Base 19 29 54 Pair/Base 9 13 56 Pair/Base 18 21 58 Pair/Base 19 26 59 Pair/Generative 18 17 61 Single/Generative 8 16 62 Single/Base 13 29 63 Single/Generative 20 24 64 Single/Base 21 31 66 Single/Base 22 27 67 Single/Generative 11 21 68 Single/Base 33 31 69 Single/Generative 15 18 70 Single/Base 25 29 71 Single/Generative 20 27 72 Single/Base 14 28 74 Single/Base 14 26 75 Single/Generative 23 21 76 Single/Base 6 24 78 Single/Base 15 25 79 Pair/Generative 21 27 80 Pair/Base 14 27 81 Pair/Generative 16 17 82 Pair/Base 20 18 83 Pair/Generative 16 18 84 Pair/Base 13 26 88 Pair/Base 20 29 136

PAGE 151

APPENDIXF STATISTICAL INFORMATION Residual Plot of Posttest Scores c c c 30 D D D D D D D D D D D D a DC D CDC a D D D D D D D D D D D D DD D 20 D D D D D D D D D D D D D D DC Q) D D .... D D 0 0 D (J) D D D Cii 10 D 0 1-u; Q) ::: Ill 0 0 a.. 0 20 40 60 80 100 DISK# 137

PAGE 152

Histogram ofPosttest Scores 16r-----------------------------. 14 12 10 8 6 4 2 0 12.5 17.5 22.5 27.5 32.5 Posttest Total Score Stem and Leaf Plot Analysis Posttest Total Score Stem-and-Leaf Plot Frequency Stem & Leaf 5.00 1 03334 12.00 1 667788888999 20.00 2 00011111111222223444 24.00 2 555566666667777778999999 5.00 3 11222 Stem width: 10.00 Each leaf: 1 case(s) 138 Std. Dev = 5.18 Mean= 22.9 N = 66.00

PAGE 153

Correlation Graphs 30 l s 0 .... I a: 0 10 20 30 PosHesl T 01111 Score o i .... i c ii5 .. .... iii ., CL. 10 20 30 o PosHesl Tolal Score Single/Base Trealmenl or-----------------------, .... j 0 PosHesl Total Score Single/Gen8111tive Trealmenl 139

PAGE 154

! 1.. IL .. ]j IL ! :. 30 10 0 -10 10 20 30 40 PosHest Total Score Pair/Base Tratment 0 10 20 30 40 PosHest Total ScoreTreatment 140

PAGE 155

Component Analysis: Generative and Base Programs Base Generative Total Score-Labs 37 6.210* 27 4.841 Total Score-Assessment 37 4.124* 27 2.097 Total Score-Management 37 7.164* 27 1.880 Test Case A Total 37 4.864* 27 2.515 Labs 37 4.396* 27 2.235 Assessment 37 1.859 27 .171 Management 37 1.743 27 .465 Test Case B Total 37 5.875* 27 3.849* Labs 37 3.553* 27 2.580 Assessment 37 3.019* 27 2.725 Management 37 3.064* 27 .779 Test Case C Total 37 3.903* 27 2.869* Labs 37 2.543 27 2.733 Assessment 37 1.526 27 -1.072 Management 37 4.933* 27 3.057* p < .005 141

PAGE 156

Time APPENDIXG SAMPLE STUDENT COMMENTS Too much time 1. Too time consuming to be incorporated into the rest of the medical school schedule with too low a return of knowledge per time spent. 2. Too much time involved. Computers too slow 1. I found this program exceptionally slow. 2. These computers are way to slow to make this program effective. Not enough time 1. I probably would have spent more time and learned a little more if I didn't have a test tomorrow. 2. The program could be good if we had more time to spend on it. Instructional Method Do not like computer-based instruction 1. Questions are not possible using this method of instruction. 2. Rob does not like computer-based learning. Prefer lecture 1. Our low scores on the post test were due to not learning very specific details which in my opinion would be better learned from a handout or book. 2. I think the material could be learned in less time with simple text learning. Did not learn from program 1. Our evaluation of the program reflects our disbelief that our answers were incorrect. If this is true, we did not learn a damn thing. 2. Disappointingly, our posttest scores were barely better than our pretest. Posttest cases differ from practice cases 1. The learning cases did not seem to correlate with the test cases as the test was HCV and the learning was HBV. We were just as lost on posttest cases 1 & 2 as we were before we began the learning. 142

PAGE 157

2. It would have been beneficial to have the cases in the pre and posttests in the learning exercises, so we can definitely improve. Program is a good educational tool 1. This program was effective in making me think about how to order tests and learn information about patients .... This was an excellent exercise to practice clinical-based medicine for next year. 2. It was a pretty good program for integrating knowledge of lab tests and physiologic conditions which we have studied. I thought it did a pretty good job of helping me to think through decisions and the rationale behind different lab tests/interventions .... I liked how it concentrated on decision making. Program Specific Feedback Lack of immediate feedback on test cases 1. We felt extremely discouraged by the fact that there was no feedback on the answers we got wrong (which were numerous)-what were the correct answers? 2. It would have been nice to have explanations for the answers on the exams. Inconvenient access to information sources 1. Information in this program is way too inconvenient to access. 2. It was too time consuming and difficult to find the information I needed. Inability to change answers 1. This program must have the ability to take back answers. 2. I would like to be able to "unclick" an answer ifl change my mind. Collaboration Preferred 1. Please make the definitive decision that people work better and learn more in pairs .. .I know I would have learned much, much more had I been working with a partner. 2. We definitely enjoyed the group learning experience and found it very helpful. Not preferred 1. It would be better not to try to force people to do it in pairs. 143

PAGE 158

REFERENCES Bednar, A. K., Cunningham, D., Duffy, T. M., & Perry, J. D. (1992). Theory into practice: How do we link? InT. M. Duffy & D. H. Jonassen (Eds.), Constructivism and the technology of instruction: A conversation (pp. 17-34). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Bednar, A. K., Cunningham, D., Duffy, T. M., & Perry, J. D. (1995). Theory into practice: How do we link? In G. J. Anglin (Ed.), Instructional technology: Past. present. and future (2nd ed.) (pp. 100-112). Englewood, CO: Libraries Unlimited. Bransford, J.D., Franks, J. J., Vye, N.J., & Sherwood, R. D. (1989). New approaches to instruction: Because wisdom can't be told. In S. Vosniadou & A. Ortony (Eds.), Similarity and analogical reasoning (pp. 470-497). Cambridge, UK: Cambridge University Press. Bransford, J.D., & Vye, N.J. (1989). A perspective on cognitive research and its implications for instruction. In L. B. Resnick & L. E. Klopfer (Eds.), Toward the thinking curriculum: Current cognitive research (pp. 173-205). Alexandria, VA: ASCD. Brown, A. L., & Campione, J. C. (1996). Psychological theory and design of innovative learning environments: On procedures, principles, and systems. In L. Schauble & R. Glaser (Eds.), Innovation in learning: New environments for education (pp. 91-127). Hillsdale, NJ: Erlbaum. Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher. 18 (1), 32-42. Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Chicago: Rand McNally & Company. Carrier, C. A., & Sales, G. C. (1987). Pair versus individual work on the acquisition of concepts in a computer-based instructional lesson. Journal of Computer-Based Instruction, 14, 11-17. 144

PAGE 159

Cognition and Technology Group (1992). An anchored instruction approach to cognitive skills acquisition and intelligent tutoring. In J. W. Regian & V. J. Shute (Eds.), Cognitive approaches to automated instruction (pp. 135-170). Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers. Crook, C. (1994). Computers and the collaborative experience of learning. New York: Routledge. Dalton, D. W., Hannafin, M. J., & Hooper, S. (1989). The effects of individual versus cooperative computer-assisted instruction on student performance and attitudes. Educational Technology Research and Development. 37 (2), 15-24. Damon, W., & Phelps, E. (1989). Critical distinctions among three approaches to peer education. International Journal ofEducational Research, 13 (1), 9-19. Dewey, J. (1938). Experience and education. New York: Collier Books. Di Vesta, F. J. ( 1989). Applications of cognitive psychology to education. In M. C. Wittrock & F. Farley (Eds.), The future of educational psychology (pp. 37-73). Hillsdale, N.J.: Erlbaum. Di Vesta, F. J., & Peverley, S. (1984). The effects of encoding variability, processing activity, and rule-examples sequence on the transfer of conceptual rules. Journal ofEducational Psychology. 76 (1), 108-119. Doctorow, M. J., Wittrock, M. C., & Marks, C. B. (1978). Generative processes in reading comprehension. Journal of Educational Psychology, 70, 109-118. Duffy, T. M., & Knuth, R. A. (1990). Hypermedia and instructions: Where is the match? 4 types of hypertext use. In D. H. Jonassen & H. Mandl (Eds.), Designing hypertext for learning (pp. 199-225). Berlin: SpringerVerlag. Dunlap, J. C., & Grabinger, S. (1996). Make learning meaningful. In P. A.M. Kommers, S. Grabinger, & J. C. Dunlap (Eds.), Hypermedia learning environments: Instructional design and integration (pp. 227-238). Mahwah, N.J.: Lawrence Erlbaum Associates. Feltovich, P. J., Spiro, R. J., & Coulson, R. L. (1989). The nature of conceptual understanding in biomedicine: The deep structure of complex ideas and the development of misconceptions. In D. Evans & V. Patel (Eds.), The cognitive sciences in medicine (pp. 113-172). Cambridge, MA: MIT Press. 145

PAGE 160

Forman, E. A., & Kraker, M. J. (1985). The social origins oflogic: The contributions ofPiaget and Vygotsky. In M.W. Berkowitz (Ed.), Peer conflict and psychological growth (pp. 23-39). San Francisco, CA: Jossey-Bass Inc., Publishers. Gokhale, A. A. (1995). Collaborative learning enhances critical thinking. Journal of Technology Education [On-line], 1 (1). Available: http://scholar.lib.vt.edu/ejournals/JTE Grabinger, R. S. (1996) Rich environments for active learning. In D. H. Jonassen (Ed.), Handbook of research for educational communications and technology (pp. 665-692 ). New York: Simon & Schuster Macmillan. Grabowski, B. L. (1996). Generative learning: Past, present, and future. In D. H. Jonassen, (Ed.), Handbook of research for educational communications and technology (pp. 897-918). New York: Simon & Schuster Macmillan. Grabowski, B. L. (1997). Mathemagenic and generative learning theory: A comparison and implications for designers. In C. R. Dills & A. J. Romiszowski (Eds.) Instructional development paradigms (pp. 257-267). Englewood Cliffs, NJ: Educational Technology Publications. Hannafin, M. J. (1992). Emerging technologies, lSD, and learning environments: Critical perspectives. Educational Technology Research & Development. 40 (1), 49-63. Hannafin, M. J., Hannafin, K. M., Hooper, S. R., Rieber, L. P., & Kini, A. S. (1996). Research on and research with emerging technologies. In D. H. Jonassen (Ed.) Handbook of research for educational communications and technology (pp. 378-402). New York: Simon & Schuster Macmillan. Hannafin, M. J., & Rieber, L. P. (1989). Psychological foundations of instructional design for emerging computer-based instructional technologies: Part I. Educational Technology Research & Development, 37, 91-101. Harlen, W., & Osborne, R. (1985). A model for learning and teaching applied to primary science. Journal of Curriculum Studies, 17 (2), 133-146. 146

PAGE 161

Hartman, D. K., & Spiro, R. J. (1989). Explicit text structure instruction for advanced knowledge acquisition in complex domains: A post-structuralist perspective. Paper presented at the annual meeting of the American Educational Research Association. San Francisco, CA. Hooper, S. (1992). Cooperative learning and computer-based instruction. Educational Technology Research and Development. 40 (3), 21-38. Hooper, S. & Hanafin, M. J. (1991). Psychological perspectives on emerging instructional technologies: A critical analysis. Educational Psychologist, 26, 69-95. Hooper, S., Sales, G, & Rysavy, S. (1994). Generating summaries and analogies alone and in pairs. Contemporary Educational Psychology, 19, 53-62. Hooper, S., Temiyakarn, C., & Williams, M. D. (1993). The effects of cooperative learning and learner control on highand average-ability students. Educational Technology Research & Development. 41 (2), 5-18. Jacobson, M. J. (1990). Knowledge acquisition, cognitive flexibility, and the instructional applications of hypertext: A comparison of contrasting designs for computer-enhanced learning environments. [CD-ROM] Abstract from: PsycLit Item: AAG9124433. Dissertation Abstracts International. 52 (06). Jacobson, M. J., Maouri, C., Mishra, P., & Kolar, C. (1995). Learning with Hypertext learning environments: Theory, design, and research. Journal of Educational Multimedia and Hypermedia. 4 (4), 321-364). Jacobson, M. J., & Spiro, R. J. (1994). A framework for the contextual analysis oftechnology-based learning environments. Journal of Computing in Higher Education. 5 (2), 3-32. Jacobson, M. J., & Spiro, R. J. (1995). Hypertext learning environments, cognitive flexibility, and the transfer of complex knowledge: An empirical investigation. Journal of Computing Research. 12 (4), 301-333. Johnsey, A., Morrison, G. R., & Ross, S.M. (1992). Using elaboration strategies training in computer-based instruction to promote generative learning. Contemporary Educational Psychology, 17, 125-135. 147

PAGE 162

Johnson, D. W., & Johnson, R. T. (1989). Cooperation and competition: Theory and research. Edina, MN: Interaction Book Company. Johnson, D. W., & Johnson, R. T. (1992). Implementing cooperative learning. Contemporary Education. 63 (3 ), 173-180. Johnson, D. W., Maruyama, G., Johnson, R., Nelson, D., & Skon, L. (1981). Effects of cooperative, competitive, and individualistic goal structures on achievement: A meta-analysis. Psychological Bulletin, 89, 47-62. Johnson, R. T., Johnson, D. W., & Stanne, M. B. (1985). Effects of cooperative, competitive, and individualistic goal structures on computer-assisted instruction. Journal of Educational Psychology, 77, 668"'677. Johnson, R. T., Johnson, D. W., & Stanne, M. B. (1986). Comparison of computer-assisted cooperative, competitive, and individualistic learning. American Educational Research Journal. 23 (3), 382-392. Jonassen, D. H. (1988). Integrating learning strategies into courseware to facilitate deeper processing. In D. H. Jonassen (Ed.), Instructional designs for microcomputer courseware (pp. 151-181 ). Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers. Jonassen, D. H. (1991). Objectivism versus constructivism: Do we need a new philosophical paradigm? Educational Technology Research & Development, 39 (3), 5-14. Jonassen, D. H. (1994). Thinking technology: Toward a constructivist design model. Educational Technology. 34 (3), 34-37. Jonassen, D. H., Ambruso, D. R., & Olesen, J. (1992). Designing a hypertext on transfusion medicine using cognitive flexibility theory. Journal of Educational Multimedia and Hypermedia 1, 309-322. Jonassen, D. H. & Grabinger, R. S. (1993). Applications ofhypertext: Technologies for higher education. Journal of Computing in Higher Education. 4 (2), 12-42. King, A. (1989). Verbal interaction and problem-solving within computer Assisted cooperative learning groups. Journal of Educational Computing Research, 5 (1), 1-15. 148

PAGE 163

Kolodner, J. L. (1993). Case-based reasoning. San Mateo, CA: Morgan Kaufmann Publishers, Inc. Krajcik, J. S., Blumenfeld, P. C., Marx, R. W., & Soloway, E. (1994). A collaborative model for helping middle grade science teachers learn project based instruction. The Elementary School Journal. 94 (5), 483-497. Linden, M., & Wittrock, M. C. (1981). The teaching of reading comprehension according to the model of generative learning. Reading Research Quarterly. 17 (1), 44-57. Lomax, R. G. (1992). Statistical concepts: A second course for education and the Behavioral sciences. White Plains, N.Y.: Longman Publishing Group. Makuch, J. R., Robillard, P. D., & Yoder, E. P. (1992). Effects ofindividual versus paired/cooperative computer-assisted instruction on the effectiveness and efficiency of an in-service training lesson. Journal of Educational Technology Systems. 20 (3), 199-208. Mevarech, Z. R. (1993). Who benefits from cooperative computer-assisted instruction? Journal of Educational Computing Research. 9 ( 4 ), 451-464. Nastasi, B. K., & Clements, D. H. (1991). Research on cooperative learning: Implications for practice. School Psychology Review, 20 (1), 110-131. Nastasi, B. K., & Clements, D. H. ( 1992). Social-cognitive behaviors and higher order thinking in educational computer environments. Learning and Instruction, 2, 215-238. Nastasi, B. K., & Clements, D. H. (1993). Motivational and social outcomes of cooperative computer education environments. Journal of Computing in Childhood Education. 4 (1), 15-43. Osborne, R. J., & Wittrock, M. C. (1985). The generative learning model and its implications for science education. Studies in Science Education, 12, 59-87. Perkins, D. (1992). Smart schools: From training memories to educating minds. New York: The Free Press. 149

PAGE 164

Perkins, D. (1992). Technology meets constructivism: Do they make a marriage? InT. M. Duffy & D. H. Jonassen (Eds.), Constructivism and the technology of instruction: A conversation (pp. 45-55). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Perkins, D. (1992). What constructivism demands ofthe learner. InT. M. Duffy & D. H. Jonassen (Eds.), Constructivism and the technology of instruction: A conversation (pp. 161-165). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Phillips, D. C., & Soltis, J. F. (1991). Perspectives on learning (2nd ed.). New York: Teachers College Press. Reigeluth, C. M. (1995). Educational systems development and its relationship to ISD. In G. J. Anglin (Ed.), Instructional technology: Past, present. and future (2nd edition) (pp. 84-93). Englewood, CO: Libraries Unlimited, Inc. Reiser, R. A., & Salisbury, D. F. (1995). Instructional technology and public education in the United States: The next decade. In G. J. Anglin (Ed.), Instructional technology: Past. present. and future (pp. 254-262). Englewood, CO: Libraries Unlimited, Inc. Resnick, L. B. ( 1987). Learning in school and out. Educational Researcher, 13-20. Resnick, L. B. (1989). Introduction. In L. B. Resnick (Ed.), Knowing. learning. and instruction: Essays in honor of Robert Glaser, (pp. 1-24). Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers. Rieber, L. P. (1992). Computer-based microworlds: A bridge between constructivism and direct instruction. Educational Technology Research & Development. 41 (1), 93-106. Salomon, G. (1985). Information technologies: What you see is not (always) what you get. Educational Psychologist. 20 ( 4 ), 207-216. Salomon, G., & Globerson, T. (1987). Skill may not be enough: The role of mindfulness in learning and transfer. International Journal ofEducational Research, 11, 623-638. Salomon, G., & Globerson, T. (1988). When teams don't function they way they ought to. International Journal ofEducational Research, 13 (10), 89-99. 150

PAGE 165

Scardamalia, M., & Bereiter, C. (1994). Computer support for knowledge Building communities. The Journal of the Learning Sciences. 3 (3), 265-283. Shlechter, T. M. (1990). The relative instructional efficiency of small-group computer-based training. Journal of Educational Computing Research. 6 (3), 329-341. Sherman, G. P., & Klein, J.D. (1995). The effects of cued interaction and ability grouping during cooperative computer-based science instruction. Educational Technology Research & Development. 43 (4), 5-24. Slamecka, N.J., & Graf, P. (1978). The generation effect: Delineation of a phenomenon. Journal of Experimental Psychology: Human Learning and Memory, 4, 592-604. Slavin, R. E. (1996). Research on cooperative learning and achievement: What we know, what we need to know. Contemporary Educational Psychology, 21, 43-69. Spiro, R. J., Coulson, R. L., Feltovich, P. J., & Anderson, D. K. (1988). Cognitive flexibility theory: Advanced knowledge acquisition in ill-structured domains. Proceedings of the 1Oth Annual Conference of the Cognitive Science Society. pp. 375-383. Spiro, R. J., Feltovich, P. J., Coulson, R. L., & Anderson, D. K. (1989). Multiple analogies for complex concepts: Antidotes for analogy-induced misconception in advanced knowledge acquisition. InS. Vosniadou & A. Ortony (Eds.), Similarity and analogical reasoning (pp. 489-531 ). Cambridge, UK: Cambridge University Press. Spiro, R. J., & Jehng, J. (1990). Cognitive flexibility and hypertext: Theory and technology for the nonlinear and multidimensional traversal of complex subject matter. In D. Nix & R. J. Spiro (Eds.), Cognition. education. and multimedia: Explorations in high technology (pp. 163-205). Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers. Spiro, R. J., Vispoel, W. P., Schmitz, J. G., Samarapungavan, A., & Boerger, A. E. (1987). Knowledge acquisition for application: Cognitive flexibility and transfer in complex content domains. In B. K. Britton & S.M. Glynn (Eds.), Executive control processes in reading (pp. 177 -199). Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers. 151

PAGE 166

SPSS Incorporated (1996). Statistical Package for the Social Sciences Release 7.5 for Windows [software program]. Chicago. IL: SPSS Inc. Stein, B. S., & Bransford, J.P. (1979). Constraints on effective elaboration: Effects of precision and subject generation. Journal ofVerbal Learning and Verbal Behavior. 18 (6), 769-777. Stodolsky, S. S. (1984). Frameworks for studying instructional processes in peer work-groups. In P. L. Peterson, L. C. Wilkinson, & M. Hallinan (Eds.), The social context of instruction: Group organization and group processes {pp. 107-124). Orlando, FL: Academic Press, Inc. Teasley, S.D., & Roschelle, J. (1993). Constructing a joint problem space: The computer as a tool for sharing knowledge. InS. Lajoie & S. Derry (Eds.), Computers as cognitive tools (pp. 229-258), Hillsdale, NJ: Lawrence Erlbaum Associates. Vygotsky, L. S. (1978). Internalization of higher cognitive functions. In M. Cole, V. John-Steiner, S. Scribner, & E. Souberman (Eds.), Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press. Watson, J. (1990). Cooperative learning and computers: One way to address Student differences. The Computing Teacher, 18, 9-15. Webb, N. M. (1985). Cognitive requirements of learning computer programming in group and individual settings. Association for Educational Data Systems Journal, 18, 183-194. Webb, N. M. (1987). Peer interaction and learning with computers in small groups. Computers in Human Behavior, 3, 193-209. Webb, N. M. (1989). Peer interaction and learning in small groups. International Journal ofEducational Research. 13,21-39. Webb, N. M., & Lewis, S. (1988). The social context oflearning computer programming. In R. E. Mayer (Ed.), Teaching and learning computer programming: Multiple research perspectives, (pp. 179-206). Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers. 152

PAGE 167

Weinstein, C. E. (1978). Elaboration skills as a learning strategy. In H. F. O'Neil, Jr., Learning Strategies (pp.31-55). New York: Academic Press. Whipple, W. R. (1987). Collaborative learning: Recognizing it when we see it. AAHE Bulletin, 3-6. Whitehead, A. N. (1929). The aims of education. New York, NY: MacMillan. Wilson, B. G. (Ed.), (1996). Constructivist learning environments: Case studies in instructional design. Englewood Cliffs, NJ: Educational Technology Publications. Wittrock, M. C. (1974a). Learning as a generative process. Educational Psychologist, 11 (2), 87-95. Wittrock, M. C. (1974b). A generative model ofmathematics education. Journal for Research in Mathematics Education, 5 ( 4 ), 181-196. Wittrock, M. C. ( 1985). Teaching learners generative strategies for enhancing reading comprehension. Theory into Practice, 24 (2), 123-126. Wittrock, M. C. (1990). Generative processes of comprehension. Educational Psychologist, 24, 345-376. Wittrock, M. C. ( 1992). Generative learning processes of the brain. Educational Psychologist, 27, 531-541. Wittrock, M. C., & Alesandrini, K. (1990). Generation of summaries and analogies and analytic and holistic abilities. American Educational Research Journal, 27, 489-502. Yager, S., Johnson, D. W., & Johnson, R. T. (1985). Oral discussion, group-to-individual transfer, and achievement in cooperative learning groups. Journal ofEducational Psychology. 77 (1), 60-66. 153