Citation
Problem-based learning instruction versus traditional instruction on self-directed learning, motivation, and grades of undergraduate computer science students

Material Information

Title:
Problem-based learning instruction versus traditional instruction on self-directed learning, motivation, and grades of undergraduate computer science students
Creator:
LeJeune, Noel F
Publication Date:
Language:
English
Physical Description:
xii, 184 leaves : ; 28 cm

Subjects

Subjects / Keywords:
College students ( lcsh )
Computer science -- Study and teaching (Higher) ( lcsh )
Problem-based learning ( lcsh )
Self-culture ( lcsh )
Motivation in education ( lcsh )
College students ( fast )
Computer science -- Study and teaching (Higher) ( fast )
Motivation in education ( fast )
Problem-based learning ( fast )
Self-culture ( fast )
Genre:
bibliography ( marcgt )
theses ( marcgt )
non-fiction ( marcgt )

Notes

Bibliography:
Includes bibliographical references (leaves 168-184).
General Note:
School of Education and Human Development
Statement of Responsibility:
by Noel F. LeJeune.

Record Information

Source Institution:
|University of Colorado Denver
Holding Location:
|Auraria Library
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
50741524 ( OCLC )
ocm50741524
Classification:
LD1190.E3 2002d .L53 ( lcc )

Full Text
PROBLEM-BASED LEARNING INSTRUCTION VERSUS TRADITIONAL
INSTRUCTION ON SELF-DIRECTED LEARNING, MOTIVATION, AND
GRADES OF UNDERGRADUATE COMPUTER SCIENCE STUDENTS
by
Noel F. LeJeune
B. S., Louisiana State University, 1970
B.S., Metropolitan State College, 1988
MCIS, University of Denver, 1993
A thesis submitted to the
University of Colorado at Denver
in partial fulfillment
of the requirements for the degree of
Doctor of Philosophy
Educational Leadership and Innovation
2002


This thesis for the Doctor of Philosophy
degree by
Noel F. LeJeune
has been approved
by
Ellen Stevens
Laura Goodwin
Michael Marlow
Date


LeJeune, Noel F. (Ph.D., Educational Leadership and Innovation)
Problem-Based Learning Instruction Versus Traditional Instruction on Self-
Directed Learning, Motivation, and Grades of Undergraduate Computer Science
Students
Thesis directed by Associate Professor Ellen Stevens
ABSTRACT
A problem-based learning (PBL) teaching method was compared with a
traditional lecture-based teaching method to determine the effects on
undergraduate Computer Science (CS) students self-directed learning (SDL) and
programming assignment grades. An integrated construct of SDL included a)
SDL readiness b) SDL skills, c) SDL performance, and d) students course
motivation.
Quasi-experimental designs were used to compare a PBL teaching method
and a traditional lecture-based method in two sections of a CS1 course taught by
the same instructor. Each of the SDL components and grades were measured for
students experiencing traditional instructional methods and problem-based
learning methods. Readiness was measured with the Self-Directed Learning
Readiness Scale, skills with the Motivated Strategies for Learning
Questionnaire-Part B, performance with time spent on SDL tasks, and course
motivation with the Motivated Strategies for Learning Questionnaire-Part A. The
grade measurement was the course instructors percentage score given to
students programming assignments.
Results showed that the effect of teaching method was statistically
significant for the SDL performance measure with the PBL section
in


demonstrating greater performance. The effect of teaching method was not
significant on SDL readiness, skills or course motivation measures. A lack of
statistical differences between the two methods for these measures was attributed
to no effect of PBL on students SDL or small sample size resulting in reduced
statistical power or lack of student engagement in PBL resulting in ineffective
treatment.
The effects of method, time, and method x time interaction were significant
on the grades measure. The traditional teaching method group had higher grades
than the PBL group. Both teaching methods exhibited declining grades over time.
Factors such as increased difficulty of assignments and stricter grading schemes
over time or differing characteristics of group members such as prior CS
knowledge, age, time spent on assignments, and competing employment and
other course demands were identified.
Recommended future study includes improved measures of students SDL
practices rather than students own perceptions, assessment of student practice of
PBL, and qualitative study of students motivation and SDL performance.
This abstract accurately represents the content of the candidates thesis. I
recommend its publication.
Signed
Ellen Stevens
IV


DEDICATION
I dedicate this dissertation to my wife, Lynn Callaway, who, having
completed her dissertation, never let me forget the definition of a good
dissertation.


ACKNOWLEDGEMENT
The faculty of the Graduate School of Education at the University of
Colorado at Denver has provided wonderful support and inspiration in my
studies. Particular thanks goes to my dissertation committee members, Laura
Goodwin, Jim Loats, and Mike Marlow. Special appreciation goes to my advisor,
Ellen Stevens, for her guidance, support, wisdom, and knowledge so freely
shared throughout my entire doctoral program. The PSTL Lab partners have
been a valued source of discourse and support.
Dr. Patricia Tucker, in addition to providing encouragement and support,
allowed me to experiment with her classes and accommodated many requests
that made the research possible. Professor Ruth Yara has also been a strong
supporter and source of inspiration for many years. Dr. Charlotte Murphy, Chair
of the Department of Mathematical and Computer Sciences at MSCD, is most
appreciated for her support and FRIP sponsorship in completing this work while
I served on the faculty at MSCD. My colleagues in the Department of
Mathematical and Computer Sciences at MSCD have also kept the spirit alive.
Dr. Shahar Boneh, Statistician in the MSCD Mathematical and Computer
Sciences Department, also provided excellent consultation on statistical analyses.
Finally, thanks to Dr. Tuckers students for their participation.


CONTENTS
Figures....................................................xi
Tables....................................................xii
CHAPTER
1. INTRODUCTION................................................1
Purpose..................................................1
Conceptual Framework.....................................3
Problem-Based Learning...............................5
Self-Directed Learning...............................7
Summary.............................................10
Research Questions......................................11
Methodology.............................................12
Structure...............................................13
2. LITERATURE REVIEW.........................................14
Self-Directed Learning..................................15
Definitions of Self-Directed Learning...............15
Integrated Definition of Self-Directed Learning.....17
Motivation..........................................21
Vll


Summary of Self-Directed Learning
22
Problem-Based Learning.....................................23
Definition and Characteristics of Problem-Based Learning.... 24
Learning Objectives....................................26
Tutors.................................................26
The Process............................................28
Benefits and Drawbacks of Problem-Based Learning.......29
Relationships of PBL and SDL...............................31
Summary....................................................33
3. METHODOLOGY...................................................34
Design.....................................................35
Subjects...................................................39
Setting and Materials......................................41
Independent Variable.......................................43
Problem-Based Learning Teaching Method.................43
Traditional Lecture-Based Learning Teaching Method.....44
Dependent Variables........................................45
Self-Directed Learning Readiness.......................45
Self-Directed Learning Skills..........................47
Self-Directed Learning Performance.....................50
viii


Motivation............................................52
Grades................................................53
Procedures...............................................53
Data Analysis Procedures.................................56
Summary..................................................57
4. RESULTS.....................................................58
Data Analysis............................................58
Assumptions for a Repeated Measures ANOVA.............59
Demographic Data.........................................61
Results of Major Analysis................................63
Summary of Results by Research Question..................69
Research Question 1...................................70
Research Question 2...................................71
Research Question 3...................................72
Summary..................................................74
5. DISCUSSION..................................................75
Self-Directed Learning Components........................76
Self-Directed Learning Readiness......................77
Self-Directed Learning Skills.........................79
IX


Self-Directed Learning Performance
80
Course Motivation............................82
Grades..........................................82
Problem-Based Learning Teaching Method..........86
Quality of Problem-Based Learning Treatment..88
Limitations.....................................93
Recommendations for Future Research.............94
Conclusions.....................................96
APPENDIX
A. INSTRUCTORS GUIDE: PBL EXERCISE 1...........99
B. NON-PBL EXERCISE 1..........................116
C. INSTRUCTORS GUIDE: PBL EXERCISE 2..........118
D. NON-PBL EXERCISE 2..........................137
E. DEMOGRAPHICS SURVEY.........................139
F. SDLRS QUESTIONNAIRE.........................141
G. MSLQ QUESTIONNAIRE..........................147
H. PSP TIME LOGS...............................161
I. REPEATED MEASURES ANOVA ASSUMPTIONS.........165
REFERENCES..........................................168
x


FIGURES
Figure 1.1 Problem-Based Learning Teaching Strategy................... 6
Figure 1.2. Components of Self-Directed Learning.......................9
Figure 3.1. Quasi Experimental Design for SDL Readiness, Skills, Motivation,
and Grades.......................................................36
Figure 3.2. Quasi Experimental Design for SDL Performance...........39
Figure 3.3. Calendar of Events........................................55
Figure 4.1. Programming Assignment Grades.............................69
I
Figure 5.1. Programming Assignment Grades...........................'.. 83
xi


TABLES
Table 2.1. Barrows Taxonomy of Problem-Based Learning....................25
Table 3.1. Demographic Data..............................................41
Table 3.2. MSLQ Part B Learning Strategies.............................49
Table 4.1. Demographic Data..............................................62
Table 4.2. t-Test of Demographic Data....................................62
Table 4.3. Descriptive Statistics for Self-Directed Learning Readiness...64
Table 4.4. ANOVA Table for Self-Directed Learning Readiness..............64
Table 4.5. Descriptive Statistics for Self-Directed Learning Skills......65
Table 4.6. ANOVA Table for Self-Directed Learning Skills.................65
Table 4.7. Descriptive Statistics for Self-Directed Learning Performance.66
Table 4.8. ANOVA Table for Self-Directed Learning Performance............66
Table 4.9. Descriptive Statistics for Motivation.........................67
Table 4.10. ANOVA Table for Motivation...................................67
Table 4.11. Descriptive Statistics for Grades............................68
Table 4.12. ANOVA Table for Grades.......................................68
Table 5.1. Selected Demographic Data.....................................85
Table 5.2. Problem-Based Learning Characteristics........................88
Table 5.3. Reported Total Effort (Time) on Assignments by Week...........91
xii


CHAPTER 1
INTRODUCTION
Purpose
Computer Science graduates need strongly developed problem-solving
skills, collaboration skills, and self-directed learning abilities to be successful
professionals in the workforce (Hartman & White, 1990; Shaw, 2000). Once on
the job, software developers need to continually update their knowledge to
remain competent in a world of rapid growth and change. Since constant formal
education by itself is often impractical for these continual learning challenges,
graduates who also develop self-directed learning skills will be better prepared to
respond to change than those who do not (Pomberg, 1993).
Undergraduate Computer Science education focuses on the technical
aspects but may inadequately prepare students for continued self-directed
learning (Shaw, 2000). The typical undergraduate curriculum includes
coursework covering such topics as programming, algorithms, data structures,
software design, concepts of programming languages, computer organization,
and computer architecture (Computing Science Accreditation Board, 2001).
Academia and industry (Shaw, 1976) recognize deficiencies in communication
1


skills, collaborative skills and problem-solving skills (Hartman & White, 1990;
National Science Foundation Advisory Committee, 1998). Undergraduates must
experience these soft skills as well as the technical or hard skills (Wilson,
Hoskin, & Nosek, 1993).
Providing these additional skills is necessary if Computer Science
graduates are to be successful and competent in this ever-changing profession.
Hartman (1990) suggested, ... the two most important skills which a [Computer
Science] student embarking on his career can have are communication skills and
problem solving skills. Without either of these things he is doomed to failure, or,
at best mediocrity (p. 216). Shaw (2000) added, ... traditional [Computer
Science] education makes scant provision for helping students keep their
knowledge current and that ... pressures on educational institutions will
require changes in what we teach software developers and how we teach it (p.
373). Therefore, helping students become better self-directed learners must be a
priority for Computer Science educators.
Traditional teacher-centered instruction using lecture and outside of class
programming assignments do little to foster soft skills development. Problem-
based learning, on the other hand, as a teaching technique could be a natural
extension to many existing Computer Science courses where programming
assignments are the norm. Problem-based learning provides opportunities for
both collaborative learning and the development of problem-solving skills
2


(Cockrell, Hughes Caplow, & Donaldson, 2000; Jonassen & Kwon, 2001;
Koschmann, Kelson, Feltovich, & Barrows, 1996). There are claims that
problem-based learning also develops self-directed learners (Barrows, 1994).
While this relationship seems logical, specific research on the influence of
problem-based learning teaching techniques on students' self-directed learning is
needed.
Students taught with problem-based learning are better able to apply their
knowledge (Schmidt, 1983), are better problem solvers (Albion & Gibson, 1998;
Vernon & Blake, 1993), and develop better communication skills (Lieux, 1996).
However, a question remains concerning self-directed learning: What is the
influence of problem-based learning on students self-directed learning? This
question suggests the overarching topic for this research. Furthermore, this work
focuses specifically on the use of problem-based learning teaching methods for
undergraduate Computer Science education. The conceptual framework in the
following section situates this study within the context of developing better self-
directed learners while using problem-based learning. (Detailed discussions of
problem-based learning and self-directed learning are found in chapter 2.)
Conceptual Framework
The conceptual framework provides a context for the research and the basis
for each research question. The framework describes self-directed learning as an
3


integrated construct located both within and separate from problem-based
learning.
Educating prospective computer scientists requires a more holistic
approach than merely teaching the principles and practices of the profession. The
failures and shortcomings of many graduates often result from poorly developed
soft skills rather than from deficiencies in the principles of Computer Science
(Hartman & White, 1990). The more notable soft skills students need are
problem solving, collaborative skills, well-developed communications skills, and
self-direction (Shaw, 2000).
A new approach to Computer Science education should provide both the
domain knowledge as well as opportunities to leam and develop these soft skills.
With this goal comes the question: How can we teach the core body of
knowledge while, at the same time, developing better self-directed learners? As
Shaw (1976) suggested, one possible solution involves our teaching methods. We
should employ methods that give students opportunities to develop their self-
directed learning, develop their communication skills, become more
accomplished problem solvers, and leam the subject matter. Problem-based
learning teaching methods may offer specific solutions to this conundrum.
4


Problem-Based Learning
Problem-based learning is defined as an instructional technique that uses
ill-defined, complex problems as the impetus for learning (Barrows, 1994;
Koschmann et al., 1996; Ram, 1999). Students collaboratively define learning
issues, define and use learning resources, and share acquired knowledge with the
guidance of a tutor/facilitator. Students create solutions for the problem. Plenary
sessions foster review and reflection upon the learning as well as the problem-
based learning process itself. Problem-based learning provides experience in
problem solving, collaborative work, self-direction, and teaches students subject
matter content.
Problem-based learning methods develop problem-solving skills while they
also teach students the subject matter. Furthermore, problem-based learning uses
collaborative learning, thus providing valuable experience with another critical
soft skill. Problem-based learning also uses specific self-directed learning skills
noted as the skills component of self-directed learning in Figure 1.1. Skills
such as problem recognition and learning resource identification and acquisition
that are used during problem-based learning (Barrows, 1994) are also used by
self-directed learners (Rutland & Guglielmino, 1987). These additional aspects
are illustrated in Figure 1.1.
5


Figure 1.1 Problem-Based Learning Teaching Strategy
6


The nature of the activities in problem-based learning suggests that the use
of problem-based learning teaching methods should have a positive influence on
the development of self-directed learning skills. Other facets of self-directed
learning such as readiness, actual performance, and motivation may also be
affected by the use of problem-based learning.
Self-Directed Learning
Self-directed learning has many descriptions. It is recognized as an
instructional method (Knowles, 1975), a personality attribute of the learner
(Brockett & Hiemstra, 1991; Candy, 1991; Hiemstra, 1992), or a process for
learning (Knowles, 1975). Knowles defined self-directed learning as a process
in which individuals take the initiative, with or without the help of others, to
diagnose their learning needs, formulate learning goals, identify resources for
learning, select and implement learning strategies, and evaluate learning
outcomes (1975, pg.18). Candy (1991) combines both the personal attributes of
personal autonomy and self-management with the learning activities of self-
instruction and learner-control to describe self-directed learning. Grow (1991b)
defines self-directed learners within an institutional setting as those who, within
a teacher-controlled setting, take greater charge of their own motivation, goal-
setting, learning, and evaluation (p. 203). Any comprehensive definition should
7


recognize both the personal attribute and the instructional method since they are
related and inseparable (Pilling-Cormick, 1996).
Synthesizing these views reveals self-directed learning as a combination of
capabilities and motivations of the learner. An integrated definition of self-
directed learning is key to understanding what is necessary to be a self-directed
learner. Any such view must include the following four components: 1) traits of
the learner that reflect on his or her propensity toward self-directedness (or
readiness), 2) capabilities or skills of the learner to undertake a self-directed
project; 3) the actual performance (behaviors) of the process of undertaking the
self-directed learning project, and 4) the individuals motivation toward the
learning project. Figure 1.2 illustrates the integrated construct for self-directed
learning. The psychological/personal component is the individuals readiness for
self-directed learning. The skills include both basic learning skills and those
necessary for conducting a self-directed learning activity such as defining
learning goals, finding the necessary resources, conducting the learning
activities, and self-assessing the process and learning. Performance/behavior is
the component that represents the actual doing self-directed learning. A
propensity toward self-directed learning (readiness) and having the skills to
conduct self-directed learning do not make one a self-directed learner. However,
it is the actual doing it that makes one a self-directed learner. This is the
8


performance and behaviors demonstrated by a self-directed learner. Putting these
potential attributes in practice also requires motivation.
u-OlrectadLoj,
9 ---^ ^
Figure 1.2. Components of Self-Directed Learning
Motivation plays a significant role in the practice of self-directed learning
(Long, 2001; Pintrich, 1995). While motivation is portrayed as a discrete
component within the overall concept of self-directed learning in Figure 1.2, it
overlaps each of the other components since it affects all. The dotted outline
suggests it permeates the entire construct. Motivation influences an individuals
9


perception of their skills to accomplish the task at hand. Performance is also
strongly affected by the perception of skills as well as the actual skills of the
learner (Bitterman, 1988; Confessore, 1991; Long, 2001). Self-efficacy is a key
element in ones actual performance (Hoban, Sersland, & Raine, 2001). Since
... people rarely choose to do tasks that they expect to fail (Stipek, 1998, p.
137), self-directed learning performance is tightly linked to motivation.
Motivation also varies with the context of the activity and the learners perceived
needs (Long, 2001).
This proposed concept for a multi-dimensional construct includes the
potential for self-directed learning such as Guglielminos Readiness as a
psychological or personal characteristic, the skills (actual or self-perceived), the
driving motivation factor, and the actual performance.
Summary
Computer Science education needs teaching strategies that provide the
opportunity to develop students self-directed learning while teaching the subject
matter content. A teaching strategy using problem-based learning is feasible for
many Computer Science courses, especially the many courses that require
programming assignments. With a problem-based learning teaching method, it
may be possible to better develop students self-directed learning while teaching
the subject matter. An integrated, multi-dimensional view of self-directed
10


learning includes components of personal characteristics, skills, behavior, and
motivation. The connection between problem-based learning and the components
of self-directed learning for undergraduate Computer Science students has yet to
be examined. This issue constitutes the main topic of inquiry in this work. The
specific research questions and hypotheses are stated in the next section.
Research Questions
The overarching research question for this study is: What are the changes
in undergraduate Computer Science students self-directed learning
characteristics after experiencing problem-based learning? With this question
and the conceptual framework above, specific research questions are:
1. Are there significant differences between students experiencing a problem-
based learning teaching method and students experiencing traditional lecture-
based teaching method on:
a. Students self-directed learning readiness?
b. Students self-directed learning skills?
c. Students self-directed learning performance?
d. Students course motivation?
e. Students programming assignment grades?
2. Are there significant differences across time on:
a. Students self-directed learning readiness?
11


b. Students self-directed learning skills?
c. Students self-directed learning performance?
d. Students course motivation?
e. Students programming assignment grades?
3. Is there a significant interaction between teaching method and time on:
a. Students self-directed learning readiness?
b. Students self-directed learning skills?
c. Students self-directed learning performance?
d. Students course motivation?
e. Students programming assignment grades?
Methodology
A quasi-experimental design was used to determine the differences in self-
directed learning, course motivation, and grades of undergraduate Computer
Science experiencing problem-based learning versus students experiencing
traditional teaching methods. The design compared teaching methods of
problem-based learning method (the treatment) with a traditional lecture-based
teaching method for the control group.
Two regular programming assignments for the course were modified to
create two successive problem-based learning assignments; this and the
traditional lecture method constitute the independent variable. Five dependent
12


variables, each of which originates from the research questions are self-directed
learning readiness, self-directed learning skills, self-directed learning
performance, motivation, and grades. The independent and dependent variables
are fully defined in Chapter 3, Methodology.
Structure
Chapter 1 has presented an overview of the purpose of the study,
background information suggesting the need for the study, a conceptual
framework, the research questions, operational definitions, and an overview of
the methodology. Chapter 2 provides a review of the pertinent literature. Chapter
3 describes the methodology including the design, instruments for measurement,
experimental procedures, and methods for analysis. Chapter 4 contains the
findings. Chapter 5 summarizes the findings and presents the answers to the
research questions. This final chapter also discusses the implications of these
results for future practice and future research.
13


CHAPTER 2
LITERATURE REVIEW
This chapter begins by examining self-directed learning and the
components included in an integrated view of self-directed learning. The
discussion then moves to problem-based learning as a teaching method. Next, the
Relationships section describes how self-directed learning components are
related to problem-based learning activities to suggest an expectation of positive
changes in students self-directed learning after a problem-base learning
experience as compared to the traditional teaching method.
Evidence suggests problem-based learning experiences might positively
influence students self-directed learning (Barrows, 1994; Blumberg & Michael,
1992; Ryan, 1993; Taylor, 1986) in part because problem-based learning
activities share some skills and behaviors with those of self-directed learning
(Barrows, 1994; Hmelo, Gotterer, & Bransford, 1997). For example, the goal
orientation found in descriptions of problem-based learning seems similar to
motivation as described for self-directed learning.
14


Self-Directed Learning
The study of self-directed learning has a rich and continually evolving
history in spite of a lack of a consensual definition (Bulik & Romero, 2001;
Long, 1990). Most authors credit Houles work reported in 1960 as the beginning
of modem-day investigations (Confessore & Confessore, 1992a, 1992b). Houle
(1988) proposed self-directed learners do so either to satisfy a goal, for the sake
of learning itself, or for the enjoyment of the learning environment and activity.
Tough (1978) followed with a seven-year study of frequencies and methods of
self-directed learning projects. Self-directed learning has been examined from
viewpoints ranging from psychological traits (Brockett, 1985; Brockett &
Hiemstra, 1991; Candy, 1991; Hiemstra, 1992) to instructional methods
(Knowles, 1975) to a teachinglearning situational construct (Pilling-Cormick,
1996). The complexity of self-directed learning has been acknowledged recently
by Long (2001) who posits it may involve all of those viewpoints. Some of these
definitions are briefly reviewed in the next section before discussing an
integrated view self-directed learning.
Definitions of Self-Directed Learning
Knowles recognized self-directed learning as an instructional method when
he described the processes for conducting a self-directed learning project
(Knowles, 1975). He also linked his concept of adult learning (andragogy) to
15


psychological traits he associated with adult learners, for example, a desire
among adults for greater responsibility for their own learning (Knowles, 1990).
Others with a greater emphasis on the psychological or personality perspective of
self-directed learning include Guglielmino (1977), Brockett and Hiemstra (1991),
and Candy (1991). However, Brockett and Hiemstra (1991) also recognized
instructional methods and learner traits as important. Even their Personal
Responsibility Model (PRO) emphasizing psychological characteristics
distinguished the teaching-learning transaction as "self-directed learning" while
the primary characteristics of the student were labeled "learner self-direction".
Candy (1991) also combined the personal attributes and the learning activities;
personal autonomy in the form of willingness and self-management was the
primary focus for self-directed learning. Candy, on the other hand, restricts the
learning activities of self-directed learning to non-institutional settings where the
learner had only self-imposed structure and requirements. Grow (1991b) is less
restrictive and included institutional settings as valid environments for self-
directed learning. Grow (1991b) included psychological, process, and
environmental factors when defining self-directed learners as those who, within
a teacher-controlled setting, take greater charge of their own motivation, goal-
setting, learning, and evaluation (p. 203). Pilling-Cormick (1994; 1996)
emphasized the environmental factors that were either conducive or detrimental
to self-directed learning while recognizing both personal attributes and
16


instructional methods (Pilling-Cormick, 1996). Her model for self-directed
learning primarily focused on the process with three major components:
educator, student and locus of control (student versus teacher). Crantons (1992)
description of self-directed learning emphasizes the process, outcomes, and
goals.
With many different views of self-directed learning throughout the
literature, it is difficult to arrive at a single definition. While the views seldom
conflict on substantive issues, each expert has a viewpoint or specialty that may
represent only a part of the complex whole. An integrated definition of this
complex, multi-faceted concept follows.
Integrated Definition of Self-Directed Learning
A definition restricted to any one of the many traits associated with self-
directed learning is inappropriate. Long (2001, p. 10) offered a restatement of his
theoretical position concerning self-directed learning by recognizing the
complexity of the topic. An integrated, possibly holistic, view emerges.
Many variables may affect the manifestation of self-direction. They seem
to include, but are not limited to, [italics added] (a) personality and other
psychological constructs, (b) aptitude and familiarity with the content to be
learned, (c) learning context including powerful others' expectations,
teaching techniques employed, and degree of learner autonomy and
control. Other variables are (d) social relation with other learners,
facilitator, and other resource people, and (e) immediate personal and
professional situation in which the learner finds himself or herself
17


Not only is the complexity recognized in his statement, but also the need to
address other components is noted with the not limited to statement. The most
commonly discussed aspect of self-directed learning has been the
psychological/personal trait listed first in Longs list. A skills component is
suggested within Longs items (b) and (d) while motivation appeared in item (c).
He also stated motivation may be more important than current research indicates
by the few studies dealing with the topic (p. 9).
The integrated construct of self-directed learning included a combination of
a) psychological/personal traits for self-directedness, b) skills or capabilities for
conducting ones own learning projects, c) performance/behaviors applying those
skills to the self-directed learning activities, and d) motivation for the particular
learning project. While there were likely other variables as Long suggests, these
four were the most prominent and significant throughout the literature. The
remainder of this section discusses each of these four components.
Personal/Psychological Characteristics. Guglielminos (1977) definition
focused on personal characteristics represented in the Self-Directed Learning
Readiness Scale (SDLRS). The eight characteristics are:
1. Openness to learning opportunities
2. Self-concept as an effective learner
3. Initiative and independence in learning
4. Informed acceptance of responsibility for ones own learning
18


5. Love of learning
6. Creativity
7. Positive orientation to the future, and
8. Ability to use basic study skills and problem-solving skills
These eight factors incorporate two of the integrated characteristics
personality and skill; seven of the eight are related to personality. This leaves
performance and motivation missing from Guglielminos conception of self-
directed learning. However, over 70% of the self-directed learning
methodological research from the last two decades focused on the Self-Directed
Learning Readiness Scale (Brockett et al., 2000).
Self-Directed-Leaming Skills. Some set of skills is necessary to conduct a
self-directed learning project whether as a completely independent project or
within a formal institutional setting. The specific skills can be inferred from
analyzing the process followed by self-directed learners. Knowles (1975)
described five activities that represent the core skills necessary:
The process in which individuals take the initiative with or without the help
of others, in diagnosing learning needs, formulating learning goals,
identifying human and material resources for learning, choosing and
implementing learning strategies, and evaluating learning outcomes (p. 18).
19


First in practice is formulating learning goals. The learner must have the
ability to determine the goals from the context of the situation. Often this implies
defining the problem that must be solved. Once the problem is known, a self-
directed learner must recognize the knowledge and skills that must be acquired to
solve the problem. A comparison of what one currently knows and does not
know constitutes diagnosing learning needs. Knowles referred to this activity as a
Gap Analysis (Knowles, 1975). After diagnosing learning needs, self-directed
learners must have the ability to identify human and material resources for
learning. Two sets of skills are needed; those to identify resources and those to
use the resources. The use of resources transitions into the actual skills for
learning. This represents the tasks of choosing and implementing appropriate
learning strategies. Learning strategies include such activities as rehearsal,
elaboration, organization, critical thinking, and metacognitive self-regulation
(Pintrich & DeGroot, 1990). Ancillary learning strategy skills required for the
successful self-directed learner involve resource management, effort regulation,
help seeking, peer learning, and time management (Pintrich, Smith, Garcia, &
McKeachie, 1991).
Self-Directed-Leaming Performance/Behavior. A self-directed learner not
only has the readiness and skills for self-directed learning, he or she does it.
Students can be guided through the activities and taught the skills of self-directed
learning (Grow, 1991c; Rutland & Guglielmino, 1987). Candy (1991) asserted,
20


one leams responsibility and self-direction through experiences in which one is
given the opportunity to be self-directed and responsible for ones actions (p.
319). One commonly used tool for practicing self-directed learning with a well-
structured process is the learning contract (Blackwood, 1994; Caffarella, 1983;
Caffarella & Caffarella, 1986; Guglielmino & Guglielmino, 1994). Learning
contracts make the process of self-directed learning explicit and visible. Grows
(1991b) model for teaching self-direction suggests several other techniques that
emphasize matching the students level of self-direction with corresponding
teaching methods and classroom activities. This form of scaffolding keeps
students within their zone of proximal development (Vygotsky, 1978) for
optimizing learning, avoiding frustration, and positively contributing to student
motivation (Pintrich & Schunk, 1996, pp. 74, 175).
Motivation
The motivations for self-directed learning were first described by Houle
(1961). He suggested three reasons learners pursue self-directed learning
projects: a) to satisfy a goal or need (goal-oriented), b) for the love of learning
(learning-oriented), or c) for the experience and enjoyment of the learning
activities and associated ambience (activity-oriented). Houle acknowledged these
motivating reasons are not mutually exclusive so a learner may be moved to
participate in self-directed learning projects by combinations of these. Of these,
21


goal-orientation has been identified as a significant factor in self-directed
learners pursuing degrees from higher education institutions (Grow, 1991b;
Ponton, Carr, & Confessore, 2000).
Goal orientation was seen as the students perception of reasons for
engaging in the learning task. While it is only one of several components of
motivation, it appeared especially significant to self-directed learners (Bitterman,
1988). Other internal motivational factors included the perceived value of the
task as the learners evaluation of how interesting, how important, and how
useful the task itself is. Self-directed learners generally recognize a need and are
able to perceive value in the tasks. Expectancy components of motivation
included students belief that their efforts would result in positive outcomes, that
their performance expectations would be met, and that their self-efficacy for the
task was sufficient (Pintrich et al., 1991). Extrinsic goal orientation factors
include grades, rewards, performance assessment, and evaluations.
Summary of Self-Directed Learning
An integrated view of self-directed learning is preferable to one that
examines a single aspect of the concept. However, the possible number of
components and their relationships suggests a holistic viewpoint may be more
accurate than a simple integrated one. Nevertheless, a reasonable reduction of
complexity results in a model with four predominate variables standing out in the
22


literature. This model for self-directed learning is comprised of
psychological/personal traits, a set of skills, a recognizable
performance/behavior, and motivation.
Problem-Based Learning
Problem-based learning originated with medical education at McMaster
University in the mid-1960s with the intent of improving students problem
solving skills while teaching basic subject matter content (Caplow, Donaldson,
Kardash, & Hosokawa, 1997). The fundamental precept was that learning
proceeded from the need to know in order to solve a problem, thus enhancing
learning. Charlin, Mann, and Hansen (1998) said that problem-based learning (a)
requires active processing of information, (b) activates prior knowledge, (c)
provides a meaningful context, and (d) stimulates opportunities for elaboration
and organization of knowledge. In addition to these learning benefits, problem-
based learning provided experiences in problem solving, opportunities for
collaborative work, and use of communications skills.
Many other disciplines including engineering, education, and the sciences
have experimented with or adopted problem-based learning methods as part of
courses or entire programs (Allen, Duch, & Groh, 1996; Arambula-Greenfield,
1996; Cawley, 1997; Groh, 2000; Grundy, 1996; Todd, 1997; Woods, 1996).
23


Definition and Characteristics of Problem-Based Learning
Although there are many variations, Albanese and Mitchell (1993, p. 53)
defined problem-based learning as an instructional method characterized by
the use of patient problems as a context for students to leam problem-solving
skills and acquire knowledge about the basic and clinical sciences. Problem-
based learning has been distinguished from other problem-centered methods such
as the case method, in that the problem provides the motivation for learning basic
concepts. The problem is presented before the learner is exposed to the subject or
content knowledge. The need to understand the problem drives learning.
As early as 1986, Barrows (1986)offered a taxonomy for problem-based
learning recognizing ... the many variables possible can produce wide
variations in quality and in the educational objectives that can be achieved (p.
481). This taxonomy can ... help teachers choose a problem-based learning
method most appropriate for their students (p. 481). Table 2.1 illustrates the
range of problem-based learning instruction and provides a basis for selecting
characteristics to incorporate in problem-based learning. This taxonomy
represents the broadest range ofwhat may be included in problem-based learning.
24


Table 2.1. Barrows Taxonomy of Problem-Based Learning.
Lecture-Based Cases Teacher presents information in lectures plus a case or two (vignettes) to demonstrate relevance Not usually considered problem-based learning.
Case-Based Lectures Case vignettes or more complete case histories are presented before lecture. Students analyze existing knowledge prior to lectures of new material.
Case Method Students are given a complete case for study and research in preparation for subsequent class discussion.
Modified Case-Based Small tutorial groups of students are presented with a case. Students pursue limited lines of inquiry from alternatives presented. Additional information is provided as requested by students.
Problem- Based Problems are presented within an authentic context. Students use free inquiry. Active, teacher-guided exploration and evaluation using facilitation and tutorial skills is used.
Closed-Loop, Problem- Based Problem-based as above with iteration cycles where each cycle concludes with students reflecting and evaluating 1) resources used, 2) reasoning processes followed, and 3) information acquired (learning).
The problem. The problem is central to the concept of problem-based
learning. Authentic problems in an authentic context are used to develop content
knowledge, problem solving skills, collaborative skills, and learner self-direction
(Barrows & Tamblyn, 1980). The problems, in addition to being crafted with the
learning objectives in mind, are complex, and ill-structured (Barrows, 1994;
Koschmann et al., 1996; Stepien & Pyke, 1997). These types of problems are not
fully or clearly understood by the students at the outset. Learners must extract or
define the problem from the body of information initially available. As more
information is acquired, the problem definition is likely to evolve along with a
25


better understanding of both the problem and the knowledge needed for
resolution. Another problem characteristic is that there may not be a single,
simple, and correct solution or the correct solution is not likely to be known
except in hindsight. Finally, these types of problems are not likely to have a fixed
or previously established procedure for reaching a solution. The problem-solvers
must, at least, define an approach to the problem from very high level approaches
and, at most, determine an entirely new approach.
Learning Objectives
Learning objectives must be incorporated within the problem. Defining
learning objectives for a problem-based learning experience range from wholly
student defined to wholly teacher defined (Blumberg, Michael, & Zeitz, 1990;
Blumberg & others, 1990; Duek & Wilkerson, 1991). In any case, the problem is
designed to incorporate learning objectives appropriate to the module, course or
program (Dolmans, 1993; Dolmans, Snellen-Balendong, Wolfhagen, & van der
Vleuten, 1997; Stepien & Pyke, 1997).
Tutors
Effective problem-based learning methods do not rely on students
following the process without direction and support. Tutors provide guidance and
direction by working closely with each small group during the problem
26


identification, learning issues definition, and reflection activities. An important
responsibility of the tutor is to emphasize the problem-based learning processes
rather than teach subject matter content. The tutors primary role must be guiding
students through the use of metacognitive skills needed for the problem at hand
and for future practice. This concept of metacognitive thinking skills provides
the key to the positive, active role of the tutor (Barrows, 1988, p. 3).
Tutors must be skilled in both the problem-based learning method as well
as in reasoning skills (Barrows, 1988). Other tutor tasks include the use of
questions to promote reasoning and critical thinking skills, facilitation of the
group processes without directing, and assuring that the groups derived
processes are externalized. Tutors should distinguish the role of a content
knowledge expert and the problem-based learning tutor role. Writers in the field
are divided on whether the tutor and the course instructor should be the same
person (Dolmans & others, 1993; Dolmans & Others, 1994, 1996; Gijselaers,
1994; Moust & Schmidt, 1995; Schmidt, van der Arend, Kokx, & Boon, 1995;
Schmidt & Moust, 1995; Schmidt & Others, 1993; Wilkerson, 1995; Wilkerson,
1996; Wilkerson & Hundert, 1997). However, students tutored by subject matter
experts are somewhat better achievers and tend to spend more time on self-
directed learning (Schmidt & Others, 1993). Problem-based learning requires the
learners to seek out the content knowledge as part of the learning experience
although they will call upon subject matter experts as a resource. Finally, a tutor
27


should stimulate reflection in the group on the newly acquired knowledge, the
relationship and integration of this new knowledge with previous knowledge, and
the potential for application (Barrows, 1988). The group reflection also seeks to
uncover new learning needs. Lastly, the tutor encourages the learners to assess
their problem solving skills, their processes as problem-based learners, and
discover areas for improvement (Barrows, 1994). Problem-based learning studies
indicate that this reflection component is necessary for successful learning
outcomes (Barrows, 1994, p. 74).
The Process
The problem-based learning process begins when an authentic problem is
presented to a small group of students. The group size is generally four to seven
students, however variations for larger groups have been reported (Rangachari,
1996; Woods, 1996). Once students are presented with the problem, they perform
an analysis to determine what they collectively know about the problem and what
they need to know to solve the problem. This phase requires extensive
collaboration and communication within the group. The groups efforts to define
their existing knowledge and the knowledge needed to solve the problem provide
experience with self-directed learning skills. Next students individually utilize
resources they discover for themselves and with the assistance of the problem-
based learning facilitation to acquire the knowledge and skills necessary to solve
28


the problem. The group members then reconvene to share their individually
acquired knowledge and continue the problem solving activity, again using
collaborative learning. This cycle of assessacquireshare repeats until a
satisfactory solution is achieved. A key element is a reflection activity that
concludes the problem-based learning process. This last stage, critically
necessary for learning, consists of self and peer evaluation of abilities as
problem-solvers, self-direction, and as members of the group (Barrows, 1994).
Benefits and Drawbacks of Problem-Based Learning
The most commonly cited benefit of problem-based learning is an
increased ability to apply the knowledge acquired using problem-based learning
(Albanese & Mitchell, 1993; Barrows, 1994; Hmelo et al., 1997). Also, the
development of reasoning skills in problem solving is coupled with the ability to
use knowledge in practice (Dolmans et al., 1997). Problem-based learning
students demonstrated a higher hypothesis-driven reasoning ability than data-
driven reasoning (Hmelo et al., 1997). This ability to work with an initially
limited set of data to formulate both problem and possible solutions represents
real-world situations better than is possible in traditional, lecture-based
instruction. With the hypothesis as a start, the learner acquires additional
knowledge that either supports or rejects the position. Support tends toward
problem solution while rejection forces reevaluation and the generation of a new
29


hypothesis. In contrast, data driven reasoning tends to become analysis
paralysis rather than problem solving. In this scenario, while knowledge may be
acquired, its relevance and applicability is frequently not realized.
Problem-based learning achieves its successes, in part because the
experience (a) requires active processing of information, (b) activates prior
knowledge, (c) provides a meaningful context, and (d) stimulates opportunities
for elaboration and organization of knowledge (Barrows, 1994).
There are drawbacks to the use of problem-based learning. Teaching with
problem-based learning requires a significant investment in designing problems
and implementing the tutoring process (Barrows, 1988; Stepien & Pyke, 1997).
For many instructors, traditional lecture formats may be both more comfortable
and less effort (Bligh, 2000). Another concern found throughout the literature is
that problem-based learning outcomes must not sacrifice students learning of the
subject matter. Research has shown that while problem-solving skills are better
when using problem-based learning methods, simple knowledge recall of facts
may be slightly less than compared with traditional methods (Vemon & Blake,
1993). However, comparisons are not always conclusive. Albanese and Mitchell
(1993) in a meta-analysis of the literature reported for six of the ten studies
[comparing outcomes], the overall basic science test scores of students in
conventional curricula were higher than those of students in problem-based
learning curricula (negative ES); however, only three of these scores were
30


statistically significant at the .05 level (p. 57). They concluded that, while the
expectation that problem-based learning students will not do as well as
conventional students on basic science tests appears to be generally true, it is not
always true (p. 57). In an independent meta-analysis, Vernon and Blake (1993)
reached similar conclusions. Some studies found significant differences favoring
traditional methods while others did not.
Relationships of PBL and SDL
While the many benefits of problem-based learning were discussed
previously, the development of self-directed learners deserves special
consideration. The problem-based learning literature claims the development of
self-directed learners as a benefit (Barrows, 1994; Dolmans, Schmidt, &
Gijselaers, 1995; Ryan, 1993; Taylor, 1986). However, these claims are not
supported by research but are either theorized or postulated by supporters of
problem-based learning. While the claims are not supported, neither are they
refuted. The topic has not been sufficiently investigated.
A careful analysis of the literature on problem-based learning and on self-
directed learning suggests definite relationships. Blumberg and Michael (1992)
state that problem-based learning ... has as a primary goal the students
development of self-directed learning skills (p. 3). However, research to
determine the attainment of that goal is sparse and has taken a very narrow
31


viewpoint. One focus has been the generation of learning issues as a measure of
self-directed learning (Dolmans et al., 1995). Other research investigated the
development of a few selected self-directed learning skills when a significantly
teacher-directed problem-based learning method was employed (Blumberg &
Michael, 1992). In the latter study, Blumberg and Michael showed that in a
partially teacher-directed problem-based learning situation, problem-based
learning students used the library and its resources more than traditional students,
self-reported more learning resource usage, and perceived a higher proficiency in
self-directed skills (Blumberg & Michael, 1992).
The more extreme, curricula-based, medical school format of problem-
based learning expects a high degree of learner self-direction. For other
implementations of problem-based learning, some self-directed skills are
incorporated in the problem-based learning activities and usually provide more
structure and scaffolding (Stepien, Senn, & Stepien, 2000; Wegner, Holloway, &
Crader, 1997). Clearly, some activities used in problem-based learning require a
set of skills also used in self-directed learning. This relationship suggests that
using problem-based teaching methods would give students the opportunity to
better develop these particular skills. Relationships between the
psychological/personal characteristics of self-directed learning and problem-
based learning are not obvious. The goal oriented motivational component of
self-directed learning appears to coincide with the goal directedness established
32


in problem-based learning. It is reasonable to infer that using problem-based
learning might affect students goal-oriented motivation. The problem-based
learning research literature minimally addresses the development of the skills
component of self-directed learning (Blumberg & Michael, 1992; Ryan, 1993).
Summary
The literature reviewed suggests that teaching with a problem-based
learning method may influence students self-directed learning. Four components
of self-directed learning were identified in the literature reviewed and an
integrated concept of self-directed learning was described. This literature also
suggested that grades of students experiencing problem-based learning is likely
to be minimally different from those taught with traditional lecture-based
methods. Finally, relationships between problem-based learning and self-directed
learning also suggest that teaching with problem-based learning methods is likely
to affect students self-directed learning.
33


CHAPTER 3
METHODOLOGY
This research compared problem-based learning instruction with traditional
methods on students self-directed learning, motivation, and grades. The three
research questions were:
1. Are there significant differences between students experiencing a problem-
based learning teaching method and students experiencing traditional lecture-
based teaching method on:
a. Students self-directed learning readiness?
b. Students self-directed learning skills?
c. Students self-directed learning performance?
d. Students course motivation?
e. Students programming assignment grades?
2. Are there significant differences across time on:
a. Students self-directed learning readiness?
b. Students self-directed learning skills?
c. Students self-directed learning performance?
d. Students course motivation?
e. Students programming assignment grades?
34


3. Is there a significant interaction between teaching method and time on:
a. Students self-directed learning readiness?
b. Students self-directed learning skills?
c. Students self-directed learning performance?
d. Students course motivation?
e. Students programming assignment grades?
This chapter describes the research methodology. The first section
discusses the design and rationale for selecting a quasi-experimental approach.
Then subjects, sampling procedure, setting and materials are described. Sections
on the independent variable and dependent variables provide operational
definitions. The sixth major heading, Data Collection Procedures, also includes
information on treatment and measurement. Data analysis procedures follow.
Design
A quasi-experimental design was used to compare problem-based learning
instruction with traditional methods on self-directed learning characteristics,
motivation, and grades. The treatment group had specific problem-based learning
modules taught with problem-based learning methods (the treatment) while the
traditional lecture-based teaching method was used for the control group. Since
35


students exercise their freedom of choice for a particular section of a course,
random assignment of subjects was not possible.
The design for self-directed learning readiness, self-directed learning skills,
motivation, and grades is illustrated in Figure 3.1. The design for self-directed
learning performance lacked a pre-treatment measurement since no data was
available prior to the experiment for this variable. That design is depicted in
Figure 3.2. Circles represent an observation or measurement time in both figures.
The Xs represent treatment periods for the treatment group and the
corresponding non-treatment periods for the control group. Time progresses from
left to right in the figure with the groups labeled on the left.
CD
c
c
o
CD
Treatment
Section
Traditional
Section
o X PBLi o XpBL2 O
o Xtrad O Xtrad O
1 2 3 4 5
Time
Figure 3.1. Quasi Experimental Design for SDL Readiness, Skills, Motivation,
and Grades
36


Each treatment consisted of a single problem-based learning exercise that
spanned two weeks of calendar time. Two sequential treatments were used at
Times 2 and 4 illustrated in the figure. Each section of the course met twice
weekly for one hour and fifty minutes. The control group met on Monday and
Wednesday evenings at 7 p.m. while the treatment group met on Tuesday and
Thursday evenings at 5 p.m. Pre-treatment measurements (Time 1) for both
groups were made prior to the first treatment. Mid-treatment measurements
(Time 3) were made after the first treatment and again at Time 5 after the final
treatment.
The control group was taught using the traditional lecture-based method
while the treatment group was being taught with problem-based learning. This
traditional method consisted of lectures on the same topics that were identified as
learning objectives used in creating the problem scenarios for the problem-based
teaching. Thus, during the treatment period, the learning objectives provided by
the course instructor were identical. Only the teaching method varied.
The same instructor taught both the control and experimental sections. The
instructor deliberately and carefully synchronized the topics for both sections
prior to and after the treatment period. Except for the problem-based learning
modules, the sections were taught using the same materials, assignments, and
exams. The problem-based learning instructional method used the same two
programming assignments adapted from those for the control section. (The
37


problem-based learning assignments are reproduced in A. INSTRUCTORS
GUIDE: PBL EXERCISE 1 and C. INSTRUCTORS GUIDE: PBL EXERCISE
2. The corresponding control section assignments are in B. NON-PBL
EXERCISE 1 and D. NON-PBL EXERCISE 2).
Self-directed learning performance data were not available prior to the
beginning of the experiment since time tracking was not required prior to
students work on programming assignment #6 (the first problem-based module
for the treatment group). Prior programming assignments differed in that their
minimal difficulty would not have provided meaningful time tracking data on
self-directed learning performance. The design shown in Figure 3.2 provided
measures for self-directed learning performance representing two periods
associated with the two programming assignments. No data were collected for
Time 1 shown in the design. Collected data are associated with work performed
during experiment Times 2 and 3 and collected upon completion of each
assignment. Since students completed the assignments at varying times, the
collected data only approximates collection at Times 3 and 5.
38


Treatment Section X PBLi o XpBLz O
Traditional Section Xtrad O Xtrad O
1 2 3 4 5
Time
Figure 3.2. Quasi Experimental Design for SDL Performance
Subjects
The experiment was conducted with two sections of undergraduate
Computer Science students enrolled in Computer Science 1 (CSI 1300) at
Metropolitan State College of Denver (MSCD). These students represent typical
Computer Science undergraduates needing to develop greater self-directed
learning abilities. MSCD is a four-year undergraduate institution offering
Bachelors degrees. The course is described as:
... the first course in the computer science core sequence. Students will
learn a modem programming language and the basic skills needed to
analyze problems and construct programs for their solutions. The emphasis
of the course is on the techniques of algorithm development, correctness
and programming style. Students are also introduced to the fundamentals of
software engineering and the software development life cycle
(Metropolitan State College of Denver, 2001).
39


Students enrolled in this course were predominately undergraduate
Computer Science or Mathematics majors. The sample sizes were limited to
those students from each group, control and treatment, agreeing to participate in
the study. Initial sample sizes consisted of 18 students (of 23 total students
enrolled in the section) in the control section and 19 students (of 20 total students
enrolled in the section) in the treatment group. Eight students in the control group
and eight in the treatment group completed the study. These same 16 students
were the only ones completing the course with a passing grade. The final sample
consisted of students who generally attended class during the study, completed
the study questionnaires, and completed the associated programming
assignments. Students were included in the final analysis only if all three
components were satisfied. Although partial data for this study were obtained on
12 students from each section (questionnaire scores for course motivation, self-
directed learning skills, and self-directed learning readiness), only eight in each
section completed the final programming assignment used for the post-treatment
grades in the study. Only data for these 16 students was used in the final
statistical analysis.
The treatment group consisted of five males and three females as compared
with seven males and one female in the control group. The treatment group was
more diverse than the control group having one Asian American, four Caucasian
40


and three Hispanic students while the control group had one Asian American and
seven Caucasian students.
Additional demographic data were collected to describe the sample
population. The samples mean age was 29.67. The concurrent number of
courses students were enrolled in was 2.93 while the number of previous
Computer Science courses was 1.87. The average total college credits of the
sample were 64.27. The weekly work hours ranged from zero to 55 with a mean
of 29.9 for the entire group. Students grade point averages were 3.25 for the
treatment group and 3.08 for the control group. Data for each group are shown in
Table 3.1.
Table 3.1. Demographic Data
Treatment Grou p Control Group
Mean Std. Dev. n Mean Std. Dev. n
Age 26.50 7.82 8 31.88 10.15 8
Num Classes 3.25 .89 8 2.63 1.06 8
CSI courses 1.25 1.16 8 2.38 1.41 8
Total Credits 62.43 40.11 8 65.88 44.30 8
Work HRS 36.07 15.92 8 24.50 17.94 8
GPA 3.25 .67 6 3.08 .89 4
Setting and Materials
The setting was a standard smart classroom on the college campus.
Smart classrooms are equipped with LCD projectors for displaying computer
41


output, document cameras for projecting text materials, and overhead projectors
for displaying transparencies. This particular classroom configuration was used
for both sections. The room has four front-facing rows of tables that each seat
approximately eight students. Each seat has power and Internet connections for
students with laptop computers. However, students seldom used laptops during
class. The instructor routinely used all three types of media, computer, projected
textbook pages, and overhead transparencies, during classes. Example source
code prepared by the instructor was also frequently provided to students in both
sections. These materials were learning resources complimenting the textbook.
The problem-based method used two programming assignments as the
problem focus for teaching. Specific content learning objectives defined by the
instructor for a segment of the course were incorporated in the design of the
problems for the problem-based learning treatment. See A. INSTRUCTORS
GUIDE: PBL EXERCISE 1 and C. INSTRUCTORS GUIDE: PBL EXERCISE
2. These materials also included information for using the problem-based
learning teaching method along with guides for facilitation/tutoring the process.
The problem-based treatment began with the sixth week of the semester. The
corresponding two programming assignments for the control section are in B.
NON-PBL EXERCISE 1 and D. NON-PBL EXERCISE 2. The teaching
technique for the control group was the standard lecture-based format previously
used for both sections of the course.
42


Independent Variable
In this study, the teaching method was the independent variable with two
levels. One level was the problem-based learning teaching technique while the
second level was the traditional lecture-based teaching method.
Problem-Based Learning Teaching Method
The central element of problem-based learning is a problem that is
carefully selected to meet specific learning objectives, including content
knowledge areas. Students in problem-based earning are expected to define their
learning objectives, with significant guidance of the tutor/facilitator, as they
explore the problem. This teaching method a) uses an authentic, ill-structured
problem as the focal point for study, b) follows a specific process (discussed
below) for investigation and inquiry, and c) is facilitated with an emphasis on the
processes for inquiry and learning rather than merely providing out of context
subject matter content. Furthermore, specific content knowledge related to the
problems learning objectives is not usually presented prior to the problem but is
discovered by the students while seeking solutions to the problem. Students
determine the knowledge needed to solve the problem, define and use resources
to develop solutions, and review their performance of both acquiring knowledge
and following the problem-solving process. This approach is contrasted with first
learning and then applying knowledge.
43


For this study, the course instructor was the primary subject matter expert
while the researcher was the primary problem-based learning tutor/facilitator for
the treatment class. The researcher and instructor collaborated to maximize the
quality of the in-class tutoring. Carefully scripted problem-based learning guides
were used to provide scaffolding and instruction on the problem-based learning
(see Problem Logs included with each problem-based learning module, A.
INSTRUCTORS GUIDE: PBL EXERCISE 1 and C. INSTRUCTORS GUIDE:
PBL EXERCISE 2).
Traditional Lecture-Based Learning Teaching Method
In the traditional lecture-based method, the instructor introduced topics
pertinent to the learning objectives defined for a particular module. The lecture
format was used, often accompanied with PowerPoint slides provided by the
textbook authors. Lectures were also based on source code examples used to
illustrate learning objectives. Any student questions related to the lecture were
answered when posed. Explicit programming assignments with the same due
dates as those for the problem-based learning group were given to the control
group students. Assignments were made on the first class of the same week for
each group. The control group assignments are reproduced in B. NON-PBL
EXERCISE 1 and D. NON-PBL EXERCISE 2). Figure 3.3 provides a calendar
of the events for the experiment.
44


Dependent Variables
There were three research questions addressing the teaching method, the
differences over time, and the interaction of method and time. For each question
five dependent variables were identified. The following discussion includes the
operational definition of each dependent variable as well as its measurement
method.
Self-Directed Learning Readiness
The Guglielmino Self-Directed Learning Readiness Scale (SDLRS)
(Guglielmino, 1977) was used to measure self-directed learning readiness. This
instrument includes 58 items using a 5-point Likert response scale. Factor
analysis of the SDLRS provides eight characteristics of self-directed learners
(Guglielmino, 1977). These are:
1. Openness to learning opportunities
2. Self-concept as an effective learner
3. Initiative and independence in learning
4. Informed acceptance of responsibility for ones own learning
5. Love of learning
6. Creativity
7. Positive orientation to the future, and
8. Ability to use basic study skills and problem-solving skills
45


Guglielmino asserted that these factors correlate favorably with the
definition of a highly self-directed learner as defined by the Delphi survey of the
experts. She found that the SDLRS could account for 76% of the variance in
effectiveness as a self-directed learner (Guglielmino, 1977, p. 73).
McCune (1988) found the SDLRS to be the most widely used instrument
for measuring self-direction in learning research. SDLRS scores have shown a
relatively high validity when used as a measure of readiness for self-directed
learning (Bonham, 1991; Finestone, 1984; Guglielmino, 1997; Long, 1987; Long
& Agyekum, 1983, 1984). Subsequent literature indicates its continued use
(Confessore & Confessore, 1992a). In addition to its widespread use, research
also supports its ability to indicate levels of self-directed readiness. Guglielmino
(1997) cited Borg & Gall (1989) and Mehrens & Lehmann (1984) as stating the
... expert judgment is commonly used to ascertain whether an instrument has
content validity (p. 213). The Delphi technique used by Guglielmino relied on
the experts on self-directed learning to provide specific topics upon which the
instrument was based.
Many studies (Finestone, 1984; Hall-Johnsen, 1985; Hassan, 1981; Jones,
1989) successfully correlated scores on the SDLRS with behaviors consistent
with concepts of self-directed learning. Hall-Johnsen (1985) found a positive,
predictive relationship between the number of self-planned projects conducted
46


and the time spent on these with SDLRS scores. She found that self-concept as
an effective, independent learner was identified as the readiness factor that best
predicted the number of self-planned projects (R = .20) and the time spent on
them (R = .42). She also reported that at least five individual items on the
SDLRS appear to be very effective (r = 1.00) in predicting extent of involvement
in self-planned projects. Studies conducted by Finestone (1984), Hassan (1981),
and Jones (1989) demonstrated validity by successfully correlating behaviors
such as initiative, acceptance of responsibility for learning, and a strong desire to
learn with SDLRS scores (the Pearson product-moment correlation was .48 (p
=.0179)).
Reliability studies of the SDLRS have reported high Cronbach alpha
estimates. Chronbach-alpha coefficient values of .87 (Guglielmino, 1977), .87
(Hall-Johnsen, 1985; Hassan, 1981), and .92 (Finestone, 1984; Skaggs, 1981)
support the reliability of the SDLRS. Another reliability estimate based on a
sample of 3,151 individuals from a wide variety of settings throughout the United
States and Canada had the highest reported Chronbach-alpha coefficient at .94
(Guglielmino, 1989).
Self-Directed Learning Skills
The Motivational Strategies for Learning Questionnaire (MSLQ), Part B
measured self-directed learning skills. Part B of the MSLQ defined learning
47


strategies as Cognitive & Metacognitive Strategies and Resource Management
Strategies. This is a self-report instrument consisting of 50 items that use a 7
point Likert scale ranging from 1 = not at all true of me to 7 = very true of
me. See G. MSLQ .
The category of Cognitive & Metacognitive Strategies consists of five sub
categories of a) rehearsal, b) elaboration, c) organization, d) critical thinking, and
e) metacognitive self-regulation. The resource management strategies consist of
a) time & study environment, b) effort regulation, c) peer learning, and d) help
seeking. These are elaborated in Table 3.2. MSLQ Part B Learning Strategies.
48


Table 3.2. MSLQ Part B Learning Strategies
Learning Strategy Description
Cognitive & Metacognitive Strategies
1. rehearsal Reciting or naming items to be learned influences the attention and coding process
2. elaboration Paraphrasing, summarizing, creating analogies, and generative note- taking helps the learner integrate and connect new information with prior knowledge
3. organization Selection of appropriate information and construct connections active, effortful endeavor resulting in close involvement with the task
4. critical thinking Application of previous knowledge to new situations to solve problems, reach decisions, or make critical evaluations
5. metacognitive self-regulation Awareness, knowledge, and control of cognition focus is on planning, monitoring, and regulating cognitive activities
Resource Management Strategies
1. time & study environment Management and regulation of time and study environment
2. effort regulation Regulation of effort related to learning goals and application of other learning strategies
3. peer learning Realization of benefits of collaboration for increased comprehension and development of new insights
4. help seeking Recognition of deficiencies and the implementation of strategies to define, seek, and utilize resources
Adapted from (Pintrich et al., 1991).
49


The skills frequently discussed throughout the literature on self-directed
learning are well represented by the items addressed by the learning strategies of
the MSLQ-PartB (Candy, 1991; Cheren, 1983; Grow, 1991a; Hrimech, 1995;
Knowles, 1975).
Unlike the SDLRS that was designed specifically to measure a construct of
self-directed learning, the MSLQ was not specifically created for self-directed
learning. The correspondence of self-directed learning skills with the learning
strategies items of the MSLQ Part B provides a reasonable argument for using
this instrument as a measure of the skills component for self-directed learning.
The instrument itself has been shown to ... represent a coherent conceptual and
empirically validated framework for assessing student motivation and use of
learning strategies in the college classroom (Pintrich, Smith, Garcia, &
Mckeachie, 1993, p. 810). Their internal consistency and reliability analyses
found a ... relatively good reliability in terms of internal consistency and two
confirmatory factor analyses indicated, The general theoretical framework and
the scales that measure it seem to be valid (p. 810).
Self-Directed Learning Performance
Self-directed learning performance is defined as the extent to which
students demonstrate the following performances in relation to the learning
project. Knowles (1975)defined these as:
50


Setting learning objectives,
Identifying deficits in ones own knowledge in relation to the
learning objectives,
Identifying resources to address the deficits,
Using resources for learning, and
Self-assessment of learning outcomes.
These activities were reported through students journaling using a
modified version of time and activity logging developed for software
programming course activities (Humphrey, 1997, pp. 21-9). Modifications that
specify the types of activities provide the ability to track time and effort on self-
directed learning performance related to the project. See H. PSP TIME Students
used Engineering Notebooks, provided to both groups specifically for this
study, to record their time on each task and activity. Tune logs were collected
weekly from each group. The same log forms were used to collect the
time/performance data for both sections of the course.
Students were asked to record all time, by specific activity, spent on each
of the two assignments during the experiment. The time entry forms included
detailed descriptions of each activity code to assist in logging the correct activity
and the time spent for that activity. The self-directed learning activities in the
above bulleted list each corresponded to an activity code on the time logs. These
51


data were collected weekly and entered into the Excel spreadsheet by activity
code. The total times and the total of all self-directed learning times were then
checked for correctness and imported into SPSS for analysis. The self-directed
learning time represented a single dependent variable.
Motivation
The Motivational Strategies for Learning Questionnaire (MSLQ), Part A
was used to measure course motivation. This part of the MSLQ has 31 self-
report, Likert type items in the same format as Part B. The three primary
constructs measured by this instrument are a) value, b) expectancy, and c)
affective elements of motivation. The value component consists of intrinsic goal
orientation, extrinsic goal orientation, and task value. Expectancy includes both
control of learning beliefs and self-efficacy for learning and performance.
Finally, the affective component is manifested as test anxiety. The internal
consistency is high for the motivational scales with reported coefficient alphas of
.90 for task value and .93 for self-efficacy for learning. Test anxiety and intrinsic
goal orientation values were .80 and .74 respectively while the extrinsic goal
orientation at .62 and control of learning beliefs of .68 showed more variability
(Pintrich et al., 1993). Since course motivation is an integral component of
students self-directed learning (discussed in the conceptual framework section of
52


Chapter 1), the MSLQ Part A was an ideally suited instrument for measuring
this construct.
Grades
Grades are operationally defined as the grade assigned by the instructor on
programming assignments for the related subject matter content. The grades used
were the programming assignment grades prior to the treatment groups first
assignment, the programming assignment grade associated with the first
problem-based learning experience (programming assignment #6), and the grade
for the assignment associated with the second experience (programming
assignment #7). The design of the problem-based modules was such that the
completed assignments output should be identical to that of the control groups
assignments output. This allowed the instructor to grade both groups programs
with the same criteria. All grades were reported on a scale from 0-100%.
Procedures
Data collection consisted of administering the SDLRS, MSLQ (Part A and
B), collecting individual student activity-time logs (journals), and obtaining
programming assignment grades from both groups. These data were collected
prior to commencement of problem-based learning for the treatment group,
between the first and second treatments, and after the second and final treatment.
53


The demographic data were collected concurrently with the pre-treatment
administration of the other instruments. Students in the treatment group and the
control group were subjected to the same measurement activities on
corresponding first class meetings of the measurement week (the control group
on Monday nights class and the treatment group on Tuesday nights class). The
actual calendar of events is shown below in Figure 3.3. The questionnaires were
administered during experiment weeks 1, 4, and 7 while time logs were collected
each week. The first in-class problem-based learning treatment took place during
weeks 1 and 2 with the introduction of programming assignment #6. The second
in-class treatment began with programming assignment #7 in week 4 and
extended through week 5. Students continued to work on the programming
assignments and experience problem-based learning through week 8.
54


Mon Tue Wed Thur Fri
9/24 Recruit Explain Time iiimim 9/26 H 1|§§§|
Week 1 (Semester Week 5) Handout Q1 Assign Proaram #6 iMEMU Collect Q1 ^ PBL Ex 1 ? SSxjRProgram #6%
10/1 10/3 iiiijllllllll
Week 2 Collect Time Logs #6-1
(Semester Week 6) illlillli
IA/aaIt A 10/8 Collect Time Logs #6-2 WfflfflM P§|IP!!! 10/10 EXAM #1 lliiillll

(Semester Week 7) EXAM #1 Prep iiiHiiiiiii! 111
Week 4 10/15 Handout Q2 Assign 10/17 Collect Q2 IlllSIill PI
(Semester Week 8) Program #7 Collect Time Logs #6-3 IgCtg PBL Ex 2 V g^gProgram #7j
10/22 lillilii 10/24 liiiiiiliililiiiii
Week 5
(Semester Week 9) Logs #7-1 Collect Time Logs #6-4 jiifeilliiPll IliiipiiPli iiiiiiliilfiiiii NS Program J SS #6 Due i OXwvwwvwvV
10/29 laiMmmt 10/31
Week 6 (Semester Week 10) Collect Time
Logs #7-2 Collect Time Logs #6-5 iiliii!!!
11/5 11/7 xxxxxxxxxx^xxxV Program #7
Handout Q3 iiiiiiiiiiiii Collect Q3 Miliii Due
Week 7 (Semester Week 11) Collect Time Logs #7-3
11/12 liMliiliil 11/14
Waalr A Collect Time llUlltiil! EXAM #2
(Semester Week 12) Logs #7-4 iiiiiiiiiiliiiiiiiiiiii
Control Section
Problem-B ased
Learning Section
Figure 3.3. Calendar of Events
55


Data were collected in paper format, transcribed into an Excel Workbook,
scored using spreadsheet computations, and checked for computational and
transcription errors. Upon completion of data collection, the Excel file was
imported for analysis into the software program, Statistical Package for the
Social Sciences (SPSS 10.05 for Windows, 1999).
Data Analysis Procedures
This study involves a single independent variable, the instructional method
of a problem-based learning method versus the traditional lecture-based teaching
method and multiple dependent variables. The dependent variables are assumed
to be interval level. An appropriate statistical technique is an analysis of variance
(ANOVA)(Cohen & Reese, 1994; Hair, Anderson, Tatham, & Black, 1998;
Hertzog & Rovine, 1985; Krzanowski, 2000). When several measurements are
taken over time from the same respondent, a repeated measures analysis is
needed. Therefore, the repeated measurement of the same students self-directed
learning readiness, skills, performance, course motivation, and grades mandates a
repeated measures ANOVA (Han et al., 1998; Hertzog, 1994; Krzanowski, 2000;
O'Brien & Kaiser, 1985; Tabachnick & Fidell, 2001).
56


Summary
The primary objective of this study was the determination of differences in
undergraduate Computer Science students self-directed learning after
experiencing problem-based learning versus traditional instructional methods. A
quasi-experimental design compared a treatment group and a control group with
measurements over time of the students self-directed traits and grades. For each
of the five dependent variables, the research questions were:
1. Are there significant differences between students experiencing a problem-
based learning teaching method and students experiencing traditional lecture-
based teaching method?
2. Are there significant differences across time?
3. Is there a significant interaction between teaching method and time?
The five dependent variables were: 1) students self-directed learning
readiness, 2) students self-directed learning skills, 3) students self-directed
learning performance, 4) students course motivation, and 5) students grades on
programming assignments.
57


CHAPTER 4
RESULTS
This study examined a problem-based learning teaching method as
compared to a traditional lecture based method to determine the effects on
undergraduate Computer Science students' self-directed learning, course
motivation, and programming assignment grades. The independent variable was
a problem-based learning teaching method versus the traditional lecture-based
method. Course motivation (MSLQ-A), self-directed learning skills (MSLQ-B),
self-directed learning readiness (SLDRS), self-directed learning performance
(SDL task time), and grades were the dependent variables. An alpha level of .05
was used for all statistical tests.
The organization of this chapter begins with a description of the sample
and the associated descriptive statistics. The next section discusses the primary
data analysis including the assumptions for a repeated measures ANOVA. The
Summary of Results section then answers each research question.
Data Analysis
The primary data analysis focused on the self-directed learning traits after
students experienced problem-based learning. The statistical significance of any
58


changes in the dependent variables over time both within the treatment group and
between the treatment group and control group was determined with a repeated
measures ANOVA. However, prior to this, data analysis was needed to ensure
the necessary criteria were met for a repeated measures ANOVA. Each of these
assumptions is addressed in the next section.
Assumptions for a Repeated Measures ANOVA
Independence. Analysis of variance assumes independent observations of
the dependent variable. The repeated observation of the dependent measures in
this design (pre-, mid-, and post-treatment of the experimental group) violates
this assumption of independence. The repeated measures analysis compensates
for the violation of this most important assumption of independence (Hair et al.,
1998, p. 347).
Independence between groups was not guaranteed. However, there is
evidence to suggest the observations between groups were independent. The
students in each group generally lacked the opportunity to confer with those of
the other group. The groups met on alternate nights, students had heavy work
schedules, and were generally taking two additional classes. One student
questioned; What is the other section doing? The response from a only one
student was the same thing. Later private discussion with the respondent
revealed some awareness but no conferring on substantive topics. The policy of
59


the instructor and researcher was to avoid, as much as possible, discussing one
section with the other. Overheard comments of students also left the impression
that most were not aware another section was involved in the study until after
mid-way through the experiment. Thus, it was assumed their was sufficient
independence between groups.
Equality of Variance-Covariance Matrices. This assumption calls for the
equality of the variance-covariance matrix (Girden, 1992; Hertzog & Rovine,
1985; O'Brien & Kaiser, 1985). Violation of this assumption increases the Type I
error in the main effects and interactions as well as results in a loss of power
(O'Brien & Kaiser, 1985, p. 317). The Levene test of the homogeneity of
variance for each dependent variable across all level combinations of the
between-subjects factors determined that the error variance of the dependent
variables was equal across groups with the only possible exceptions being the
pre-treatment self-directed learning skills and the pre-treatment grades. See I.:
Levene's Test of Equality of Error Variances. However, this problem is
inconsequential since "... a violation of this assumption has minimal impact if
the groups are of approximately equal size" (Hair et al., 1998, p. 348). The
groups in this study were equal in size.
Sphericity. In the repeated measures analysis, all variances of the repeated
measurements should be equal and all correlations between the pairs of repeated
measurements should also be equal. Violations of sphericity inflates the Type I
60


error rate. Mauchly's test for sphericity indicated that the assumption of
sphericity was met (see I. : Mauchly's Test of Sphericity).
Normality. Another assumption for a repeated measures ANOVA is that
the dependent measures are normally distributed. Tests of normality confirmed
that all dependent variable measurements appear to be normally distributed
except for the grades of the control group on programming assignment #6 and
assignment #7. The Shapiro-Wilk test (see I.: Tests of Normality) was an
appropriate tool because of the small sample size (SPSS, 1999)). Normal Q-Q
Plots of expected normal values versus observed values exhibit linear
correlations as expected for normally distributed data and corroborate the
Shapiro-Wilk test. Because MANOVA is relatively robust to violations of
normality (Hair et al., 1998; O'Brien & Kaiser, 1985), the non-normal
programming assignment grades is less problematic.
Thus the criteria necessary for a repeated measures ANOVA were either
met or determined to have little adverse impact.
Demographic Data
Demographic data for the treatment and control groups are summarized in
Table 4.1. There were no statistically significant differences between the control
group and the experimental group on any of the demographic characteristics.
61


Table 4.2 shows the results of the independent samples t-Tests for the metric
demographic data.
Table 4.1. Demographic Data
Treatment Grou p Control Group
Mean Std. Dev. n Mean Std. Dev. n
Age 26.50 7.82 8 31.88 10.15 8
Num Classes 3.25 .89 8 2.63 1.06 8
CSI courses 1.25 1.16 8 2.38 1.41 8
Total Credits 62.43 40.11 7 65.88 44.30 8
Work HRS 36.07 15.92 8 24.50 17.94 8
GPA 3.25 .67 6 3.08 .89 4
Table 4.2. t-Test of Demographic Data
t-test for Equality of Means
t df Sig. (2-tailed) Mean Difference Std. Error Difference 95% Confidence Interval of the Difference
Lower Upper
Age 1.19 14 .255 5.38 4.53 -4.34 15.09
Num Classes -1.28 14 .222 -.63 .49 -1.67 .42
CSI courses 1.74 14 .104 1.13 .65 -.26 2.51
Total Credits .157 13 .878 3.45 21.95 -43.98 50.87
Work HRS -1.36 14 .194 -11.56 8.48 -29.751 6.626
GPA -.36 8 .729 -.176 .49 -1.306 .955
62


Results of Major Analysis
The effects of the teaching method, time, and the interaction of method and
time were determined for each dependent variable. The following descriptive
statistics tables for each dependent variable present the means, standard
deviation, and sample sizes by teaching method for each of the three
measurement times. These tables also include summary statistics by teaching
method and times. The ANOVA tables for each dependent variable are presented
immediately after the associated descriptive statistics table. Statistically
significant results are briefly noted and include plots of estimated marginal
means over time for each teaching method.
63


Table 4.3. Descriptive Statistics for Self-Directed Learning Readiness
TIME
1 2 3
224.12 222.38 223.50 224.12
Traditional (31.98) (30.33) (25.63) (31.98)
.5 o Xi Xi n = 8 n= 8 n = 8 n = 24
O H ^ 235.63 232.38 233.38 235.63
PBL (31.28) n = 8 (29.15) n = 8 (31.63) n = 8 (31.28) n = 24
229.88 227.38 228.44
(31.13) n = 16 (29.20) n= 16 (28.27) n= 16
Table 4.4. ANOVA Table for Self-Directed Learning Readiness
SV SS df MS F p-value
Method (M) 1312.521 1 1312.521 .505 .489
s: M 36351.958 14 2596.568
Time (T) 50.375 2 25.187 .428 .656
MxT 6.542 2 3.271 .056 .946
s: M x T 1646.417 28 58.80
64


Table 4.5. Descriptive Statistics for Self-Directed Learning Skills
TIME
1 2 3
208.75 210.5 223.12 214.13
cf ^3 .55 O X Xi Traditional (35.65) n= 8 (47.22) n = 8 (27.81) n= 8 (33.91) n = 24
cd QJ 5 H ^ 222.00 214.75 216.88 217.88
PBL (13.21) n = 8 (30.74) n = 8 (31.81) n = 8 (22.80) n = 24
215.38 212.63 220.00
(26.86) n = 16 (38.55) n = 16 (29.05) n= 16
Table 4.6. ANOVA Table for Self-Directed Learning Skills
SV SS df MS F p-value
Method (M) 168.750 1 168.750 .067 .799
s: M 35064.583 14 2504.613
Time (T) 444.500 2 222.250 .637 .537
MxT 762.000 2 381.000 1.091 .350
s: M x T 9774.167 28 349.077
65


Table 4.7. Descriptive Statistics for Self-Directed Learning Performance
TIME
1 2 3
No Data 102.17 72.50 87.33
00 .a o X! Traditional (52.21) n= 6 (70.99) n = 6 (56.43) n= 12
§ JS PBL (46.44) n = 6 (83.89) n = 6 (42.82) n = 12
No Data 151.08 107.50
(69.50) n= 12 (82.62) n= 12
Table 4.8. ANOVA Table for Self-Directed Learning Performance
SV SS df MS F p-value
Method (M) 42252.042 1 42252.042 8.420 .016
s: M 50179.417 10 5017.942
Time (T) 11397.042 1 11397.042 3.292 .100
MxT 1162.042 1 1162.042 .336 .575
s: M x T 34616.417 10 3461.642
The effect of method on performance is significant. The effect of teaching
method was significant with the problem-based learning group mean time of
171.25 as compared to the traditional groups 87.33 minutes (see Table 4.7).
However, the effect of time and the method x time interaction were not
significant. The possible explanations are discussed in chapter 5.
66


Table 4.9. Descriptive Statistics for Motivation
1 TIME 2 3
Teaching Method Traditional 167.75 (23.73) n= 8 167.75 (24.38) n = 8 169.62 (21.87) n= 8 168.38 (21.87) n = 24
PBL 165.38 (17.25) n = 8 159.50 (19.40) n = 8 156.88 (15.88) n = 8 160.58 (15.20) n = 24
166.56 (20.08) n = 16 163.63 (21.70) n= 16 163.25 (18.07) n = 16
Table 4.10. ANOVA Table for Motivation
SV SS df MS F p-value
Method (M) 728.521 1 728.521 .685 .422
s: M 14893.458 14 1063.818
Time (T) 105.292 2 52.646 .679 .515
MxT 216.542 2 108.271 1.396 .264
s: MxT 2172.167 28 77.577
67


Table 4.11. Descriptive Statistics for Grades
TIME
1 2 3
98.000 84.69 77.97 86.89
C2 Traditional (1.60) (12.04) (11.08) (7.05)
.5 o J3 si n = 8 n = 8 n = 8 n = 24
PBL (4.89) (15.32) (18.24) (11.92)
n = 8 n= 8 n= 8 n = 24
96.13 72.97 70.55
(4.01) (17.99) (16.47)
n = 16 n= 16 n= 16
Table 4.12. ANOVA Table for Grades
SY SS df MS F p-value
Method (M) 2355.501 1 2355.501 8.187 .013
s: M 4027.914 14 287.708
Time (T) 6380.362 2 3190.181 44.585 <.001
MxT 779.362 2 389.681 5.446 .010
s: M x T 2003.484 28 71.553
The effect of teaching method, time, and the interaction of method x time
on grades was significant. The means for grades by teaching method were 86.89
for the traditional and 72.88 for the problem-based group. The means over time
were 96.13, 72.97, and 70.55 for times 1, 2, and 3 respectively (see Table 4.11).
The significant interaction is illustrated in Figure 4.1. Grades of both groups
dropped dramatically between the first assignment (Time 1) and subsequent
68


assignments (Times 2 and 3). The drop is much greater for the problem-based
learning teaching method group. Possible explanations are discussed in chapter 5.
GRADES
TIME
Figure 4.1. Programming Assignment Grades
Summary of Results by Research Question
This section presents the results of the statistical analyses organized by
individual research question. The repeated measures analysis of variance
69


revealed no statistically significant differences for any of the dependent variables
with the exception of grades.
Research Question 1
The first question asked whether there are significant differences between
students experiencing a problem-based learning teaching method and students
experiencing traditional lecture-based teaching method for a) self-directed
learning readiness, b) self-directed learning skills, c) self-directed learning
performance, d) students course motivation, and e) programming assignment
grades.
Self-Directed Learning Readiness. There were no statistically significant
differences in self-directed learning readiness scores regardless of the teaching
method. The F ratio for the main effect was F(l,14) = .505,p =.489. (Table 4.4,
page 64).
Self-Directed Learning Skills. There were no statistically significant
differences in self-directed learning skills scores regardless of the teaching
method. The F ratio for the main effect was F(l,14) = .067,p = 799. (Table 4.6,
page 65).
Self-Directed Learning Performance. The effect of teaching method was
statistically significant for self-directed learning performance. The F ratio for the
main effect was F(l,10) = 8.42, p =.016. (Table 4.8, page 66).
70


Students Course Motivation. Differences in students course motivation
were not statistically significant regardless of the teaching method. The F ratio
for the main effect was F(l,14) = .685, p = 422. (Table 4.10, page 67).
Programming Assignment Grades. Differences in grades were statistically
significant. The F ratio for the main effect was F(l,14) = 8.187,/? =.013. The
mean of the lecture-based groups grades was higher. (Table 4.12, page 68).
Research Question 2
The second question asked whether there are significant differences over
the three time periods for a) self-directed learning readiness, b) self-directed
learning skills, c) self-directed learning performance, d) students course
motivation, and e) programming assignment grades.
Self-Directed Learning Readiness. There were no statistically significant
differences in self-directed learning readiness scores over the three time periods
(pre-, mid-, or post-treatment). The F ratio for the time effect was F(2,28) = .428,
p =.656. (Table 4.4, page 64).
Self-Directed Learning Skills. There were no statistically significant
differences in self-directed learning skills scores over the three time periods (pre-
, mid-, or post-treatment). The F ratio for die time effect was F(2) = .637, p
=.537. (Table 4.6, page 65).
71


Self-Directed Learning Performance. There were no statistically significant
differences in self-directed learning performance scores over the two times
(corresponding to assignments #6, T2 and assignments #7, T3). No data were
available for self-directed learning performance prior to the beginning of the first
treatment period. The F ratio for the time effect was F(l,10) = 3.292, p =.100.
(Table 4.8, page 66).
Students Course Motivation. Differences in students course motivation
were not statistically significant over the three time periods (pre-, mid-, or post-
treatment). The F ratio for the time effect was F(2,28) = .679, p =.515. (Table
4.10, page 67).
Programming Assignment Grades. Grades were statistically different over
time within-groups. The F ratio for the time effect was F(2,28) = 44.585, p <
.001. (Table 4.12, page 68). The grades of both groups declined significant from
the pre-treatment time to mid- and post-treatment times.
Research Question 3
The third question asked whether the interaction of time and teaching
method would be have a significant effect on a) self-directed learning readiness,
b) self-directed learning skills, c) self-directed learning performance, d) students
course motivation, and e) programming assignment grades.
72


Self-Directed Learning Readiness. The self-directed learning readiness
scores showed no statistically significant differences for the method x time
interaction. The F ratio for the method x time interaction was F(2,28) = .056, p
=.946. (Table 4.4, page 64).
Self-Directed Learning Skills. There were no statistically significant
differences in self-directed learning skills scores for the method x time
interaction. The F ratio for the method x time interaction was F(2,28) = 1.091,/?
=.350. (Table 4.6, page 65).
Self-Directed Learning Performance. There were no statistically significant
differences in self-directed learning performance scores for the method x time
interaction. The F ratio for the method x time effect was F(l,10) = .336, p =.575.
(Table 4.8, page 66).
Students Course Motivation. Differences in students course motivation
were not statistically significant for the time*method interaction. The F ratio for
the method x time effect was F(2,28) = 1.396,/? =.264. (Table 4.10, page 67).
Programming Assignment Grades. There was a statistically significant
method x time interaction. The F ratio for the time effect was F(2,28) = 5.446,/?
= 010. (Table 4.12, page 68).
73


Summary
In summary, the effects of teaching method, time, and method x time
interaction was not statistically significant on students self-directed learning
readiness, self-directed learning skills, or course motivation occurred. These self-
directed learning components did not differ for the students experiencing a
problem-based learning teaching method nor was there any differences for the
traditional lecture-based method.
The effect of teaching method on self-directed learning performance was
statistically significant with the problem-based method groups performance
scores greater than those of the traditional lecture-based groups. However, the
effect of time and the method x time interaction were not statistically significant
for self-directed learning performance.
Programming assignment grades appeared to differ significantly with
teaching method, over time, and with the method x time interaction. The
traditional lecture-based method group consistently demonstrated statistically
higher grades than the problem-based learning method group.
74


CHAPTER 5
DISCUSSION
A problem-based learning teaching method was compared with a
traditional lecture-based teaching method to determine the effects on
undergraduate Computer Science students self-directed learning and
programming assignment grades. An integrated construct of self-directed
learning included a) self-directed learning readiness b) self-directed learning
skills, c) self-directed learning performance, and d) students course motivation.
Quasi-experimental designs were used to compare a problem-based
teaching method and a traditional lecture-based method in two sections of a CS1
course taught by the same instructor. Each of the self-directed learning
components and grades were measured for students experiencing traditional
instructional methods and problem-based learning methods. Readiness was
measured with the Self-Directed Learning Readiness Scale, skills with the
Motivated Strategies for Learning Questionnaire-Part B, performance with time
spent on self-directed learning tasks, and course motivation with the Motivated
Strategies for Learning Questionnaire-Part A. The grade measurement was the
course instructors percentage score given to students programming
assignments.
75


The results, described in Chapter 4, revealed significant differences in
performance but showed no significant differences for either group in students
self-directed learning readiness, self-directed learning skills, or course
motivation. The effect of teaching method was statistically significant on
problem-based learning performance with the treatment group spending more
time on self-directed learning tasks. The effects of teaching method, time, and the
interaction of method x time were statistically significant on grades. All grades of
the group taught with problem-based learning methods were lower than those of
the group taught with traditional lecture-based methods. However, the
programming assignment grades of both groups significantly declined over time.
The remainder of this chapter discusses these findings, provides possible
explanations for the lack of significant differencess in self-directed learning
traits, examines the problem-based learning treatment, addresses limitations of
the study, and offers topics for further investigation.
Self-Directed Learning Components
The conceptual framework proposed an integrated self-directed learning
construct composed of self-directed learning readiness, self-directed learning
skills, self-directed learning performance, and students course motivation. For
each of these dependent variables, the three research questions asked: a) Are
there significant differences between students experiencing a problem-based
76


learning teaching method and students experiencing traditional lecture-based
teaching method? b) Are there significant differences across time? And c) Is
there a significant interaction between teaching method and time?
Students in the both groups exhibited no significant differences in the
readiness, skills, or motivation. One possible explanation that applies for each
dependent variable is the low statistical power resulting from extremely small
sample sizes (eight in each group). Only large effects are likely to be observed
with these sample sizes.
Self-Directed Learning Readiness
The lack of an observed difference in students scores for self-directed
learning readiness could be because a) the problem-based learning had no effect,
b) the effect size was too small to observe given the small sample size, c) the
level, quality, or duration of the problem-based learning treatment was
insufficient to have an observable effect or d) the already relatively high self-
directed learning readiness left little room for increase.
The overall self-directed learning readiness scores (mean of approximately
229) were considered above average and just below high. SDLRS scores are
categorized as Low (58-188), Below Average (189-203), Average (204-218),
Above Average (219-232), and High (233-290) with an overall population mean
of 214 (Jones, 1989).
77


Table 4.3 (page 64) shows that the problem-based learning groups
readiness scores were consistently higher over time although the difference is not
statistically significant (see Table 4.4, page 64). The problem-based group would
be categorized as high in self-directed learning while the traditional lecture-
based group remained in the above average range. Both groups are certainly
above average, although not statistically different from each other.
There are several possible explanations for no significant differences in
SDLRS scores. In a study using learning contracts as a tool to teach self-
direction, Caffarella and Caffarella (1986) found no differences in SDLRS scores
of students measured at the start and end of the course. They did find some,
although limited, impact on self-directed learning. Two of their conclusions may
be pertinent to the findings in this study. The SDLRS measures attitudes towards
self-directedness rather than specific abilities so differences in competencies may
not be reflected in differences in attitudes. The ceiling effect may also be a factor
in the lack of differences. Caffarella and Caffarella (1986) argued that high initial
scores on the SDLRS leave little room for significant increases. Although the
scores of the CS1 undergraduates were in the mid 220s to mid 230s as
compared to the graduate students scores at 240 in the Caffarella study, the
ceiling effect may have played some role here too.
Examination of the scores in Table 4.3 (page 64) reveals that students were
very stable in their attitudes toward self-directed learning readiness. The
78


students report of their perceptions of self-directed learning readiness did not
change.
Self-Directed Learning Skills
The failure to find a difference in self-directed learning skills can be
attributed to most of the same factors as those for self-directed learning
readiness: a) the problem-based learning had no effect, b) the effect size was too
small to observe given the small sample size or c) the level, quality, or duration
of the problem-based learning treatment was insufficient to have an observable
effect. Table 4.5 (page 65) shows that the problem-based learning groups skills
scores were initially higher but declined and remained essentially flat over time
although the difference is not statistically significant (see Table 4.6, page 65).
However, the traditional groups scores revealed an increase for the last
measurement. Again, these differences were not statistically significant.
The possibility remains that the MSLQ-B does not measure the exact skill
set required for self-directed learning. In Table 3.2 (page 25), the skill sets for the
MSLQ-B include many skills important to self-directed learning but these may
not represent all the specific skills required. The assessed skills may be necessary
but not sufficient to fully describe self-directed learning skills. Although beyond
the scope of this study, correlations of subcategories from the instrument with
self-directed learning might prove useful. Future research is needed to investigate
79


specific measures of self-directed learning skills both as perceived by the student
and as demonstrated.
Self-Directed Learning Performance
The analyses for self-directed learning performance showed that the effect
of teaching method was significance at the .05 level. The F ratio for the teaching
method effect was F(l,10) = 8.420 p =.016 (see Table 4.8, page 66). However,
the effect for time was not significant (F ratio for the time effect was F(l,10) =
3.292, p =.100). The method x time interaction was also not significant with an F
ratio of F(l,10) = .336, p =.575. (Table 4.8, page 66). The following observations
are made with the recognition that the sample size was small. No data were
available on self-directed learning performance time prior to the beginning of the
experiment so only time on assignments #6 and #7 were available. The prior
programming assignments were not sufficiently complex that time tracking by
the activity codes would have been meaningful. Description of these pre-
treatment assignments is more fully discussed under Grades, page 82.
Table 4.7, page 66, shows the significantly higher performance times for
the problem-based learning group. The use of the problem-based learning
teaching method required students to spend more time on the self-directed
learning activities. Without the guidance and structure of the problem-based
learning method, the lecture-based group reported less time thinking about their
80


learning needs, how to address them, and reflecting on their own learning than
did the problem-based group.
The declines in self-directed learning performance over time shown in
Table 4.7 maybe explained by the need for less time to complete the second
assignment. The second assignment allowed some reuse of skills and knowledge
necessary for the first assignment. Both groups required less total time to
complete the second assignment (only 55% of the time needed for the first
assignment for the treatment group and 68% for the control group).
There were concerns about the quality of the reported time data. Students
were asked to keep time logs for all their activities associated with each
programming assignment. All time spent on each assignment should have been
designated with an activity code designed to identify self-directed learning
performance. Initially, students diligently recorded their time and activity codes.
However, students reported difficulty in accurately partitioning time into
appropriate activity codes. Some students may have given up accurately
reporting correct activity codes.
The quality of the time reporting data and inadequate sample size reduce
the confidence in these findings. Further discussion of the performance
component is found in recommendations for future research.
81


Course Motivation
Problem-based learning and traditional teaching methods were not shown
to have a significant effect on the motivation component. The problem-based
learning group experienced an insignificant decline in motivation scores over
time (shown in Table 4.9, page 67). The use of authentic problems in the
problem-based learning method did not appear to impact student motivation. The
students experience of a new, unfamiliar teaching method may account for the
lack of effect. Although beyond the scope of this study, an examination of the
individual motivational components of the MSLQ-A might offer greater insight.
A final consideration is that the direct connection between motivation
measured by the MSLQ-A and motivation for self-directed learning may be too
amorphous to yield meaningful results. However, since differences in motivation
scores were not statistically significant in this study, further discussion is not
warranted.
Grades
The statistical analysis showed that the effect of teaching method, time, and
the interaction of method x time on grades was significant. The problem-based
method group consistently earned lower grades over time for the programming
assignments than the traditional method group. The obvious conclusion is that a
82


traditional lecture-based teaching method yields better grades. However,
additional investigation revealed other factors that likely influenced grades.
Although analysis of the demographic data showed no statistical differences
between the groups, some of these factors may have contributed to lower grades
for the treatment group.
Both groups pre-treatment grades were extremely high compared to their mid-
and post-treatment scores. These declines and the differences between the groups
are dramatically illustrated in Figure 5.1 below.
GRADES
TIME
Figure 5.1. Programming Assignment Grades
83


The average grades on the pre-treatment programming assignments were
98% for the control group and 94.25% for the treatment group. These
anomalously high pre-treatment grades can be accounted for by two factors.
First, the pre-experiment assignments lacked the higher level of difficulty found
in assignments #6 and #7. The second factor is that the grading scheme prior to
the experiment was more lenient than that for the experiment assignments.
The first assignment score consisted of an aggregation of short assignments
requiring the students to type and run programs. Students were given paper
copies of simple programs and code components of programs from which they
created their own program. These tasks were primarily a test of their ability to
configure their programming environment rather than define, design, and
implement a program. The assignments for programs #6 and #7 were
significantly more difficult requiring students to define, design, and implement a
solution to the problem on their own.
The grading scheme varied between the first assignment and subsequent
assignments. Students were allowed to submit multiple times with instructor
feedback each time for the first assignment before the final grade was assigned.
However, the grades for assignments #6 and #7 were based on a single
submission of the students program without prior instructor feedback.
The change in assignment difficulty and the more stringent grading of the
last two assignments explains the lower grades as compared to the first
84


I
assignment. Since both groups experienced the drop, it is difficult to attribute the
difference over time primarily to teaching method.
The treatment groups consistently lower grades may also have been
influenced by factors other than, or in addition to, teaching method. Compared to
the traditional method group, students in the problem-based group had completed
fewer computer science courses, worked more hours outside of school each
week, carried a heavier concurrent course load, and were younger. Table 5.1
compares these factors for the two groups. While none of these differences were
statistically significant, the influence on programming ability could contribute to
differences in grades.
Table 5.1. Selected Demographic Data
PBL Group
Mean
Previous CS Courses 1.29
Work HRS peer Week 36.07
Number Concurrent Classes 3.29
Age 27.14
Traditional Group
Mean
2.38
24.50
2.63
31.88
The actual experience level of programming expertise for the control group
seems to have been greater with an average of one more course than the
treatment group. The control group also spent 11.5 hours less working each week
and was taking less other courses than the treatment group.
85


Problem-Based Learning Teaching Method
Since the research was directed toward comparing problem-based learning
instruction versus traditional lecture-based instruction on students self-directed
learning, the problem-based method is detailed here. The problem-based learning
treatment involved two problem scenarios associated with programming
assignments (See A. INSTRUCTORS GUIDE: PBL EXERCISE 1 and C.
INSTRUCTORS GUIDE: PBL EXERCISE 2). Each problem was presented,
investigated, and studied over two weeks class time (4- two hour class periods)
using the problem-based learning teaching method. The actual calendar time
included an additional week between the two problem-based learning
experiences during which the instructor gave a review session class and the first
exam of the semester. Instructor imposed deadlines for the programming
assignments were an additional two weeks beyond the completion of the
problem-based learning experience. The actual sequence of events is shown in
Figure 3.3. Calendar of Events (page 55).
During the problem-based learning treatment of the experimental group,
the instructor served as a subject matter expert and co-tutor while the researcher
functioned as the primary problem-based learning tutor. The Problem Logs 1-9
86


of the instructors guides for PBL were used to ensure the problem-based
learning experience was administered consistently and correctly (See A.
INSTRUCTORS GUIDE: PBL EXERCISE 1 and C. INSTRUCTORS GUIDE:
PBL EXERCISE 2 for Problem Logs 1 9). The problem-based learning method
included the characteristics described in Table 5.2.
Many of the process steps required students to individually complete the
activities begun in class (especially sequence steps 2, 3, and 4). The continuation
of the process outside of class and the engagement of students in-class were
problematic. Deficits in the overall quality of the problem-based learning
treatment received by the students warrant further discussion.
87


Table 5.2. Problem-Based Learning Characteristics
Characteristic Description
Problem Ill-defined & complex
Learning Issues Teacher defined through selection of problem though not explicitly revealed to students; Student- defined with facilitation toward problem solution
T utoring/F acilitation Tutor (the researcher) facilitated the problem-based learning process; Instructor as a subject matter expert resource and secondary tutor; Scripted guides for the instructor/tutor and guided exercises for the students
Group size Average size of 4-5 for problem clarification and definition of learning issues; process tutoring and plenary sessions in both groups and as a whole class
Process sequence 1. Problem presentation 2. Groups refine problem aspects and define needed learning issues (in-class) with facilitation (what they know, what they need to know) 3. Groups define resources (in-class) with facilitation 4. Individuals use resources for learning 5. Individuals share knowledge in groups (in- class) 6. Groups summarize results for learning issues (in-class) with facilitation 7. Individuals complete implementation of problem solution
Problem duration Approximately 2 weeks (calendar time)
Quality of Problem-Based Learning Treatment
Although the teaching method was rigorously followed, student reception,
responsiveness, and participation varied. Observations during the experiment
88