Citation
Designing effective instructional models for increasing student achievement

Material Information

Title:
Designing effective instructional models for increasing student achievement
Creator:
Balasubramanian, Nathan
Publication Date:
Language:
English
Physical Description:
x, 78 leaves : ; 28 cm

Thesis/Dissertation Information

Degree:
Doctorate ( Doctor of philosophy)
Degree Grantor:
University of Colorado Denver
Degree Divisions:
School of Education and Human Development, CU Denver
Degree Disciplines:
Educational leadership and innovation

Subjects

Subjects / Keywords:
Instructional systems -- Design ( lcsh )
Curriculum planning ( lcsh )
Academic achievement ( lcsh )
Academic achievement ( fast )
Curriculum planning ( fast )
Instructional systems -- Design ( fast )
Genre:
bibliography ( marcgt )
theses ( marcgt )
non-fiction ( marcgt )

Notes

Bibliography:
Includes bibliographical references (leaves 68-78).
General Note:
School of Education and Human Development
Statement of Responsibility:
by Nathan Balasubramanian.

Record Information

Source Institution:
|University of Colorado Denver
Holding Location:
Auraria Library
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
227206238 ( OCLC )
ocn227206238
Classification:
LD1193.E3 2007d B34 ( lcc )

Full Text
DESIGNING EFFECTIVE INSTRUCTIONAL MODELS FOR
INCREASING STUDENT ACHIEVEMENT
by
Nathan Balasubramanian
B.Sc., University of Madras, India, 1986
M.Sc., University of Madras, India, 1988
B.Ed., Annamalai University, India, 1992
M.Ed., Sheffield University, United Kingdom, 2003
A thesis submitted to the
University of Colorado at Denver and Health Sciences Center
in partial fulfillment
of the requirements for the degree of
Doctor of Philosophy
Educational Leadership and Innovation


2007 by Nathan Balasubramanian
All rights reserved.


This thesis for the Doctor of Philosophy
degree by
Nathan Balasubramanian
has been approved
Brent G. Wilson
\
R. Sukumar
October 22. 2007
Date


Balasubramanian, Nathan, Doctor of Philosophy (Educational Leadership and Innovation)
Designing Effective Instructional Models For Increasing Student Achievement
Thesis directed by Professor Brent G. Wilson
ABSTRACT
The challenge of improving the performance of students with diverse needs and abilities
has concerned teachers throughout the history of modern education. However, not until the
accountability measures instituted by the No Child Left Behind (NCLB) Act of 2001 -
requiring disaggregating the results of all subgroups of learners, by ethnicity, socio-
economic status, pupil services, and English language proficiency has this challenge of
reaching out to every student that needs attention been brought to the publics focus. This
progressive facet of the law has been a positive driving force in Balasubramanian's
research agenda. As scholar-teacher, this portfolio dissertation describes
Balasubramanian's ongoing efforts to consistently increase student learning and
achievement as he continues to work in high-needs secondary schools schools with
large populations of students from low-income, migrant, and international families by
focusing on student motivation, engagement, and cognition. All five studies described here
have shown significant normalized gains. These gains demonstrate the increase in
standards-based content knowledge of learners across all levels due to specific
instructional interventions. The effect sizes of the observed means across all the studies
were high.
This abstract accurately represents the content of the candidates thesis. I recommend its
publication.
Signed
Brent G. Wilson


DEDICATION PAGE
This dissertation is dedicated to my parents Radhakrishnan Balasubramanian and
Visalakshi Balasubramanian, my in-laws Maniam Varagoor and Vasantha Maniam for their
unconditional support, love, and prayers, and my students who inspire me every day with
questions from their inquisitive young minds.


ACKNOWLEDGMENT
Although I am the primary author in ail five publications (chapters) in this portfolio
dissertation, I acknowledge with deepest gratitude my co-author-coliaborators and
reviewers especially my Advisor, Prof. Brent G. Wilson, mentor Prof. Rodney Muth,
father-in-law, Mr. Maniam Varagoor, and wife Dr. Gita Varagoor who patiently read the
manuscripts, word for word, and made magical edits to make sure was aiways
communicating my ideas clearly. I also wish to sincerely thank the other two members of
my committee, Prof. Krzysztof Cios and Dr. R. Sukumar for their valuable contributions to
my research. In addition, I thank my brother-in-law Dr. Sriram Somasundaram who
provided valuable advice at critical junctures throughout the dissertation process. Finally,
without the critique of the IDEAL (Innovative Designs of Environments for Adult Learning)
Lab members, the instructional models in this dissertation wouid not have been so well
refined.
Chapter 2. Learning by Design: Teachers & Students as Co-Creators of Knowledge.
Reprinted with permission from Dr. Kari Kumpulainen, University of Oulu, Finland. [Co-
author: Brent G. Wilson]
Chapter 3. Innovative Methods of Teaching Science and Engineering in Secondary
Schools. Reprinted with permission from Prof. Freddy Malpica and the Journal of
Systemics, Cybernetics and Informatics. [Co-authors: Brent G. Wilson and Krzysztof J.
Cios]
Chapter 4. Increasing Student Achievement through Meaningful, Authentic Assessment.
Reprinted with permission from the Association for the Advancement of Computing in
Education www.aace.org [Ms. Isobel Stevenson, Principal, Angevine Middle School,
reviewed the manuscript and provided valuable feedback]
Chapter 5. Games and Simulations. Reprinted with permission from the Association for the
Advancement of Computing in Education www.aace.org [Co-author: Brent G. Wilson]
Chapter 6. Nurturing Teacher Excellence Using the Learn By Design Model (LBDM). Draft
of article currently under review for publication in Principal Leadership. [Co-authors: Jana
L. Frieler and Dr. Elliott Asp]


TABLE OF CONTENTS
Figures.........................................................x
Tables.........................................................xi
CHAPTER
1. INTRODUCTION.......................................................1
Journey of Happy Accidents......................................2
Theme and Cohering Line of Inquiry..............................8
Conceptual Model................................................8
Summary of Chapters............................................10
Results form my Empirical Studies..............................11
Reflection and Further Studies.................................11
2. LEARNING BY DESIGN: TEACHERS AND STUDENTS AS CO-CREATORS
OF KNOWLEDGE.......................................................16
Introduction...................................................16
Conceptual Model...............................................17
Nathans Journey A Case Study................................19
Opportunity..................................................19
"Higher Literacy Skills".....................................19
Course Management Systems....................................20
Embodied Theory behind Student Achievement...................21
Results from a Pilot Study in Nathans Class...................28
Facilitation, Teachable Moments & Media......................28
VII


Pretest and Post-test Comparisons
30
Conclusion.....................................................33
3. INNOVATIVE METHODS OF TEACHING SCIENCE AND ENGINEERING IN
SECONDARY SCHOOLS.................................................34
Introduction...................................................34
Structured Scenario Online Games...............................35
Why Structured Scenario Online Games?........................35
Curriculum-Centered Design...................................37
Conceptual Framework...........................................38
The STRONG Plus Model........................................38
Collaborative Problem Solving and Reflection.................40
Prototype of STRONG............................................40
Design-Based Research........................................40
Contextual and Experiential Learning.........................41
Next Steps.....................................................43
4. INCREASING STUDENT ACHIEVEMENT THROUGH MEANINGFUL,
AUTHENTIC ASSESSMENT..............................................44
Introduction...................................................44
Learning Management Systems....................................45
The Learning Question..........................................46
The Instruction Question.......................................46
The Assessment Question........................................47
The Alignment Question.........................................49
5. GAMES AND SIMULATIONS.............................................52
VIII


Introduction.....................................................52
Why Games and Simulations........................................53
History and Definitions........................................53
Educational Strengths..........................................54
Possible Explanations..........................................55
Challenges.....................................................55
Using Games and Simulations in the Classroom Nathan's Experience. 56
Four Critical Questions........................................56
Games and Simulations are not Teacher-Proof......................58
Recommendations..................................................59
6. NURTURING TEACHER EXCELLENCE USING THE LEARN BY DESIGN
MODEL (LBDM)............................................................60
Learn by Design Model............................................61
Unique Features of LBDM..........................................61
Emphasis on Writing............................................62
Growth Model...................................................62
Intentionally..................................................62
Value-added....................................................63
Formative Assessment...........................................63
Backwards Design...............................................63
Metacognition..................................................64
Challenges and Next Steps for Measuring Effectiveness............64
BIBLIOGRAPHY..................................................................68
ix


LIST OF FIGURES
Figure
1.1 Experimental two-way factorial design for the LBDM Project......................1
1.2 Explicitly teaching thinking to promote transfer of learning....................3
1.3 Learn by Design Model (LBDM)....................................................9
1.4 Experimental three-way factorial design for the OHS-CR2I Project...............12
x


LIST OF TABLES
Table
1.1 Higher-level literacy skills illustrated with sample activities from core subjects.5
1.2 Summary of key findings from my empirical studies..................................11
1.3 Embodied theory compared with other instructional models...........................14
xi


Chapter 1
INTRODUCTION
I do not want to be a TN again" said a teacher-leader at Overland High School (Overland)
just six hours after training on the Learn by Design Mode! (LBDM) an evidence-based
instructional model to increase student achievement. The teacher-leader was saying she
did not want to be a traditional teacher who did not explicitly focus on higher-level literacy
skills in her classroom. Another teacher wrote: I considered the teaching strategies each
trainer used and the materials they referenced. I then realized the importance of being in
the upper left quadrant in every lesson I teach." These teacher-leaders were referencing
the quadrants in the experimental two-way (Teaching x Thinking) factorial design (Figure
1.1) of LBDM, introduced earlier by their trainers.
How did these teachers at Overland become involved in educational research?
This dissertation is a report about how these teachers and I are now in the early stages of
a systemic school-wide curriculum reform project using LBDM and its required set of
intentional and pragmatic
instructional interventions to
increase student achievement
school-wide.
Teachers throughout the
modern era have been challenged
to improve the performance of
students with diverse needs and
abilities. The accountability
measures instituted by the No Child
Left Behind (NCLB) Act of 2001 -
requiring disaggregating the results
of all subgroups of learners by
ethnicity, socio-economic status, Figure 1.1 Experimental two-way factorial design
pupil services, and English language proficiency have brought the challenge of reaching
out to every student that needs attention back to the public's focus. NCLB is a complex
legislation with many impacts and repercussions, some unwelcome to educators and
students. The progressive facet of NCLB (2002) requires that
All children have a fair, equal, and significant opportunity to obtain a high-quality
education and reach, at a minimum, proficiency on challenging State academic
achievement standards and state academic assessments, (p. 1439)
This facet of the law has been a positive driving force in my research. The
narrative in the following section offers a personal account of my efforts as scholar-teacher
working in two high-needs secondary schools in Colorado to increase student achievement
by focusing on student motivation, engagement, and cognition. Then I present the theme
and cohering line of inquiry inherent in all the chapters of this portfolio dissertation. The
next section describes a conceptual model used for increasing student achievement. After
a brief synopsis of each chapter, the penultimate section summarizes the results from my
\. Thinking Explicit Focus No Explicit Focus
on Higher- on
level Higher-level
Literacy Skills Literacy
Teaching^v Skills
EE EN |
Embodied- Student Student j
theory of Learning & Learning &
instruction Achievement Achievement
TE TN
Traditional Student Student
instruction Learning & Learning &
Achievement Achievement
1


empirical studies. The final section concludes with my reflections and planned further
studies.
Journey of Happy Accidents
One way to cope with the mandate to assess and report achievement broken down by
subgroups and account for performance of these groups is to claim that achievement for all
subgroups is unrealistic without adequate resources and funding. Some schools revise
curriculum with a myopic focus only on reading, writing and mathematics. I observed this
tendency during my state-wide evaluation of the Colorado Mathematics Engineering
Science Achievement (MESA) program (Balasubramanian, 2003) an after-school
program targeted toward disadvantaged minority youth. However, what I also discovered
while interviewing MESA advisors was that students participating in activities for an hour
after school, doing applied science, engineering and technology projects, were
subsequently motivated in their core classes of language arts and mathematics. Prior to
that time even as a scientist myself, amidst a family of engineers I had not made the
connection that the skills necessary to excel in science and engineering necessitated good
reading, writing, and mathematics skills. In the words of one MESA advisor:
Career education, learning excitement, critical thinking and meta-cognition,
and increased language skills (crucial for ESL students) [helped my students]
aside from the obvious value of increased comprehension in the content areas.
Seeing the MESA students more engaged, teachers in the core classes started
asking advisors what they were doing with the students after school. Apparently students
who reached elementary and middle schools with limited cognitive skills and few resources
at home or community had become less motivated with a narrowed curriculum of reading,
writing and mathematics remediation lessons. Yet these same students, when exposed to
opportunities afforded through programs like MESA, showed in the words of one MESA
advisor, a remarkable improvement in their language arts, math and science ability.
These students were engaged and understood what they were doing.
This understanding of the link between science, engineering and technology
projects and literacy/numeracy goals was further reinforced while proctoring the Colorado
Student Assessment Program (CSAP) in April 2004. CSAP is a statewide assessment for
students in grades three to ten in reading, writing, mathematics and science that provides
educators and parents with a snapshot view of what students have learned and achieved
each year in school vis-^-vis the Colorado Model Content Standards. The students I
supervised were a small group from pupil services (special education) at a high-needs
middle school. With nothing better to do during supervision, I happened to glance at the
CSAP reading test of a student who was absent. As I started reading the assessment, I
started wondering about the skills students needed to succeed in this test. I soon
recognized a pattern students could succeed at large chunks of the test by merely
demonstrating their ability to think in this mandated exam!
With my interest piqued, I read the mathematics, writing and science tests.
Suddenly I recognized a fundamental purpose of school develop students reasoning and
thinking skills as they moved from one classroom to another. Could I develop a graphic
2


organizer that might promote teaching for transfer-where students learn knowledge and
skills in one subject-discipline that they could then master and apply not only within that
subject-discipline but also transfer across other subject-disciplines, while learning common
reasoning and thinking skills? I developed a graphic organizer and shared it with my
colleagues and students at school to break the popular paradigm of thinking within "silos
of individual subject-disciplines (columns in Figure 1.2).

ILCMSOi OtiV
R Si l',<. f
£)/*> *i-v" ^ t
HZ h 4^i>* n
It; t
y* <** .***. $
i 4r
Wy #rurefe<
Ovsgrxid ^"asvoiopasTty ;atnr> Solatutt'smamar
?, 2m
Figure 1.2: Explicitly teaching thinking to promote transfer of learning
Unfortunately, two math and science colleagues who were excited about using the
framework in their classrooms left my school to move to other middle and high schools
within the district the following year.
In the meantime, I kept pursuing ways to promote students thinking and reasoning
skills. In spring 2005,1 encountered another opportunity to collaborate with industry
professionals by way of the Colorado MESA office. The Hands-On Optics (HOO), a unique
informal science education program funded by the National Science Foundation, was
looking for volunteers to pair optics professionals with science teachers in middle schools
to excite students about science, technology, engineering and mathematics (STEM)
through the world of optics. Although we worked together only for a short time, the
discussions with my Optical Resource Volunteer (ORV), Kipp Bauchert, made me realize
how meaningful collaborations might be forged between K-12 schools and professionals
from industry to make the real-world connections transparent to students. Additionally, I
could see the pivotal role and contribution of K-12 institutions in preparing our students
with necessary thinking and interpersonal skills to stem the tide against retraining a poorly
prepared workforce. These skills would also help all students cope with the unique and
complex challenges of the 21st century (Bransford et a!., 2000), particularly with the death
of distance due to globalization and the World Wide Web (COSEPUP, 2007). At the same
3


time, reforms would need to be tuned to the SCANS competencies based on job
requirements (SCANS, 1993).
Over a lengthy period of classroom experimentation, confirmed by research in
conceptual change (Borges & Gilbert, 1999; Carey, 1999; Champagne, Gunstone &
Klopfer, 1985; Pintrich, Marx & Boyle, 1993; Posner, Strike, Hewson & Gertzog, 1982;
West & Pines, 1985), I have found value in learning activities that are unsettling to the
established expectations of both resource-deprived and resource-affluent students.
Students have a tendency to rush through building activities without much reflection. In
science labs this is due to students' preconception of experimentation as a way of trying
things out instead of testing their ideas (Bransford & Donovan, 2005). What I have found
teaching applied technology, pre-engineering and science classes is that a challenging
scenario that violates expectations can produce a STOP," where students are forced to
backtrack and reconsider the situation. Thus a combination of intensely engaging activity
and deliberative slow-down time for reflection can promote the learning of higher-level
literacy skills (Table 1.1). The unsettling activities are effective in increasing student
achievement among all subgroups of students because it challenges students, not only
those in the extremes (1.0 and 4.0 GPA students), but also the ones in the middle (2.0 and
3.0 students). This STOP to reflect step has been worked into the LBD Model (Figure 1.3).
The resulting reflection cycle, with the added STOP step especially geared toward
secondary students, is a variation on SchOn's (1983) reflective cycle of REFLECT
THINK - ACT. I came to believe that sustained practice in this reasoning cycle can help
all students become proficient or advanced in CSAP. This link was reinforced in May 2006
as I spent three days with colleagues from around the state in the CSAP Standard Setting -
Science meeting to help decide what is good enough for 10th-grade students of
Colorado. Perusing the high school CSAP tests and speaking with students and colleagues
in spring 2007 reaffirmed my conviction that this written assessment is a good measure of
our students ability to reason and think in preparation for college and work and that
students could indeed be trained to succeed at this level of performance.
As a physics scholar-teacher, I am interested in finding meaningful answers to the
question, why is physics worth teaching and learning? I have been passionate about
making physics accessible to all students. This passion has helped me move away from
the extremes because I learned early in my teaching career in 1989, that focusing on a
rigorous, math-based set of algorithms and strategies for teaching physics only helps a few
determined individuals survive. Trying to make physics exciting with fun demonstrations
still makes it a hit-and-miss opportunity. Instead, a middle road of being intentional and
transparent has helped me position physics as an endeavor in developing students' higher-
level literacy skills and effective tools for schools critical thinking, problem solving,
mathematical reasoning, inference making and visualization/modeling. In disseminating
these five constructs so that students, faculty and parents understand and relate to them, I
happily learned in May 2007 that colleagues from other subject-disciplines also easily
related to these constructs and this is further described in the chapter on nurturing teacher
excellence.
Table 1.1 presents how the five higher-level literacy skills critical thinking,
problem solving, mathematical reasoning, inference-making, and visualization/modeling -
are currently defined and targeted across the four core academic subjects; English,
mathematics, science and social studies.
4


Table 1.1. Higher-level Literacy Skills illustrated with sample activities from the
four core subjects (English, mathematics, science and social studies)
1. Critical Thinking (CT)
Critical Thinking = Purposeful Reasoning + Reaching Valid Conclusions
Science CT Illustrative Inquiry Scenario
Using only the materials provided, can you make the Piezo Buzzer beep?
1 fruit, 1 vegetable, 1 buzzer, 2 coins, 2 galvanized nails and 3 wires
Social Studies CT Example
Now that we have examined the development of justice throughout Middle Eastern history,
how would you evaluate justice as it relates to modern government/economic
practices/religious systems/social structures in the region?
Math CT Example
Using the theorems of triangles and angle postulates, how will you prove that two triangles
are congruent?
English CT Example
Using the criteria we have discussed, examine the poem to determine the difference
between your analysis and your opinion of the poem.
2. Problem Solving (PS)
Problem Solving = Overcoming Obstacles + Achieving Goals
Science PS Illustrative Inquiry Scenario
Using the choices (tank shell, golf ball, baseball, bowling ball, football, pumpkin, adult
human, piano or Buick) provided in the Projectile Motion Simulation
(http://phet.colorado.edu/web-pages/index.html)
Determine the angle at which your launched object hits the target?
Can you now hit the target by launching it at an angle that is completely different from the
original angle?
What angle should you launch a projectile to make it travel the farthest distance, with and
without air resistance?
Social Studies PS Example
Create a Bill of Rights a set of laws designed to preserve the concept of justice that
would satisfy the desires of all of your citizens for your developing government.
Math PS Example
Use a general problem-solving plan to create a rule for any number (nth term) in a
sequence using numerical strategies or manipulative models.
English PS Example
How do authors use literary elements to expand the boundaries of reality? Specifically, (a)
Explain this with specific reference to the textual evidence in Night, and (b) Analyze how
these literary elements affect the reader?
5


Table 1.1 (Contd.)
3. Mathematical Reasoning (MR)_____________________________________
Mathematical Reasoning = Abstract Concepts + Supporting Results
Science MR Illustrative Inquiry Scenario
Measure the mass of the six colored containers provided. The containers are filled with
some unknown object.
Look for a pattern among the masses of the six boxes and guess what might be
accounting for the change in the mass of these containers.
Explain (in your results) how this activity might be connected with a topic being studied in
class*.
*Students had studies Electric Forces and Fields when they were given this activity to
connect it with Millikans Oil Drop Experiment in Modern Physics.
Social Studies MR Example
Compare at least two modern Middle Eastern countries and make sure that you include
relevant statistics from the website www.abc-ciio.com in your analysis.
Math MR Example
Can you build the numbers 1 20 by using the four basic operations and only four 4s?
English MR Example
Now that you have started reading the Odyssey, analyze and explain the parallels between
the myths in ancient Greece and the myths in the present day.
4. Inference-Making (IM)
Inference-Making = Logical Reasoning + Informed Decision-Making
Science IM Illustrative Inquiry Scenario
Using only two batteries, two light bulbs and no more than 4 wires:
(i) Demonstrate how both light bulbs can be made to glow, (ii) Demonstrate how both light
bulbs might be made to glow at their brightest. Which of these arrangements would you
choose to use in the headlights of your car. Why?
Social Studies IM Example
Now that you have gone through various activities to understand the background to the
Israeli-Palestinian conflict, as a member of UN think tank, what is your pian to peacefully
resolve the current Israeli-Palestinian conflict? As you present your plan, your peers will
assess your plan on its merits, including: what they liked about your solution; concerns with
your solution; and questions on your proposed solution.
Math IM Example
Use inductive reasoning to make real-life conjectures about how you might survive in an
urban city for two-weeks with only $100.00?
English IM Example
Using specific quotes from the Nobel laureate Etie based on Oprah's interview, what can
you infer about his internal conflict?
6


Table 1.1 (Contd.)
5. Visualization/Modeling (V/M)
Visualization = Pattern Recognition + Communicating to Diverse Audience j
---------------------------------------------------------..._________________l
Science V/M Illustrative Inquiry Scenario
Create a multimedia video presentation to illustrate the difference between gravitational
and electric potential. Sample Worked Example: Concluding Video in
http://doers.us/electrostatics.htm
Social Studies V/M Example
What criteria would you use to assess the successful conclusion of the wars in Iraq and
Afghanistan?
Math V/M Example
Based on the properties of geometrys undefined terms we have discussed, can you
visualize and sketch the intersections of lines and planes?
English V/M Example
Using your knowledge of poetry sensation which we have discussed (e.g., alliteration,
iambic pentameter, etc.) to recognize why the author uses this patterns and rhythm to
communicate the poems meaning.
In summary, I have taught science, technology and pre-engineering long enough -
and to varied student populations to know that students can surprise and outperform
conventional expectations. Yes, the subject matter is challenging, but students can rise to
the task not just the gifted student, or the privileged student, but even students hiding on
the back row or skipping class. Disadvantages attached to various minority groups only
serve to heighten the satisfaction when these students succeed.
NCLB mandates that education not be restricted to the elite. My passion for
addressing the inequities in learning among critical subgroups disaggregated by
ethnicity, socio-economic status, pupil services, and English language proficiency was
reaffirmed during a chance encounter with the radical constructivist and Professor of
Physics at Boise State University, Dewey I. Dykstra in August 2007. Prof. Dykstra has long
written about the elitism inherent in physics education and has spoken about his folk theory
of physics teaching at international conferences: Physics teaching is the presentation of
the established canon by the established methods for the benefit of the deserving"
(personal communication, 2007; italics added). These words have stayed in my mind since
our conversation. My research agenda is partly an effort to respond to these historic
injustices.
I believe there is no better place than Overland to witness and be a part of this
history and change. Over the past six years, Overland has undergone major demographic
changes. Specifically, the student community has changed from a predominantly
Caucasian, middle-class to an international minority-majority school. The student
community now has a diverse population from different social, economic, ethnic, and racial
backgrounds, with 35.6% identifying themselves as African-American, 35% Caucasian,
22.3% Hispanic, 6.5% Asian, and 0.6% American-indian. Students from the school
represent over 60 countries and speak over 54 different languages at home. In addition to


the ethnic diversity and international families, the school now has a poverty rate of 41%.
During a reflective activity in the LBDM training, after listening to Muhammad Yunus story
in the 8th habit (Covey, 2004, pp. 6-9), a teacher-leader wrote: Some of us are convinced
that poor people cant learn, or minorities can't learn. Every child needs to have the
opportunity to learn and they will. Above all, this is an essential understanding for teachers
working in schools with large populations of students from low-income, migrant, and
international families. I consider it an honor and privilege to be working in this great school.
Theme and Cohering Line of Inquiry
Throughout this journey of happy accidents, described in the previous section, several
questions have guided my inquiry as scholar-teacher in the classroom and serve to tie
succeeding chapters together.
How do I as a classroom practitioner motivate and engage all students? Specifically, how
do the average normalized gains (Hake, 2007) of the different subgroups (disaggregated
on the basis of ethnicity, gender, and pupil services) compare when they are instructed
through guided-inquiry hands-on learning?
How do technology-mediated tools impact student learning and achievement? In particular,
how do the effect sizes of the different subgroups (disaggregated on the basis of ethnicity,
gender, and pupil services) compare with the use of such tools for promoting student
learning and achievement?
The core convictions guiding this scholarly inquiry and my classroom instruction have
always been:
Hands-on and minds-on activities that challenge students result in greater student
engagement and learning for all students. Focusing on students' individual ways of
thinking (meta-cognition) is a proven best practice that not only promotes transfer
of learning across subject-disciplines (Bransford & Schwartz, 1999), but also helps
increase student achievement of all students the resource-deprived and the
resource-affluent.
Well-designed games and simulations, with embedded intelligent tutoring systems,
which afford students opportunities to engage and explore core concepts through
inquiry scenarios in a scaffolded learning environment, prior to formal instruction,
will result in greater student motivation and learning. The educational strengths
and possible explanations of why games and simulations are beneficial are
described in detail in the fifth chapter.
Certainly, findings from the latest brain research studies on social intelligence (Goleman,
2006) show that the low road of engaging and motivating students first is the right way to
access the high road" of developing students higher-level thinking.
Conceptual Model
Increasing student achievement is a complex endeavor and continues to challenge
educators world-wide. The conceptual model discussed here describes how this might be
accomplished by teachers and students using a template of activities and protocols for
fostering effective teaching and learning. This required set of instructional practices in the
classroom is called the Learn by Design Model. The model has two components. First, to
8


inspire students, this model uses an embodied theory to provide an explicit template of
activities and protocols for teachers to align curriculum, assessment, and instruction. It is a
set of doable steps based on current theories of learning and cognition to increase student
achievement. The model adopts a backwards design approach (Wiggings & McTighe,
1998), the outcomes-oriented approach of identifying the desired learning goals and then
working backwards to develop assessments and meaningful learning opportunities to
promote student learning and achievement.
Second, to monitor and develop students higher-level literacy skills (Table 1.1),
this model provides an explicit template of activities and protocols that teachers should
focus on to build students' higher-level literacy skills. Activity and protocol checklists are
used to facilitate students integrated learning (Linn, Shear, Bell, & Slotta, 1999) and the
required explicit teaching will drive student achievement further. Equally important, this
Learn by Design Model (LBDM) Focuses on TWO components as
Teachers and Students become Co-Creators of Knowledge
REFLECT
Metacognition
* An Iterative
Metacogmtive Cycle THINK
STOP Used by BOTH
Teacher & Student
First component
Embodied Theory (aka 7-step Model)
Second component
Higher-Level Literacy Skills
1. Critical Thinking
2. Problem Solving
3. Mathematical Reasoning
4. Inference-Making
5. Visualization/Modeling
jT_
Student
Achievement
"71
Oscncslc
Assessment
Assortments for Learrang
f Cognition
ForxziiVG
Assess went Assessment
Assessments ss Learning Assessment! o> Learning
Moilvafcon Cognation Motivation Cognition Maturation
C 2007 Nathan Baiasubrarntmian
Figure 1.3: Leam by Design Model (LBDM)
9


mode! also illustrates how the powerful mission statement of my school district, "To inspire
every student to think, to learn, to achieve, and to care,
(http://www.cherrycreekschools.org) might be implemented within an underperforming
school.
Metaphorically speaking, educators and students have, in the past, tended to focus
on techniques to accurately count the number of passes in the gorilla/basketball
(http://viscog.beckman.uiuc.edu/grafs/demos/15.htm!) video, instead of focusing on other
essential events. This gorilla/basketball video shows an incredible experiment designed
by two psychologists, Simons and Chabris (1999). The experiment demonstrates that
when people pay close attention to an event, however simple, they easily can overlook
other important events. When viewers are asked to try and count the number of basketball
passes between three students wearing white shirts, without counting the passes made by
the three students wearing black shirts, many individuals fail to notice a gorilla passing by.
The experiment highlights the sustained blindness and disbelief in dynamic events when
people can get lost in small details and forget the big ideas.
The LBDM project described in chapter six is unique and innovative because it
explicitly focuses on the gorilla in the room the embodied theory and higher-level literacy
skills (Figure 1.3) to motivate and engage all students and thereby increase student
achievement of all levels of students. Additionally, the embodied theory and higher-level
literacy skills are structured in ways that build effectively students confidence and
competence. The projects success addresses both equity and excellence. Reducing the
achievement gap represents success with equity and raising the academic achievement of
all students represents success with excellence.
Summary of Chapters
To set the stage for understanding the results from my empirical studies summarized in the
next section, here is a synopsis of the five chapters that follow.
Chapter two, Learning by Design: Teachers & Students as Co-Creators of Knowledge,
traces the development of the Learn by Design Model as an effective instructional strategy
to advance student learning and increase student achievement.
Chapter three, Innovative Methods of Teaching Science and Engineering in Secondary
Schools, describes how the STRONG (STRuctured-scenario ONline Games) Plus Model
with the embedded "STRONG" inquiry scenarios might help all students develop good
critical thinking, mathematical reasoning, and problem-solving skills.
Chapter four, Increasing Student Achievement through Meaningful, Authentic Assessment,
elaborates on how learning management systems (LMS) and formative evaluations of
students' written and oral communication skills can result in high levels of learning for a
large numbers of students.
Chapter five, Games and Simulations, elaborates on the educational strengths of games
and recommends five guidelines for educational games to be meaningfully integrated into
classrooms.
The sixth and final chapter, Nurturing Teacher Excellence Using the Learn by Design
Model, summarizes an inspiring story of 13 early-adopter teacher-leaders whose
commitment, ownership and enthusiasm is driving a school-wide, systemic curriculum-
reform initiative with support from the school administration and school district.
10


Results from my Empirical Studies
The conceptual model discussed earlier is a result of my experimentation with designing
workable models of instruction for my guided-inquiry lessons. I believe teaching and
learning are part of a complex evolving activity system that can adapt and improve over
time through increased student and teacher participation.
This section is an attempt to summarize the results from the five chapters that
follow and how I have dealt with numerous day-to-day classroom and institutional
challenges since fall 2003 by focusing on student motivation, engagement, and cognition to
consistently increase student learning and achievement. They would make complete sense
after the reader understands the context in which these real-time studies were done
(described in the chapters) as students learned challenging content from the state content
standards. The reader is strongly encouraged to refer to the individual chapters for further
details. Table 1.2 summarizes key findings from these empirical studies and the results
illustrate how each study addressed the guiding questions described earlier in section
three.
Table 1.2: Summary of key findings from my empirical studies
Subgroups Chapter 2 Chapter 4 Chapter 5 Chapter 6
of Students Direct Instruction Hands-on Activities Hands-on Activities Games and Simulations Faculty Development
"a* 'd' CS> d d d g> 'tf
Entire Class .23 N=56 0.82 .32 N=56 1.4 .44 N=34 1.1 .58 N=40 1.7 0.49 N=13 2.7
Girls 25 N=29 0.93 28 N-29 1.2 ,68 N=11 3.3
Caucasian Male .14 N=13 0.42 .42 N=13 1.8 .41 N=18 0.93 .60 N=18 1.6
Ethnic Minorities 24 N=26 1.1 24 N=26 1.2 46 N=16 1.2 .57 N=22 1.7
Pupil Services .21 N=21 1.0 .23 N=21 1.3
The normalized gains (Hake, 2007) show the increase in knowledge of the
learners due to specific instructional interventions. Although the sample sizes are small in
these studies, these gains were statistically significant (at the established 0.05 level). A
normalized gain of .23 means that the learners in that specific group gained 23%
knowledge because of a particular instructional intervention. The large values of Cohens
d shows the effect sizes of the observed changes in the means in the different studies.
Cumulatively, they show that while the effect sizes were high, all subgroups of students
learned the challenging standards-based content in my classes
Reflection and Further Studies
A trend seen in Table 1.2 in section six is the noticeable student gains among all
subgroups of students even with partial implementation of the Learn by Design Model
(LBDM). These consistent gains are the result of challenging hands-on and minds-on
11


activities that students were engaged in all the empirical studies. As described in section
two, the STOP-step, where students are forced to backtrack and reconsider the situation,
because of the challenging scenarios distinguishes this mode! from traditional forms of
instruction. Traditional instruction relies on delivering instruction to the middle of the class
and differentiating instruction for the extremes. The challenging and unsettling activities,
instead, engage and challenge all students simultaneously. Besides, the deliberate focus
on students individual ways of thinking (meta-cognition) rekindles their intentionality and
inherent preference for goal-oriented actions. The result, like a high tide that lifts all ships
and boats alike to higher levels, all subgroups of students learn and understand at
significantly higher levels and consequently student achievement increases. At the time of
writing these reflections, Overland is applying for a grant entitled Overland High School
Curriculum Reform and Research Initiative (OHS-CR2I) for funding through the National
Center for Education Research and the Institute of Education Sciences (IES) to continue
our work on the LBDM project the systemic school-wide curriculum reform
implementation to enhance student learning in preparation for postsecondary education
and workplace readiness using a three-way (LBDM x Teacher Excellence x Parent
Support) factorial design. The three independent variables for the experimental three-way
factorial design (Figure 1.4) are a high-quality curriculum developed using LBDM, teacher
2x2x2 Factorial Design for the OHS-CR2I Project
Parent Support No Parent Support
Teacher Exce Hence No Teacher Excellence Teacher Excellence No Teacher Excellence

LBDM ***** Student Achievement *** Student Achievement **** Student Achievement ** Studert Achievem ent
No LBDM ** Student Achievement * Student Achievement * Student Achievement 0 Student Achievem ent
Figure 1.4. Experimental three-way factorial design for the OHS-CR2I Project
excellence in both instruction (Downey et al., 2004) and classroom management, and
parent support. We continue nurturing teacher excellence through our weekly small
professional learning community and team meetings for LBDM teachers. We have-plans
for more faculty professional development sessions at both the high and on-campus middle
school with grant funding.
While students continue co-creating knowledge in my classroom, I continue
refining and developing instructional interventions to increase students learning and
understanding of physics. I am collaborating with researchers from the University of
Pittsburgh and Michigan State University and we submitted an IES grant proposal entitled
An intelligent homework tutor for a variety of high-school physics courses that will create
software to help students learn more as they do their physics homework. Since fall 2007,
two of my classes are using Andes, the intelligent homework tutor, and I am comparing
student performance in these two classes with a third class (same level and material) using
12


WebAssign, another commercial homework help tool. This project will help me continue
developing the STRONG Plus Model to gather real-time data on student learning and
performance as students hone their higher-level literacy skills (Table 1.1). While Andes
would be the backend of STRONG, another collaboration that I forged in summer 2007
with Design Simulations Technologies, Inc. in Michigan, designers of Working Mode! 2005
(aka Interactive Physics), might be another front end for STRONG. Paul Mitiguy,
professor of mechanical engineering at Stanford University and lead-developer of
Interactive Physics likes the idea of using LBDM (personal communication, August 13,
2007) to write instruction and further disseminate Interactive Physics world-wide.
As Project Evaluator for the JumpStart grant in the nine-county WIRED (Workforce
Innovation in Regional Economic Development) Initiative awarded to Colorado MESA by
the U.S. Department of Labor will help me focus on science, technology, engineering and
mathematics (STEM) connections to increase student achievement. I would like to further
my science and reading (literacy) research agenda as I continue working with the state
office on evaluating the metrics of MESA.
Finally, I wonder if industry could be more actively involved in K-12 education,
considering the implications of a poorly trained workforce on our economy. What students
learn in school is not only useful for postsecondary education but also essential for their
success at work. The concepts in STEM are used on a day-to-day basis by someone in the
industry. Establishing partnerships with industry would be mutually beneficial because
students would see the real-world connection and industry would not have to spend a
billion dollars retraining its workforce.
Looking back, although the Learn by Design Model (LBDM) and the STRONG Plus
Model were developed for use in my classroom, their use by colleagues from other subject-
disciplines and continuing to develop the models along multiple fronts is beyond the scope
of what I had planned. What the evidence from the empirical studies in this dissertation has
shown is that the models can be used by a scholar-teacher in the classroom to increase
the achievement of all subgroups of students and the use of models developed by scholar-
teachers can become contagious among other teachers. Clearly, using the techniques
embedded in the models require innovative teacher-leaders who are willing to contribute
their time for planning, reflecting, sharing and collaborating with their peers and students to
create engaging technology-mediated learning activities in their classrooms. Early signs
seen by the commitment, ownership and enthusiasm of the 13 early-adopter teacher-
leaders in implementing LBDM at Overland High School are encouraging.
In summary, this dissertation exposited the LBD model an evidence-based
instructional model to increase student achievement developed as a response to
practical problems faced by two high-needs schools in Colorado to meet NCLB mandates.
Models and theories from literature (Table 1.3) were identified and adapted to develop and
demonstrate that the LBDM approach to learning and teaching can yield fruitful results in
the classroom. While the entire model has been shown to be internally valid, external
validity is still pending. A key criterion for successful implementation of the LBD model is
how teachers adopt and adapt to change within the context of this framework. As part of
future research, extant literature from organization change dynamics (Allen, Strathem, &
Baldwin, 2007; Avgerou & McGrath, 2007; James, Mann, & Greasy, 2007; Yeo, 2007) will
be adapted to identify the right kind of motivation and incentives to have teachers embrace
the LBDM framework and possibly adapt it to share even more greater success stories
13


Table 1.3. Embodied theory compared with other instructional models
Embodied Theory (aka 7-step model) (Balasubramanian & Wilson, 2007) Merrill s (2002) First Principles of Instruction BSCS 5E Learning Cycle Model (Bybee et al 1989) STAR Legacy Learning Cycle Model (CTG at Vanderbilt (1992) How People Learn (Bransford, 2000) & Teaching with the Brain in Mind (Jensen, 1998)
STOP. 1. Pre-writes (Think- writes) and Pretests 2, Building vocabulary Activation. Activate existing domain knowledge of the learner through a scaffolded- progression of inquiry activities that increase in difficulty Engage. Capture students attention, stimulate their thinking, and help them access their prior knowledge. Challenge. Demonstrate what students should know and be able to do at the end of a module Use appropriate just-in-time learning stimuli Engage students' preconceptions prior to teaching them new concepts
REFLECT. 3. Simulations and simple hands-on inquiry scenarios as challenge Demonstration. Provide investigative structured-inquiry opportunities for the learner and demonstrate knowledge and skill to the learner, when appropriate Explore. Provide opportunities for students to think, plan, investigate, and organize collected information Thoughts. Students explore what they currently know, including their naive conceptions about topic Provide deep foundational knowledge
THINK. 4. Direct Instruction Explain. Students analyze data and information to further their understanding through reflective activities Perspectives and Resources. Students compare their naive ideas with the thinking of experts & access multiple resources to meet learning objective Help students make appropriate connections within the context of a conceptual framework
ACT. 5. Guided inquiry Hands-on Activity 6. Review 7. Post-writes (Think-writes) and Post-tests Focusing on higher- level literacy skills, using the iterative meta-cognitive cycles of STOP * REFLECT THINK ACT, and the leveraged Motivation Cognition cycles are integral to LBDM Application Provide project and open-ended inquiry opportunities for the learner to apply the new knowledge and skill Extend. Provide student's opportunities for students to apply their conceptual understanding to real world scenarios Assessment. Students apply what they know. Feedback from the assessment should further student learning and motivate them to revise and improve their unaerstanding Organize knowledge in ways that facilitate information retrieval and application
Integration. Provide assessment & sharing opportunities for the learner to integrate this new knowledge & skill into their everyday life Evaluate Assess student understanding Wrap Up Final assessment and expert summary Allow students more opportunities to define learning goals and monitor their progress in achieving them
14


from other classrooms in the school. We also plan to look in to developing a model similar
to LBD to incorporate teachers as change agents in the classroom, and hope to validate
the interna! and external validity of such an approach to the success of adoption of the
LBDM implementation in other classrooms. Observing how the LBD model has become
contagious among colleagues at Overland, combined with the challenge, interactivity, and
gratification afforded through the purposeful communication within the dedicated
professional learning community, I continue reflecting on how research ideas get
disseminated and understood by professionals while garnering change. Organizational
agility is a developmental process that needs support from the top and growth form the
bottom. Despite all the affirmations for the effectiveness of LBDM from students,
colleagues, and professionals, I look forward to learning from the pitfalls and perils of
leading school-wide initiatives on curriculum reform and change through my lifetime.
15


Chapter 2
LEARNING BY DESIGN: TEACHERS AND STUDENTS AS
CO-CREATORS OF KNOWLEDGE
Abstract
This chapter addresses several concerns of teacher-practitioners as schools strive
towards increasing student achievement. It shows how one classroom teacher analyzed
students' academic performance, as measured through pre- and post-test scores, online
think-writes, product designs, explanations and reflections in a guided-inquiry module, to
find that his students made significant gains in specific learning outcomes in science and
technology. Using activity theory as a framework, the authors present a conceptual model
of teaching and learning as an evolving activity system that adapts and improves over time
through increased student and teacher participation. The case study and narrative in this
chapter illustrate how learning is enhanced when students are recognized as co-creators of
knowledge in the classroom and are able to build on their existing knowledge.
Introduction
The problem of improving performance of students with diverse needs and abilities has
concerned teachers throughout the history of modern education. More than fifty years ago
the behavioral psychologist B. F. Skinner designed his first teaching machine after
observing these challenges in his daughter's math class (Skinner, n. d.). Todays
classrooms have similar challenges and are more demanding as teachers are expected to
reach all subgroups of learners-by ethnicity, socio-economic status, pupil services, and
English language proficiency. With limited contact time (Balasubramanian, 2005a;
Bransford, 2000; Davis & Farbman, 2004; Popham, 2003), teachers and schools alone
seem to be held accountable for helping all students meet established educational
standards and perform well on high-stakes assessments.
American classrooms have not fully succeeded in this effort. Results from the 2003
Program for International Student Assessment (PISA) tests showed that 15-year-old
students from 27 countries outperformed the United States in mathematics literacy;
students from 28 countries outperformed the United States in problem solving (NL, 2005).
These results have reopened the debate about what and how students are taught in
secondary schools in the United States (Balasubramanian, 2004).
Here is a report of how one secondary school (Grade 6-Grade 12) classroom
teacher has coped with these challenges by co-opting technology as an aid since
December 2000, and consequently improved student performance in his classes. In
sections three and four, we assume Nathans voice as he provides a practitioners
perspective on efforts to help a diverse range of learners reach high educational standards
in his science and pre-engineering classes. Overall, the research is a collaborative effort
between Nathan and Brent, with Brent in an advisory role providing scholarly leadership,
and Nathan in the classroom trenches solving problems and building successful designs
for instruction.
16


Conceptual Model
In this section we provide a conceptual frame for viewing the activities of Nathan and his
students. In the next section, Nathan traces the development of his ideas about teaching
and their translation into a workable method for guided-inquiry lessons, which he terms the
teacher's embodied theory that is, theory embodied by a template of specific practices in
the classroom. The simple model below illustrates how a teachers embodied theory can
be combined with a core set of tools in this case a course management system and
related Web 2.0 tools to create a meaningful learning environment for students (see
Figure 1).
Fig. 1. Creating meaningful technology-mediated learning environments
Psychology-based learning theory can clarify how individuals process information,
form and revise schemas, and develop skills and knowledge (e.g., Driscoll, 2005). Activity
theory moves beyond individual cognition to see classroom interactions in a more objective
way as a set of nested activities within an overall system meant to pursue educational
outcomes (Kuutti, 1996). Activity theory, growing out of the work of Soviet psychologist Lev
Vygotsky, views learning as the inevitable result of intentional activity over time. Activity
systems are composed of individual agents or "subjects" (teacher and students), each
pursuing objects (learning goals, or more often, performance goals related to an activity).
Teachers and students make use of tools (technologies but also a whole host of other tools
and resources). They collaborate within a specific set of rules or conventions that dictate
meaningful interactions including some division of labor, particularly between teacher and
students, but also between students, especially in working teams.
Michael Cole and Yrjo Engestrdm pioneered the basic analysis of an activity in
activity theory (cited by Bellamy, 1996). Their ideas are widely used for understanding
human-computer interactions, workgroup processes, and learning communities. Fig. 2
represents an activity analysis applied to developing higher literacy skills" (see section
3.2) in K-12 students (adapted from Bellamy, 1996, p. 126).
17


Mediating Tools
(language, pen, paper, whiteboard, computer, etc.)
^ t \
Subject ----- Object ------------ Outcome
(Individual Student's & Teacher) * vfHigher Literacy Skills")
/ ^ \ \
Rules * * Community ^ * Division of Labor
(School, Classroom, etc.) (Peers, Parents, Teachers, etc.) (Principal, Curriculum Specialists, etc.)
Fig. 2 Cole and Engestrdms activity theory framework (adapted from Beliamy, 1996, p. 126).
The basic activity system may be defined as the entire class or a working team within the
classroom, using tools and adhering to established rules and community norms to pursue
objects of value. The activity leads to learning outcomes, whether intended by the
curriculum or sometimes independent of a curriculum (Lompscher, 1999).
An alternate model of Fig. 1 using activity theory (Fig. 2) as a framework,
illustrated in Fig. 3, reflects classroom reality. In this model, teaching and learning are part
of a complex evolving activity system that adapts and improves over time through
increased student and teacher participation.
Fig 3. Student learning environment as an evolving activity system
This figure highlights the bounded activity system typical of classrooms and how that
system takes shape over time. The classroom and its corresponding online environment
contain the basic elements of an activity system, including a guiding set of learning goals
and objects for activity, tools and resources, division of labor, and a sense of community.
The technology-based course management system and websites house the artifacts of
activity, namely, the learning resources developed by the instructor, students, and the
outside world.
Through an activity-theory lens, we see the central tenet of activity and people's
use of tools in pursuit of goals. The learning that happens in Nathan's classes (described in
18


sections three and four) is the result of complex, group-based, intentional activity, using
available tools and resources and following established rules and roles for interaction.
Nathans Journey A Case Study
Opportunity
For 17 years, i have taught physics, applied technology and pre-engineering in middle and
high schools across three continents. Since emigrating to the United States four years ago,
I have immersed myself in two full-time professional responsibilities. First, I had started a
doctoral program in educational leadership and innovation in fall 2002 at the University of
Colorado at Denver and Health Sciences Center. Second, I had taught applied technology
and pre-engineering at a middle school for three years, and now teach physics and physics
engineering technology at a high school in Colorado. Both schools are considered high-
needs because they have a large population of students from low-income migrant families
and the schools overall academic performances were average in 2004-2005 according to
the federal School Accountability Reports (CDE, 2006). I have viewed these school
environments as exciting professional opportunities their "average performance
providing a correspondingly greater potential for improving performance.
Higher Literacy Skills
While interviewing students for my masters thesis (Balasubramanian, 2002) and preparing
a presentation for the first Teachers-Teach-Teachers workshop at Emirates International
School in Dubai, United Arab Emirates, in fall 2000,1 recognized the need for making
classroom resources available online for students and parents. In December 2000,1
designed my first website (http://www.innathansworld.com/). This website includes
extensive resources on various topics that I am passionate about, including physics, career
development, and study skills. While this website afforded an opportunity to present
students and their parents with up-to-the-minute curriculum information and help on
physics, I recognized for the first time how few resources were available to document my
effective classroom practices over the previous eleven years.
In fall 2000, I also wondered about the real purpose of teaching physics to
secondary school students. Clearly, it had to be beyond helping these students be
successful in their International Baccalaureate (IB) and International General Certificate of
Secondary Education (IGCSE) physics examinations. I was really interested in developing
students critical thinking, mathematical reasoning, inference-making and creative problem-
solving skills, what I consider higher literacy skills that would sustain students' lifelong
learning, regardless of the career they choose. In discussions on ITForum (2003), I
explored some ideas for developing enduring higher literacy skills by promoting deliberate
reflective, critical and breakthrough thinking in our classrooms. I proposed integrating
conceptual physics with career development to make learning meaningful to the students. I
now know that focusing on applied science, technology and pre-engineering education in
K-12 can do much to help develop students higher literacy skills and enhance their
career options.
19


Acknowledging the importance of developing students "higher literacy skills"
through technology, the International Baccalaureate Organization (IBO, 2000) concluded:
Schools' technology courses should integrate theory and
practice, including much that is scientific, ethical,
mathematical, graphical, cultural, aesthetic and historical.
They should encourage students to explore the synthesis
of ideas and practices, and the effects of technology on
societies and environments . (p. 9)
These conclusions have been validated by the 90% of K-12 teachers surveyed by the
American Society for Engineering Education (Douglas et al., 2004) who agreed with the
statements: Understanding more about engineering can help me become a better teacher;
a basic understanding of engineering is important for understanding the world around us;
engineering can be a way to help teach students about business; and engineering can be a
way to help teach students history" (pp. 8-10). Clearly, pre-engineering education in K-12 is
supportive and not conflicting with a renewed emphasis on core academic subjects in
schools.
Course Management Systems
In spite of my heavy Web use, it was not until fall 2005, when I first had access to a free
course management system (CMS), that I started consistently monitoring and using
students diagnostic, formative, and summative assessments (see Fig. 5) in my classes to
create a learning repository and critical mass of authentic classroom learning materials.
Some of these resources have been recently featured in an educational technology
magazine (Scrogan, 2006).
Course management systems (CMS) are resource-sharing environments meant to
support delivery of courses from a distance. Examples are BlackBoard, Moodle, and
FirstClass. Services typically supported include document sharing, discussion forums,
multimedia presentations, games and simulations, assessments, and grade management.
In spite of some criticism concerning their embedded ideologies (e.g., Rose, 2004), CMS
have proven useful supports for classroom-based, blended, and online instruction (Wilson
et al., 2006).
This has proven true in my case. Throughout the 2005-06 school year, I used
Schoolfusion a commercial course management systems effectively in my classroom to
Monitor and manage middle-school students work and provide them immediate
feedback
Collect real-time data on students' understanding of science and engineering
concepts
Use the information gathered to guide subsequent instruction
My students accessed these online resources while engaged in inquiry-learning activities.
An analysis of students' academic performance, as measured through pre- and post-test
scores, online think-writes, product designs, explanations and reflections, showed that
these students made significant gains on target learning outcomes in science and
technology (see Balasubramanian, 2006a).
20


Popham (2003) noted that the target learning outcomes handed down by the
states and districts are often less clear than teachers need them to be for the purpose of
day-to-day instructional planning (p. 6). In the following section, I illustrate how I used 41
target learning outcomes from the state science standards (Balasubramanian, 2005b) to
design and develop a guided-inquiry module (Balasubramanian, 2006b). The module:
(a) presents water filtration and the associated concepts in an engaging
way to middle school students
(b) reviews the water (hydrologic) cycle and related vocabulary with
students
(c) provides students an opportunity to design and build a water filter
using only activated carbon, sand, gravel, cotton, plastic cup, wood
structural supports, and hot glue
(d) empowers students to test their filtered water for
conductivity (remove conducting particles so electricity cannot pass),
pH (neutralize pH to make it 7 for a basic solution of salt and baking
soda in water),
turbidity (clean dirty water with tea, vinegar and coffee grounds), and
flow rate (captured filter water should have a flow rate greater than 2 ml/s).
Even as students learn extensive content from the science standards through the
water filter project, the embodied theory (section 3.4.3) provides a roadmap for designing
guided-inquiry lessons that engage secondary school students. More importantly, these
lessons focus on developing students higher literacy skills" and prepare them for their
standardized tests in reading, writing, math, and science. Finally, the module empowers
students by providing them valuable skills for lifelong learning. Implementation of this
guided-inquiry module led to significant increases in student achievement for all subgroups
of learners in spring 2006.
Embodied Theory behind Student Achievement
To foster a nurturing learning environment and student-centered instruction in my science
and technology classrooms, I have students work in teams on authentic and challenging,
yet fun problems. By facilitating these activities in the classroom and reflecting on my own
learning, I recognize the importance of both motivational and cognitive elements in this
adaptive process (Balasubramanian, Wilson & Cios 2005; Balasubramanian & Wilson,
2006). Motivation in particular is a key for many students one that is sometimes
neglected in the compulsory educational systems now in place. The educational theories I
encountered in my doctoral program are both embedded and embodied within guided-
inquiry modules. The modules are a product of these learning theories, combined with my
best creative thinking about how to embody and apply these ideas in real-life classrooms.
Finally, a significant element of serendipity enters as students encounter challenges and
learning materials and respond to them thoughtfully. To some extent the modules are a
product of negotiation and conversation with constituents similar to the idea of design-
based research that is increasingly popular in the literature (The Design-based Research
Collective, 2003). Indeed I consider students to be my collaborators in designing effective
21


learning experiences for them. The sections below give more detail about the water-
filtration module and its conceptual basis.
Motivating students through a token microeconomy."
Helping secondary school students understand and be excited about science and
engineering can be challenging, partly due to negative experiences many have already had
in science classrooms. After presenting students with some initial challenging activities as
a springboard to capture their attention, like moving a ping-pong ball from one beaker to
another without touching either beaker (Movie #5, Balasubramanian, 2006c), I explain that
science is a systematic inquiry directed toward an understanding of natural systems, which
in turn creates new knowledge. The essence of science is not so much what the subject
of the inquiry is, but in how the inquiry is carried out. A complete science education
includes learning the processes, themes, principles, and tools of science. Technology and
science are closely related. You can unlock the power of technology when you understand
the science behind it. You can find out about new technology when you explore the
frontiers of science. Engineering, on the other hand, requires the careful use of limited
resources for solving problems in creative ways using science and technology. Besides,
access to resources is always a challenge at high needs secondary schools. Although the
thinking of scientists, engineers and technologists are not so stereotypical, I use Gilbert's
(1978) synthesis of science and engineering to highlight two distinct approaches to
problem-solving (Fig. 4).
Thinking like a Scientist Thinking like an Engineer
Approaches nature with humility, for there is so much we do not know we are surrounded by a vast sea of ignorance Approaches nature with certainty, because there is so much we know that we have not applied we are surrounded by a vast sea of intelligence
Is content to find out what the world is like as it is Is intent on remaking the world
Has a well-developed methodology, and will do wherever it leads Knows precisely where to go, and will use any methodology to get there
Makes no value judgment of nature it is what it is Begins with value judgment of nature and seeks to create changes that people will value
Seeks knowledge as an end, valuable for its own sake; and worth great expenditures to gain it Seeks knowledge as a costly means that should be applied efficiently if the costs are not to detract from the valuable ends
Fig. 4 Indicators of how scientist's and engineers think
To motivate secondary school students and sustain their full interest and
engagement throughout the learning process, I have used fake money for students to
spend on supplies since fall 2004 in all my classes, after accidentally discovering its
effectiveness in also motivating students. These token "microeconomy" dollars are not only
an incentive mirroring choices and constraints in the real world, but the money also
provides students both individually and collectively constant, immediate, and objective
22


feedback on their performance in each class. The use of dollars challenges them to
become creative problem solvers who are trying to maximize their limited resources.
Before fall 2004, I talked to students about using resources wisely at the beginning of each
school year and before each project. However, it was not very effective. In fact, when
students were building air racers with railroad board paper in fa!! 2004, they used both
paper and glue sticks recklessly. In just one class, students would consume one packet of
24 hot glue sticks. However, from the second week, when decided that students had to
pay five "dollars" to buy a glue stick, they suddenly became very responsible and used
each stick almost to the last bit before they bought another. This serendipitous discovery
was an eye-opener for me, as I no longer have to walk around monitoring resource use in
my classes.
Here is how the system is presently implemented. Students start each
year/semester/quarter with seed money of $50. Subsequently, they earn money in their
classes through their active participation (Balasubramanian, 2005c) and then in turn buy all
the materials or lease tools used in the classroom (like hot glue sticks, foam boards,
cardboard, railroad board, string, marbles, straws, glue, x-acto knives, glue guns, laptops,
probe-ware and so on). These resources cost varying amounts, from $ 1 $200, and
students use them to build and test their creations in their classes.
The monetary system of earning and trading with money has grown beyond the
physical resources. The microeconomy is now tied in with students acting as consultants,
earning royalties from patenting their prototypes, etc. Enjoying the opportunities afforded
with money or borrowing money in a few cases, from Good Bank Inc., (if they had good
credit history) or the alternate Shark Loans Inc. coupled with the social capital they earn
(green I helped card or red I asked for pointers card) has been fascinating in its
dynamic and its power (Balasubramanian, 2005c). In particular, observing a handful of
students borrowing money from my loan shark company because of their poor credit
history (of classroom behavior), when they ran out of money, was interesting. These
students are desperate to earn and return the money at the earliest to avoid hefty interest
payment (20% per week). It makes me wonder if the statistic of more individuals declaring
bankruptcy in the United States than the numbers graduating from college (Godfrey et al.,
2006) could not easily be reversed if more teachers instituted a microeconomy model of
classroom management in their secondary school classrooms. Besides, the social capital
component helps more students move beyond a mercenary approach to a more give-and-
take collaborative approach afforded through meaningful interactions in the classroom.
These goals of collaboration and empowerment stand in contrast to some uses of token
economies, which place more emphasis on behavioral control.
The way in which students, colleagues and parents have resonated with this token
economy amazes me. Moreover, the instantaneous feedback students receive, its highly
contextual nature, and ability to support over a dozen interactions a minute during teacher-
led instruction all of these things make it a highly motivating classroom management
strategy. With a concrete number for processing their learning gains, students easily
recognize where they started (in $) every class and how far they have reached (in $) at the
end of each class.
In fall 2005 I started the school year with the idea of studying the impact of
monetary monitoring on resource utilization and student performance in two of my applied
technology block classes (90 minutes each). One ciass served as a control group where
23


students did not use monetary monitoring and got whatever they wanted. The other class
was the experimental group they had to buy their classroom resources, i presented both
groups with the same problem build a tallest free standing structure that is wind resistant
and resembles a real building using oniy paper clips and straws (Movie #14,
Balasubramanian, 2006c).
I abandoned the study after just the first week because students in the
experimental group were careful with the use of resources and came up with elegant
designs. They had to pay $2 for each straw and $10 for each paper clip. Conversely, the
control-group students, however, nonchalantly depleted these resources. Specifically,
while students in the experimental group barely used one box of paper clips (100
count/box) and one box of straws (100 count/box) in two classes, students in the control
group used over seven boxes of paper clips (over 700) and four-and-a-half boxes of straws
(over 450) in the same time.
Beginning fail 2006,1 moved to teach physics and physics engineering technology
at a high-needs high school in Colorado. In this school, again, the juniors and seniors, and
their parents, have resonated with the microeconomy" model, just like the earlier middle-
school students and their parents. These students also use their resources carefully while
creating elegant and well thought out designs and experiments because they have to buy
and lease their classroom resources.
Bloom's revised taxonomy and levels of thinking.
When I asked middle-school students why and what they liked about hands-on activities, I
heard several fascinating perceptions. One group said they liked doing it, figuring out how
it works. Others said. Putting stuff together was easy; don't have to think as much; don't
have to write as much; and just have to pay attention instead of having to read a lot of
stuff. These same students however thought hands-on activities were sometimes difficult.
They added:
Building it might sometimes be hard because you have it
the wrong way; write-ups and explanations after the
hands-on are sometimes hard; not knowing how to solve a
problem, thinking about it, measuring it right; making
choices, reading a blueprint, putting it together; sometimes
it is frustrating because you cant figure it out; sometimes
your team disagrees about doing things and its majority;
not knowing how to put things together; and remembering
all the stuff sometimes like in a digital multimeter.
As teachers, we know that organizing hands-on activities can be challenging
because these activities require extensive planning, time commitment, organization, and
modified teaching strategies. These challenges are compounded by other constraints in
the classroom, like resolving group dynamics when working in teams, participating
effectively during individual teams discussions and building activities (with 7-10 teams,
typically in each class), promoting greater social collaboration within and between teams,
and coping with students been there, done that attitude that hinders their learning
(Balasubramanian, Wilson, & Cios, 2005). In spite of these obstacles, I use hands-on
24


activities extensively in my classes as culminating activities because even as students
build and test their creations or improve their product's performance, they spontaneously
generate interesting questions. As the subject-matter expert in the classroom, it becomes
much easier for me to seize these teachable moments and help my students think through
their designs, carry out their investigations, and answer their own questions.
Hands-on activities, as valuable as they are, must be connected to formal terms
and the established content of the science curriculum. Recognizing this, I embraced a
revised two-dimensional Bloom's Taxonomy (Anderson & Krathwohl, 2001) to plan and
organize the cognitive elements of my instruction. I framed the learning outcomes in such a
way that students could easily see the transition from simple to complex levels of thinking
for the different projects. For the filter project, even as students design, build and test their
water filters, they discovered the answers to over 37 leading questions in a revised two-
dimensional Bloom's Taxonomy (Balasubramanian, 2005b).
The two-dimensional framework also gave me an opportunity to present the
learning outcomes using a medals-podium analogy. Although the fundamental intent was
to have all students assume greater responsibility for what they learn and win, I believed
that even when students demonstrated simple forms of thinking, like remembering factual
knowledge, their thinking must be recognized with a bronze medal. The farther and deeper
students were willing to think, the more creative and metacognitive they became, and
consequently their thinking and actions must be recognized with a gold medal. While the
intent was to have more students be reflective and creative gold medal" winners, the
structure provided a hierarchy for those learners who were predisposed towards linear and
sequential thinking. This kind of epistemological development, helping students understand
the value of creative and higher-order thinking, is a valuable learning outcome in its own
right.
Embodied theory revisited.
The active doing aspect of inquiry activities motivated several middle-school students
who talked about putting stuff together" being easy." Others suggested that you dont
have to think as much when doing hands-on activities. The same students were quick to
point out, though, that building was sometimes hard and frustrating because, when they
had a problem and couldnt figure it out, they had to think about it. Clearly, hands-on
activities were highly motivating for these middle-school students but were sometimes
cognitively challenging too even for students preconditioned to avoid thinking whenever
possible. Now could I balance the two motivation and cognition to make learning
engaging for these students and consequently increase their conceptual understanding?
This question and the updated Gilbert's Behavior Engineering Model (Chevalier, 2003)
continue to drive the embodied theory for increasing student achievement (see Figure 5).
This see-saw analogy is intended to show the need for both cognitive and
motivational elements and that motivational elements seem to have a significantly
greater leverage than the cognitive elements. Further, the embedded four-step reasoning
process becomes a cycle as students' actions turn back into reflections.
Middle-school students have a natural tendency for just completing activities
without reflection. To extend SchOns (1983) idea of reflection-in-action, i added the stop
before "reflect" in the conceptual framework to add an element of cognitive dissonance,
25


anticipation and intentionality to students learning. wanted to break their rhythm and force
reflection at the outset. In an effort to uncover mistaken preconceptions, I begin by asking
students to respond to a hypothetical scenario and question (Balasubramanian, 2006e) -
similar to a story problem about the content.
Students responses to these think-writes offer insights about their background
knowledge. The built-in feedback in the pretest then gives them an opportunity to find out
what they know and do not know. Having activated their background knowledge with my
diagnostic assessments, students then access an online crossword (Balasubramanian,
2006d) to learn the essential vocabulary in a game-like environment.
STOP
ACT | REELECT / 7
Student Achievement
J 7s

PTVOT
Sumntative
Assessment
Formative
Assessment
Diagnostic
Assessnstnr
Motivatjcai! Elements
(mduAu?, unscKsecoissny
Cognitive Elements
I'ticiiKimp VSedafc-Pcdaim Analogy)
Fig. 5 Embodied theory for increasing student achievement
In my initial design, I did not provide a paper handout. However, at the high school,
one student suggested that she would benefit from a paper version of the essential
vocabulary in her kinematics module. Consequently, I started using a paper handout to
supplement the online crosswords. While using a paper handout that contains all the clues
26


for the crossword, students quickly learn the essential vocabulary while trying to achieve
their highest percentage scores. have had no restrictions on the number of times they
may attempt the online crossword, either at school or at home. The more they practice and
demonstrate their mastery, the greater their monetary gains. The microeconomy
stimulates them to try to do their best and earn plenty of dollars before they are presented
their next challenge. Students have to solve, using a simulation and/or a small hands-on
activity, a simple problem. For the filter-project, students have to arrange six containers,
each containing anthracite, fine sand, garnet gravel, garnet sand, gravel, and rocks, in the
correct order in which they are arranged in a real filter at the water treatment plant. Then
they write down their reasons for their arrangement using both photographs and the actual
samples. Through this activity, students are introduced to two concepts: weight and
density. And again, their writing offers a good deal of insight into their understanding,
revealing if they are on the mark or conceptualizing something very differently (Popham,
2003, p. 88).
By this time, most students have found a clear purpose: to look, listen, and learn
the concepts I then present through direct instruction. By direct instruction, I mean teaching
students explicitly how and why things work by telling them. To give them adequate
opportunities to review the resources presented during direct instruction, I use an online
PowerPoint slide show and movies of students explaining the tests for the filter-project
(Balasubramanian, 2006f). In some classes where I have a prescribed textbook, students
review the material with their textbook, my concept map designed with Inspiration, follow-
up homework, and mini classroom quizzes.
Once students have this rudimentary understanding, I present their final challenge
as a guided-inquiry, hands-on lab activity. By guided-inquiry hands-on activity, I mean
helping students learn by doing, including asking them questions, identifying questions to
investigate (different from simply answering questions), thinking about them, designing
investigations, conducting investigations, and finally formulating and communicating their
conclusions in a structured, challenging and goal-oriented environment. For the filter
project, students have to design a water filter using only activated carbon, sand, gravel,
cotton, plastic cups, wood structural supports, and hot glue to neutralize pH, reduce
turbidity, remove conducting particles, and capture the filtered water. After drawing their
designs and planning how much material they would buy, students have to purchase the
material for building their teacher-approved designs.
The guided-inquiry hands-on activity, followed by tests of students designs and
evaluation by their peers, leads to deeper understanding of the underlying concepts.
According to Perkins (1998), students flexibility in thinking and performing hands-on
activities, beyond the rote and the routine, is one metric for measuring their deep
understanding. The results of students' tests of their water filters showed several students
asking more questions (Balasubramanian, 2006f), making modifications to their designs
and undertaking more investigations. Finally, when they have all had a chance to build,
test, modify, and test their designs, as a class we review the concepts that we set out to
learn in the two-dimensional Bloom's taxonomy (Balasubramanian, 2005b). Students then
take their post-tests to complete the module. The individualized feedback received via the
microeconomy also keeps them motivated along the way. This 7-step process (illustrated
in Fig. 5), I have found, results in significant learning gains for all subgroups of students in
my classes. In the following section we see how inquiry activities, including some
27


unforeseen by the instructor, led to substantial learning gains for students.
Results from a Pilot Study in Nathans Class
Facilitation, Teachable Moments and Media
Several researchers (Balasubramanian, Wilson, & Cios, 2005; Yeo, Loss, Zadnik,
Harrison, & Treagust, 2004) have observed that hands-on inquiry learning without domain
knowledge merely entertains students and results in their inadequate conceptual
understanding. Many resource-deprived students reach schools with limited cognitive skills
and are consequently less motivated. Wilson (1997) observed that direct instruction to
impart domain knowledge in sterile learning environments left students unenlightened and
unable to see its real-world relevance. The intentional, technology-mediated "stops" thrust
on students as diagnostic assessment (pretests, pre-writes, online crossword) and direct
instruction (movies, PowerPoint instruction, and concept maps designed with Inspiration)
have served as checkpoints for reflection. The periodic stops afford students more time
and opportunity to access, process, review, and utilize these resources both in and outside
the classroom.
However, the real fun begins, for both the students and teacher, when students
actually design and engage in hands-on learning activities. For the materials module,
students designed and built their water filters by using only activated carbon, sand, gravel,
cotton, plastic cup, wood structural supports, and hot glue. When they tested their filters,
they spontaneously started asking questions: "How do you design a filter to get a better
flow rate? Does the amount of sand affect the flow rate? Does the order of the layers make
a difference for filtration and flow rate? Did compressing the cotton make a difference?
How many tests do you have to pass to drink the water?" and on and on
(Balasubramanian, 2006e). These spontaneously generated questions are major indicators
of schemas in revision. As some sixth graders reflected, "most people passed three of the
four tests and none of the people passed the turbidity test with the laser." Students'
passion for designing filters that could pass all four tests (conductivity, pH, turbidity, and
flow rate) was fascinating and led to a remarkable investigation involving measurement,
unit conversions, hypotheses testing, and density. The teachable moment serendipitously
surfaced when students wanted to know how they could "pass" the turbidity test. This gave
me an opportunity to highlight sand's adsorbing and absorbing abilities.
The supplementary activity started one day when I asked a sixth grade student to
bring a piece of sponge (used to remove flux and excess solder in a soldering iron) from
the tool room at the back of my class to illustrate absorption. She brought one along and I
then asked the class, "What would happen to this sponge if we soak it in water?" They said
it would become bigger and heavier because the sponge absorbs the water. They visually
and physically verified their hypothesis by soaking it in running water. However, one
student was skeptical and asked "How do you know the sponge become bigger when its
wet then its dry? [sic]" This was a legitimate question and we had not been diligent enough
to record the dimensions or masses of the dry and wet sponge. Thinking nonchalantly that
I could resolve it by bringing another piece of dry sponge from the back of the class, I
asked the student to bring another piece of sponge. However, when she could not find one,
I had to bring a "compressed" sponge from a new soldering iron. Just then, another student
28


had a new question. 'Which would be denser, the dry or the wet sponge?" Acknowledging
that it was a great question, I went on to explain how density depends on both mass and
volume and then guided them through the design of an experiment for investigating the
density of dry and wet sponge. We made our educated guesses about the densities of the
wet and dry sponge before experimenting and students demonstrated their measuring
skills with a ruler and a tripie beam balance. When we started recording and calculating the
density with our measurements, the problem became interesting.
Initially, almost the entire class and I guessed that the wet sponge would be
denser. Our reasoning was that the change in mass was more likely to outweigh the
change in volume. However, the two girls who asked these questions to start with, guessed
that the dry sponge would be denser and seemed bent on proving their hypothesis.
Students took turns carefully measuring the dimensions and masses, and then had their
measurements verified by their peers. Since the first student started measuring the length
using the standard English units, the others continued using the same units. I recorded the
results on a data table (Balasubramanian, 2006g) and showed them how I use Google to
change units from the English to the metric system. For example, I typed in the search box,
2 3/8 inches = ? cm and clicked on search, and bingo, Google immediately returned (2
3/8) inches = 6.0325 centimeters. Students were thrilled to see this and one student
immediately asked "Can Google convert decimals into percents? [sic]" This one student
was disappointed that it could not. At any rate, after recording their measurements, we
converted them to metric units and calculated the density in metric units. Instead of
confirming the hypothesis of the majority of our class, the hypothesis of the two girls
seemed to be validated from our initial results. We were now close to the end of our class
and I asked them what they had learned from this activity.
Students said this experiment showed them:
that the wet sponge has less density than dry sponge; we
learned numbers like g, cm, length, of wet and dry
sponge, that the absorption goes in the middle and the
adsorption goes around it. I also learned that Google
cannot convert decimals into percents, and also if you
squeeze cotton it traps dirt easier; I learned that the skinny
little sponge can grow up to the size of the big one and
can weigh the same; I learned that the wet sponge has
less density; I learned that the wet sponge has less
density by measuring the mass, the weight, and the length
and the height of the wet and dry sponge. I also learned
that there is absorption and adsorption. Absorption is
when the particles go to the inside and adsorption when
the particles stay on the outside; I learned that when you
get a sponge wet, it gets bigger; I learned that the wet
sponge is less dense than dry sponge; I learned that
Google will give you answers to equations; I learned that
absorption goes to the middle of the sponge and
adsorption is on the outside [sic]."
29


Although school ended and I had to rush to a class at the University, I could not
stop thinking about the results of our experiment, i was thinking about these results all
night and decided to investigate our findings further the next day with my eighth graders. I
told them about what had happened the previous day and repeated the student's question
"Which would be denser, the dry or the wet sponge?" I asked them to design an
experiment to investigate this and they repeated the activity. This time though, we used the
same sponge, first for the dry sponge activity and then for the wet sponge activity, during
our investigation. The results this time, in contrast, confirmed our initial hypothesis that the
wet sponge was indeed denser. This was a fascinating learning experience for all of us and
I thought my students had done almost a semesters worth of science in just one class.
When I shared this thought with the eighth graders and asked them to give me an honest
rating from 1-10 on my gut statement, based on their three years of middle-school
experience, the average class rating was an eight. I repeated this claim after sharing the
new findings with my second sixth grade class as well and commended the two girls from
the first sixth grade class for leading us into this interesting investigation. The girl, who
asked the question "How do you know the sponge become bigger when its wet then its
dry? [sic]," spontaneously took ownership for preparing a PowerPoint slide show and
came up with this interesting presentation (Balasubramanian, 2006g). She was one of my
English language learners and a student with pupil services, and her outstanding slide
show is further testimony to what might be accomplished when technology becomes an
aide to motivated students and competent teachers.
Pretest and Post-test Comparisons
The results from the pilot study using a pretest-post-test design with 56 students (one
Grade 8 and two Grade 6 classes), taught at a high-needs middle-school north of Denver
during the schools "waning days (Lyman, 2006) in spring 2006, are summarized below. I
Group N Pretest Mean (%) Pret est SD (%) Post-test Mean (%) Post- test SD (%) t- value P- value Pre-Post A (%)
Entire Class 56 37.9*2.3 17. 1 52.4124 18.1 6.230 <.001 14.514.7
Caucasia n Male 13 50815 0 18. 1 57,714.0 14.4 1.326 .209 6919.0
Girls 29 34.312.6 14. 2 50.613,8 20.3 5.011 <.001 16.316.4
Ethnic Minorities 26 33812 7 14. 0 49.613.0 15.3 5448 <.001 158157
Pupil Services 21 29 412 8 12. 7 44 413.6 16.6 4238 <.001 15,016.4
Fig. 6. Summary of two-tailed, paired sample t-tests on hydrologic cycle test
(before and after direct instruction)
30


Group N Pretest Mean <%) Pret est SD (%) Post-test Mean (%) Post- test SD (%) t-value P- value Pre-Post A (%)
Entire Class 56 3941.4 10. 6 585122 16.6 10282 <.00 1 19.113.6 24 717.2
Caucasia n Male 13 I 40 7i2 2 1 8.1 654150 17.9 5.556 <.00 1
Girls 29 40.512.1 11. 2 57.212.9 15.6 7.593 <.00 1 16.715.0
Ethnic Minorities 26 38.512.0 10. 1 534130 15.3 5.336 <.00 1 14915,0
Pupil Services 21 35.1 2 0 9.4 50 313.1 14.1 4 975 <.00 1 15.215.1
Fig. 7. Summary of two-tailed, paired sample t-tests on water test
(before and after guided-inquiry hands-on activity)
developed the 60 multiple-choice questions from the two-dimensional Blooms taxonomy
(Balasubramanian, 2005b) to assess students' science, technology and pre-engineering
knowledge and skills. I used the same 60 questions for both the pretest and post-test.
Interestingly, despite the small sample sizes and minimal teacher intervention, the
mean test scores increased significantly (except for direct instruction for Caucasian male
students) from pretest to post-test for the entire class, even with disaggregated data by
gender, ethnic minorities (African-Americans and Latinos) and pupil services (SPED, ILP,
IEP & Math Lab). These gains are statistically significant (at the established 0.05 level and
p < .001) suggesting less than .1% probability that the observed differences happened by
chance. The number after in the pretest and post-test mean scores is the error, the
standard error of the mean the standard deviation of the distribution of the mean of
samples.
10
0 J
* Pretest
Post-test !
Linear (Post-1
test) I
---- Linear j
(Pretest) i
Fig. 8. Pretest and Post-test scores of four subgroups in the 18-item
hydrologic cycle test in the filter project, before and after direct instruction
31


! further examined the pretest and post-test scores of these 56 students and found
that the questions were highly correlated. This suggests that the observed changes in
students scores may not be attributed to the regression effect, a regression towards the
mean. Instead, all subgroups had actually made significant gains in their post-test scores
as Figures 8 and 9 illustrate.
The y-intercept of the trend lines in Figures 8 and 9 for the pretest and post-test
data provides interesting information. For the direct instruction, student achievement
increased from 47.4% to 58.2%, showing a 10.8% performance gain. However, for the
guided-inquiry hands-on activity, the increase in student achievement almost doubled,
increasing from 42.1% to 65.5%, showing a 23.4% performance gain.
These numbers are promising when we consider the stark inequities in
engineering education in American society. With decreasing trends in engineering in recent
years (Douglas et al., 2004), Female students make up 20% of engineering
undergraduates, but 55% of all undergraduates; African-Americans, 5.3% in engineering,
10.8% overall; and Latinos, 5.4%, compared to 6.4% overall (p. 5). Experts nationally
have noticed these trends and consciously try to recruit more minorities in science and
engineering through outreach programs. However, the Caucasian male students and their
parents, who are not aware of these trends often feel left out when institutions or teachers
talk about these equity issues. The findings from this study might comfort them, because
they show that with well designed guided-inquiry hands-on science and technology
instruction,
* Pretest
Post-test
Linear (Post-!
test) !
-----Linear
(Pretest)
ri?
^ ^


Fig. 9. Pretest and Post-test scores of four subgroups in the 42-item
water test in the filter project, before and after guided-inquiry hands-on activity
Caucasian male students also make significant learning gains in the post-test scores,
24.7%, more than the 23.4% gain in the trend line. Evidently, guided-inquiry hands-on
learning not only addresses equity issues and increases student achievement for all
subgroups of learners but it also results in significant learning gains for the Caucasian male
students.
32


Conclusion
We started this chapter by introducing the challenges and questions that teacher
practitioners have to deal with in todays classrooms. While students might come from
different backgrounds and differing abilities, learning is enhanced when students are
recognized as co-creators of knowledge in the classroom and are able to build on their
existing knowledge. In addition to providing content expertise, a teachers roie is more of a
facilitator who is responsive to learner needs and actions. We described how the
curriculum standards were operationalized by a teacher through design of a guided-inquiry
module that resulted in significant learning gains for all subgroups of learners. While
substantially hands-on and inquiry-based, the module included elements of direct
instruction and game-like activities. Moreover, the narrative in section 4.1 illustrated how
inquiry activities lend themselves to unforeseen teachable moments based on students
questions, adding a spontaneous level of true inquiry for teacher and students alike.
Our secondary school students arrive in our classrooms ready to collaborate in
both face-to-face and online environments. Teaching and learning are enhanced when
teachers use tools like online discussion forums and interactive games and simulations,
which can be embedded in course management systems to aid reflection, data collection,
and student engagement (Balasubramanian, 2006a; Balasubramanian & Wilson, 2006;
Wilson et at., 2006).
In this chapter we presented a conceptual model of teaching and learning as an
evolving activity system in which the higher literacy skills of critical thinking, mathematical
reasoning, inference making, and creative problem solving are nurtured through guided-
inquiry hands-on activities. Everyone is a winner when students and teachers accept and
exploit the evolving nature of such learning environments. Evidently, using these
techniques require innovative teacher-leaders who are willing to contribute their time for
planning, reflecting, sharing and collaborating with their peers and students to create
engaging technology-mediated learning activities in their classrooms.
33


Chapter 3
INNOVATIVE METHODS OF TEACHING SCIENCE AND
ENGINEERING IN SECONDARY SCHOOLS
Abstract
This article describes the design of an interactive learning environment to increase student
achievement in secondary schools by addressing students' preconceptions, and promoting
purposeful social collaboration, distributed cognition, and contextual learning. The paper
presents the framework that guided our design efforts to immerse all students in a
progression of guided-inquiry hands-on activities. Students find compelling reasons to
learn by responding to authentic science-based challenges, both in simulations and hands-
on activities, based on specific instructional objectives from the national standards.
Keywords: Collaboration, Design-Based Research, Games, Learning, Simulations
Introduction
Schools have numerous responsibilities, including teaching the students observation,
thinking, reasoning, communication and problem-soiving skills. Science and pre-
engineering, properly taught, can help schools fulfill these responsibilities because
students can apply the knowledge and skills learned in their academic subjects to solve
practical problems in their science classes. In particular, developing students conceptual
understanding and analytical abilities through doing authentic science-based guided inquiry
hands-on activities enhances students self-worth and confidence, and consequently
improves their school-wide academic achievement.
Inquiry-based teaching, however, requires highly structured instructional strategies and, as
Cozzens (1997) remarks, demands teachers who are knowledgeable about both scientific
content and pedagogy. Findings reported by Bransford et al. (2000) and Jensen (1998)
about effective teaching and learning strategies highlight the importance of
using appropriate just-in-time learning stimuli
engaging students preconceptions prior to teaching them new concepts
providing deep foundational knowledge
helping students make appropriate connections within the context of a conceptual
framework
organizing knowledge in ways that facilitate information retrieval and application
* allowing students more opportunities to define learning goals and monitor their
progress in achieving them.
Learning, defined by Simon (Balasubramanian & Muth, 2006) as changes that allow
systems to adapt and improve performance, is influenced by both motivational and
cognitive processes. Like Fischer et al. (2005), we believe intelligence and creativity are
34


generated and sustained through active collaboration, interactions, dialogue, and shared
interests between individuals and their socio-technical environments.
However, facilitating the learning and development of students purposeful social
collaborative skills in classrooms during team-based, hands-on problem-solving inquiry-
activities presents perennial challenges for several reasons. The lead author, during his 17
years of teaching science and technology in middle and high schools, has found the
following challenges to be the most demanding.
Motivating all students
Increasing the cognitive skills of resource-deprived students
Sustaining student engagement
Addressing students' preconceptions
Creating time to participate and contribute effectively during individual teams discussions
and building activities (with 7-10 teams typically in each class)
Promoting greater social collaboration within and between teams
Resolving problems with group dynamics
Coping with students' Been There, Done That attitude
Inducing students to build well thought out designs while advancing their metacognitive
skills
Constantly developing genuinely interesting challenges and activities.
Etheredge and Rudnitsky (2003) observed that fully implementing findings from research
and coping with classroom reality has often been overwhelming for teachers and students.
This paper describes our preliminary efforts at addressing these challenges using a design
experiment to inform both theory and practice. The conceptual framework (section 3.1)
describes the theory. Concurrently, we developed a prototype and necessary instruction for
teaching the concept (NRC, 1996) that electrical circuits require a complete loop through
which an electrical current can pass (p. 127) to middle-school students.
Structured-Scenario Online Games
Why Structured Scenario Online Games?
The middle-school wonder years are critical periods in the personal, emotional, social, and
cognitive development of students. During this period, students have a tendency to rush
through building activities without much reflection. Bransford and Donovan (2005) observe
that this is due to students preconception of experimentation as a way of trying things out
instead of testing their ideas.
Baiasubramanian and Wilson (2006) describe students enthusiasm for learning and
sharing their experience after playing the promising educational games designed by the
Nobel foundation. We define a game as an engaging interactive learning environment that
captivates a player by offering challenges that require Increasing levels of mastery. The
Laser Challenge Game (2005) designed by the Nobei Foundation exemplifies this
definition. In our classroom study, we found that all middle-school students, disaggregated
35


by gender and ethnicity, made significant learning gains after playing the challenging Nobel
games.
Believing in our five guidelines (2006) that are necessary for games and simulations to be
meaningfully integrated into classrooms, we designed STRuctured-scenario ONIine Games
(STRONG, in short) as modular, self-contained, easily accessible, multi-player, online
interactive learning environments, to direct, facilitate, and assess students' conceptual
science, technology, engineering, and mathematics (STEM) understanding through
deliberate reflection.
STRONG scenarios and challenges are designed to promote a deliberate STOP <
REFLECT THINK ACT approach to rekindle students' intentionality and inherent
preference for goal-oriented actions. Besides, as Balasubramanian (2003) discussed, such
deliberate thinking fosters self-organized learning. SchOn (1983) remarked that such
reflection-in-action" situations also foster new ways of thinking and coping with surprises.
The engaging scenarios in STRONG unfold as cliff-hanger chains of events to captivate
students attention, stimulate their motivation, and provide meaningful contexts for learning.
For instance, a dialogue between Peggy and Cassandra (fictitious names for students
online avatars, Fig. 1) in our STRONG prototype under development, sets the tone for
students finding compelling reasons to design a warning device after they have suddenly
fallen into a dark cave during a hiking adventure.
Peggy. Oh great! Now what are we going to do?
Cassandra: Sweet! Lets play cops and robbers.
Peggy. We need to get help quick.
Cassandra: Are you kidding me? This is freaking awesome.
Peggy. Are you kidding ME? This is freaking . FREAKY.
Cassandra: No way, this is the ultimate opportunity to play the best, the most extreme, the
greatest game of cops and robbers known to humankind.
Peggy. OK, just one game, but after that were getting help.
Cassandra: Deal! Im the robber, you try to find me.
Peggy. OK, go. (a couple of minutes pass)
Peggy. Uh Oh! I cant find you. This is scary. Where are - (cut off because she fell). I
tripped on a rock. Help me.
Cassandra: HA HA HA, you tripped. I mean ... are you okay?
Peggy. Yes, Im fine. I tripped on this rock.
Cassandra: Thats not a rock. Its a treasure chest from the old Captain Willy.
Peggy. I dont think we should open it, there could be something dangerous in there. Lets
get help first.
Cassandra: Oh yeah! I have my cell phone, we could just call my mom.
Peggy. Why didn't you think of this before?
Cassandra: Uh oh .. .
Peggy. What?
Cassandra: No signal, I hate my phone service, it never works
Peggy. We're doomed. Well, I guess we could open the box to see what's in it...
Cassandra: It's not a box. It's a treasure, but let's look inside, (open the box)
36


Peggy. It's some wire and . .
Cassandra: Gold?
Peggy. No a light bulb and ..
Cassandra: Gold?
Peggy. No a battery. We can put this together to make a signal to get us out of this eerie
place.
Cassandra: We could scream for help, someone might hear us as well.
Prefer 97R30W
! l -.....-! i--------------
Fig. 1 The STRONG Interface
Then a circuit construction (PhET, 2005) Java simulation pops up on the screen for
students to experiment with and build circuits for a warning device using wires, three light
bulbs, two batteries, and switches in a safe and non-threatening environment. When
students use two batteries, they learn that there is a right way and a wrong way to connect
batteries. Using three light bulbs leads to a better understanding of series and parallel
circuits.
STRONG scenarios are designed to enable more students to view surprise and failure as
potential opportunities that help them develop good problem-solving, reasoning, and critical
thinking skills as outlined in the Benchmarks for Science Literacy (AAAS, 1993).
Curriculum-centered design
From their review of educational gaming literature over a period of 28 years, Randel et al.
(1992) concluded that games could be used effectively to provoke interest, teach domain
knowledge, and shore up retention in math, physics, and language arts when specific
instructional objectives were targeted.
In our early design of STRONG, students learn, use and understand one concept from the
National Science Education Standards (NRC, 1996), "electrical circuits require a complete
loop through which an electrical current can pass" (p. 127), while building simple electrical
circuits for a warning device. Along with this concept, players of STRONG will learn and
use the knowledge and skills in three labeled strands in the Atlas for Science Literacy
(AAAS, 2001): lines of reasoning, failure, and interacting parts.
37


There are four levels in STRONG: beginner, intermediate, proficient, and advanced to
correspond with the primary, (K-2), elementary, (3-5), middle, (5-8), and high, (9-12) school
grades in the Benchmarks (1993). The outcome variables in these four ievels of STRONG
are the developmental^ appropriate STEM knowledge and skills tabulated and color-coded
at http://www.GamesToLearn.us/ConceptForSTRONGPrototype.htm. Using appropriate
scenarios, these Benchmarks (1993) are packaged as appropriate challenges for students
in the different ievels of the game, to interest both resource-deprived and resource-affluent
students in their preparation for active inquiry learning.
For instance, at the intermediate level of the game, players demonstrate understanding of
how a simple circuit is connected by wiring a warning device using only one light bulb, one
battery, and one wire and answering assessment questions correctly. The corresponding
Benchmark (1993) on failure, 11A/E2, requires students to know that something may not
work as well (or at all) if a part of it is missing, broken, worn out, or misconnected" (p. 264).
Conceptual Framework
The STRONG Plus Model
Hands-on inquiry learning without domain knowledge merely entertains students and
results in their inadequate conceptual understanding. Many resource-deprived students
reach schools with limited cognitive skills and are consequently less motivated. Wilson
(1997) observed that direct instruction to impart domain knowledge in sterile learning
environments left students unenlightened and unable to see its real-world relevance. To
cope with this dilemma, we describe the STRONG Plus framework that seeks to immerse
all
-/ /
/ /
; i
A
/ \
// \,
// Projects
/OPEN IN QUERY \
/ Accomplishments
' FORMALIZATION
& CIMttRVtl tMDEMUWlDrc
Hands-on Activities
GUIDED INQUIRY
7
STRONG
INQUIRY SCENARIOS
\L
Fig. 2 The STRONG Plus Model, illustrating our conceptual framework.
38


students in a progression of guided inquiry hands-on activities to facilitate their conceptual
STEM understanding, starting with STRONG and proceeding to iess guided forms of
inquiry learning (see Fig. 2).
The pedagogical strategy underlying this conceptual framework is adapted from Vygotskys
model of developmental teaching. Giest and Lompscher (2003) propose three stages in
Vygotskys zones of student development: learn-by-doing in students zone of actual
performance (ZAP), learn-by-inquiry in their zone of proximal development (ZPD), and
learn-by-developmental teaching where they construct and develop their understanding
when their ZPD becomes their new ZAP and so on.
Although designed to be pre-reflective of the formal subject matter, STRONG elicits, first of
all, students' rudimentary and incomplete conceptual understanding and prior knowledge in
their ZAP. Students work in teams (of two recommended) to solve challenging problems
and accomplish various goals embedded in the game. The small-team setting promotes
greater sharing of ideas among young adolescents without fear of negative judgment by
their peers, and helps elicit their preconceptions and fragile conceptual understanding
during their social interactions and peer mentoring.
McDonald and Hannafm (2003) noted that web-based games promote higher order
learning outcomes and understanding because they increase meaningful dialogue among
the students and help identify students misconceptions, both of which are not easily
obtained in traditional classrooms without conscious teacher mediation. Bransford and
Donovan (2005) refer to the success of a computer-based DIAGNOSER in increasing
students' understanding of high school physics concepts when the program helped
teachers elicit students preconceptions.
Although rudimentary, the STEM content- and context-specific student discussions
necessitated through play in STRONG, empowers students with new ways to talk, think,
and act in middle schools (Roth, 2002).
After engaging all students using the game, teachers could use the student performance
data to provide formal explanations, promote further reflection, and guided-inquiry hands-
on activities to develop students' knowledge and formal conceptual understanding in their
ZPD, before formally assessing student accomplishments.
According to Perkins (1998), students flexibility in thinking and performing hands-on
activities, beyond the rote and the routine, is one measure of their understanding. Then,
observing students creative and imaginative solutions to problems, and finally students
attitude and engagement towards challenges encountered during hands-on activities are
other authentic metrics of understanding.
Finally, students learn through developmental teaching using projects and problem solving.
In developmental teaching, students ZPD in the second stage becomes a new ZAP. This
iterative process continues through the three stages as students transition to higher levels
of learning and become more active self-directed learners.
39


The STRONG Plus model in Fig. 2 illustrates our preference for engaging all students with
the game first, then providing them with forma! explanations and opportunities for hands-on
investigations, and concluding with formal assessments and projects to promote
conceptual STEM understanding.
Collaborative Problem Solving and Reflection
Coliaborative problem solving and deliberate reflection are two cornerstones in all four
stages of the STRONG Plus model. Starting with a well-designed game increases the
domain knowledge and motivation of all students because more students would have an
opportunity to participate in stimulating and thoughtful conversations in a non-threatening
high-challenge small-group gaming environment, before engaging in less guided forms of
hands-on inquiry learning.
Reports from classroom observations, like the one from Horizon Research (Weiss et al.,
2003), show that the weakest elements observed in science and mathematics classrooms
are the limited time, opportunity, and structure for students to engage, ask questions, and
understand all the material. Tools, like STRONG, provide a basis for more doing, testing,
reflection and metacognition among middle-school students. Bransford and Donovan
(2005) describe how using ThinkerTools, a physics inquiry curriculum, the low-achieving
students from inner-city schools have shown a deeper conceptual understanding of
physics because of the metacognitive component in the reflective assessments.
STRONG requires little or no teacher intervention during play. However, students' typed
responses in the assessment fields are recorded and processed continuously during the
15-20 minutes of play. Students receive instant feedback on their performance, in the
assessment windows and reflection space, from embedded critics in the game.
Critics are agents that provide context-specific advice to users based on their inputs in a
computational environment. As observed by Cios et al. (1998), the dynamic feedback
students receive, based on the embedded fuzzy logic and machine learning techniques in
the STRONG system architecture, promote students active learning.
Prototype of Strong
Design-Based Research
Section one in this paper discussed the complexities and challenges associated with
STEM teaching and learning. Section two described how STRONG uses backward design
(Wiggins & McTighe, 1998), an outcomes-oriented approach requiring identification of
desired learning goals and then working backwards to develop meaningful learning
opportunities and assessments, to promote learning. The STRONG Plus model elaborated
on in section three described how the dilemma of informing" through direct instruction and
doing in inquiry-based learning might be reconciled.
40


We considered the development of our prototype as a design experiment because it
afforded us opportunities to theorize and address the complexities associated with
learning. Cobb et al. (2003) recommend that the primary goal of facilitating learning is to
improve initial designs by repeatedly testing and revising conjectures. These
recommendations have guided us in the development of the STRONG Plus framework and
we subsequently used this theoretical model to design a prototype that facilitates student
learning.
In addition to teacher observations and feedback, tools like STRONG will help researchers
gather real-time data on student learning and performance. Besides, student performance
on their diagnostic assessments (their online pre-tests) and post-tests are used to test and
improve the design of our prototype.
In summary, our research agenda has a two-fold purpose. The STRONG Plus model
depicts our early efforts at developing a theory. Designing a prototype as we developed
assessments and necessary instructional support materials to improve practice is another.
Contextual and Experiential Learning
The case study by Yeo et al. (2004) and our personal experiences show that interactivity
and animated graphics in games and simulations, by themselves, do not help students
learn basic scientific and engineering concepts. Students need additional supports to
promote deep conceptual understanding. The Flash animated scenarios in the game not
only provide a context and purpose but they also motivate students by enabling them to do
science.
When students are ready to test their understanding of a concept, say, electrical circuits
require a complete loop through which an electrical current can pass, they will answer six
questions that promote their higher order thinking. These six questions are generated
randomly from a library of twenty-five questions, unique to each level of the game. This will
minimize chances of students misusing the online chat to exchange notes with correct
answers.
For instance, in one type of question having several possible correct answers, a student
will have to select all choices that apply. The possible answers might include: The wire is
* warm cold; the light bulb is on off; the light bulb glows very bright and
burns out does not burn out.
Students correct, partially correct, and wrong answers have pre-assigned fuzzy logic
scores from +1 to -1. This is combined with another unique feature in STRONG asking
students How confident are you in your answer?" The confidence multiplier, varying from 1
-10, for I am guessing and I am 100% confident, respectively, multiplies the raw score
(with fuzzy values between -1 and +1), before displaying scaled team scores.
With numerous genres available, the term game has been elusive to define. Glazier
(1973), Prensky (2001), and Rasmusen (2001), have described the presence of the
41


following basic components in games: 1) Player Roles, 2) Game Rules, 3) Goals and
Objectives, 4) Puzzles or Problems (Challenges), 5) Narrative or Story, 6) Players
Interactions, 7) Payoffs and Strategies, and 8) Outcomes and Feedback. Our games,
defined as engaging interactive learning environments that captivate a player by offering
challenges that require increasing levels of mastery, include these basic components
(Table 1).
table 1: STRONG and Basic Components in our
Rudimentary Game intermediate Level
Basic Game Components STRONG
1. Player Roles Players select one of the six online avatars and watch scenarios unfold. Our current design does not give players more freedom and control over their clothes and their environment, but these power-ups will be incorporated in subsequent designs to reward higher team scores.
2. Game Rules Students take a pretest (hands-on and online), watch engaging scenarios unfold as Flash movies, use embedded electrical circuit construction Java simulations, answer six randomly selected questions, and take a post test (hands-on and online).
3. Goals and Objectives Players will learn, use and understand at least one core concept from the standards, while building simple electrical circuits for a warning device.
4. Puzzles or Problems (Challenges) Players demonstrate an understanding of how a simple circuit might be connected for wiring a warning device, using only one light bulb and a battery. Each STRONG assessment question is a puzzle or problem or challenge in itself.
5. Narrative or Story The dialogue about cops and robbers between Peggy and Cassandra when their cave is suddenly engulfed in darkness depicts a typical scenario in STRONG.
6. Players Interactions Student discussions, building various circuit designs using hands-on and Java simulations, answering six questions
42


(three for each player) for assessment even as they alternate and collaborate represents expected interactions.
7. Payoffs and Strategies What kind of confidence multiplier factors might players use? With raw scores varying from -1 to +1, multiplying it with a multiplier could change the final scaled team scores significantly.
8. Outcomes and Feedback (Embodying concepts to be learned Players learn and demonstrate understanding of the concept electrical circuits require a complete loop through which an electrical current can pass," after reflection on the critiques and feedback in the STRONG prototype.
As students play the game, real-time data on their performance will be collected into a
database. The embedded critics in the game will offer contextual clues, when necessary.
For example, a comment in the reflection space could be Have you considered connecting
this circuit in the Java simulation and seeing what happens?" The contents on the
STRONG home page http://GamesToLearn.us include relevant Benchmarks (1993),
sample worked examples, STRONG assessment, and links to the Java simulations of a
STRONG prototype.
Next Steps
Mitchell and Savill-Smith (2004) noted that players' limited pre-existing computer skills,
teacher bias towards learning methods, and possible conflict between game and learning
objectives could impact the benefits of using a game, but as knowledge engineers of
STRONG, we believe the effect of these would be minimal because of the game design.
The STRONG Plus model has guided our design efforts in developing a prototype to help
students explore and understand electrical circuits. While the existing prototype can be
played online at http://GamesToLearn.us, we continue testing and improving our initial
design.
In conclusion, a tool like STRONG empowers both students and teachers. STRONG meets
learner needs because it supports students' preference for learning by doing. STRONG is
promising for instructors because it supports teachers who engage students with hands-on
inquiry learning. A solid foundation in STEM during students critical developmental years
will help them enhance their lifelong learning goals.
43


Chapter 4
INCREASING STUDENT ACHIEVEMENT THROUGH
MEANINGFUL, AUTHENTIC ASSESSMENT
Abstract
This paper describes how the author uses an online communication and assessment tool,
SchoolFusion, to (a) monitor and manage middle-school students work and provide them
immediate feedback, (b) collect real-time data on students' understanding of science and
engineering concepts, and (c) use the information gathered to guide subsequent
instruction. Quantitative data analysis showed that the mean test scores increased
significantly from the pre-test to the post-test across the entire class. Students responses
in online think-writes also revealed students' improved conceptual understanding of
scientific and engineering principles.
Introduction
It was Friday evening, four weeks after school reopened. I noticed my colleague at
Angevine Middle School walking out of school carrying a case overflowing with over eight
reams of students mathematics worksheets for grading over the weekend. Not long ago I
too had carried reams of physics papers for grading over weekends, not always completing
my planned marking. As teachers we are often overwhelmed with a backlog of papers to
grade and continue with our daily instruction, regardless. Meanwhile, students miss the
timely feedback that impacts their learning. Obtaining and using data on students'
individual learning in the classroom on a daily basis is a challenge.
On top of all this, with increased calls for accountability in school systems and fear of
federal sanctions based on the No Child Left Behind (NCLB) Act of 2001, several schools
have had to shuffle priorities and focus on reading and mathematics, while other subjects
have been relegated to lesser importance. In Colorado, students are tested" in science
only at the end of Grade 8, and soon will be in Grade 5. However, there will be no data on
their yearly performance. Not surprisingly, the authors in the Designs for Science Literacy
(AAAS, 2001) observed that students learn too little in science (p. 51) because there is
not enough time for teaching and learning science.
Besides, going by reports from the Program for International Student Assessment (PISA)
http://www.pisa.oecd.org and the third version of the trends in Trends in International
Mathematics and Science Study http://nces.ed.gov/timss/ (TIMSS) studies, there is
growing concern and debate about students problem-solving abilities. Inspired by
Russells (2004) article in a special issue of Curriculum Inquiry, my comments in Education
Week (Balasubramanian, 2004) highlighted related questions: How do we select the most
appropriate materials to teach? How do we determine the most efficient and effective ways
to teach this material? Are we teaching as well as we can? Are we teaching as many
students as we can? Do the techniques employed in schools enhance students' self-worth
44


and confidence? Do we have ways of using and monitoring ongoing formative
assessment?
In this paper, I illustrate how teachers can motivate and empower all students through
positive and timely feedback by using an online learning management system effectively.
Learning Management System (IMS)
Besides weblogs and wiki-variations, commercial and open source internet-based ieaming
management systems marketed as content management systems or course
management systems are burgeoning. These website-in-a-box technologies are often
perceived and presented as the panacea for K-16 and lifelong education. Ellen Rose
(2004), peppered with Cassandra-like prognostications, raises valid questions about the
ideologies and assumptions underlying the emergent website-in-a-box technologies in her
paper. She traces the origins of these purportedly personalized, meaningful, empowering,
and ultimately learner-centered educational environments (p. 57) to the low-tech teaching
machines of the 1950s, computer-assisted instruction (CAI) of the 1980s, and integrated
learning systems of the 1990s that promised individualized instruction.
Ellen Rose argues that these emergent technologies are really Trojan horses that seek to
replace human interactions in the classroom. They are neither teacher-centered not
learner-centered but merely technology-centered systems, she alleges. Although
computer managed instruction (CMI) might appear to be a teachers tireless machine
servant to record students progress, pre- and post-test scores, and the like (p. 52), they
are driven by the vested interests of CMI researchers, Ellen asserts. According to her, the
emerging online website-in-a-box technologies promote a cult of efficiency, atomized
content delivery, and static classroom packages while demanding that teachers relinquish
their decision-making powers to these technologies.
In this paper, I illustrate how I use one of these commercial content management systems
effectively in my classroom to monitor, manage, and use data to inform my classroom
instruction more like a CMI without the vested interests of a CMI researcher. Although I
gathered information on students aptitudes and attitudes in past years, I had not done any
formal study on their learning. Since I had access to a new, free LMS that I use as an
online communication and assessment tool, I felt confident that I could pursue a more
systematic exploratory study using pre- and post-tests.
I was keen on finding out what, if any, students were learning in my classes. I wondered
whether l would see an improvement in student scores after instruction. Specifically, would
I see improvements that were independent of gender and ethnicity? The following sections
illustrate how I designed my courses around four critical organizational questions that
guide teaching and learning: learning, instruction, assessment, and alignment questions,
following Anderson and Krathwohl (2001, p. 6).
45


The Learning Question
Students spend just 14% of their time in school each year (Bransford et a!., 2000). What is
important for students to learn in the limited school and classroom time available?
Following a training in Fall 2002, teachers at Angevine are expected to have learning
objectives written on the board, following the SIOP Model (Echevarria, et ai., 2000). During
one of her classroom observations in October 2003, the Principal of Angevine remarked:
Try and focus on what the outcome of the learning is, rather than on the task. My learning
objectives have since, gradually, become specific.
Students in the applied technology classes were learning various scientific and engineering
principles through building activities. For instance, in Designing Beams, I had specific
expectations on students learning objectives and vocabulary, in contrast to Heavner et al.,
(2004) Breaking Beams. My objectives were students will: (a) recognize various types of
beam designs; (b) understand forces, especially forces of tension and compression; (c)
explain, where in the beam, these forces are greatest and why; (d) rank building costs,
depending on materials used, particularly for wood, concrete, reinforced concrete, and
steel; (e) design a prototype of a beam using the design process; and (f) test, calculate,
explain and evaluate the strength-to-mass ratio of their beams.
The Instruction Question
How does one plan and deliver instruction that will result in high levels of learning for large
numbers of students?
Since Fall 2004, students at Angevine also take Cornell notes during classroom instruction.
Teachers have encouraged and modeled quality note-taking in all classes, and this section
describes how students tracked their progress in their technology classes.
Students learning, their ability to adapt and improve performance, is influenced by both
motivational and cognitive processes (Balasubramanian, Wilson, & Cios, 2005). I digress
briefly to mention that in all my classes, students work in teams on numerous hands-on
and minds-on activities by doing and applying concepts learned at school. Hands-on, in
practice, translates to resource-intensive and more planning. Furthermore, early in Fall
2004,1 learned how Monopoly-like money can be a significant motivator for learning in
middle schools. My description of a creative activity (Balasubramanian, 2005d), provides a
brief description of this serendipitous discovery. This microeconomy through monetary
monitoring has evolved into a full-fledged classroom management system
(Balasubramanian, 2005c) and is part of another paper describing ongoing assessment of
students learning.
Before students received any instruction on designing beams, they took a timed online pre-
test. Such tests are easy to create using the flexible online K-12 learning management
system developed by the www.SchoolFusion.com team. I could experiment with the tool for
free because the developers offer their classroom course shells, free for life, for the first
46


three teachers in any school in United States." My classes can be viewed at
www.angevine.groupfusion.net
Students then received a half-hour instruction that addressed the six learning objectives
outlined earlier. With money being a significant motivator, students actively participated
throughout the class discussions. The tools students use in class are always free, but each
team then bought their supplies. One yard of balsa wood cost $200 and a bottle of wood
glue cost $50. After brainstorming and sketching their designs on graph paper in ten
minutes, teams had another half-hour to build their design. They used clamps to secure
their designs and let them dry over the weekend.
The Assessment Question
How does one select or design assessment instruments and procedures that provide
accurate information about how well students are learning? More importantly, how does
one use this information to inform instruction?
At Angevine, since Fall 2004, teachers are required to have students track their academic
progress through Assignment Logs, containing a record of all their graded work. Students
had well over ten graded assignments in their logs in just four weeks because they did their
quizzes and think-writes on their class website.
In Systems for State Science Assessment, Wilson and Bertenthal (2005) summarize 18
assessment approaches to cope with impending NCLB mandates. The written pre-test,
drawing and students problem-solving abilities in designing beams described so far
illustrate three strategies that assess students understanding.
During their next class, students tested their beams (Movie # 12 in Balasubramanian,
2005e). The oral presentations afford opportunities to assess students communication
skills. I use these presentations for peer-assessments too. After every presentation, the
captains of each team confer with team members and write down scores based on a 50-
point scoring rubric using five criteria: design, creativity, explanation, cost efficiency, and
test-endurance.
Following their presentations, I wrote down the test results on a transparency and we
discussed them as a class. The results of the ten teams are summarized in Fig. 1. The first
five were from the first class and the last five are from the second class. Interestingly, in
both classes, the beams that won (Team 3 and Team 10) did not withstand the most load.
Consequently, students understood the importance of strength-to-mass ratio. In the
second class, with the results being so close, Team 6 was disappointed at not winning
because their beam withstood the greatest load of 229 g. Consequently, they had
questions about Team 10s weighing. The two teams verified each others weighing using a
triple beam balance and, reluctantly, Team 6 declared Team 10 the winners.
47


Teen No ictzn :inc i ilard t n btam -. ft-? Sicorct' Vo-mjcs (zti> ;tr fee;;
!
1 \ 3 5
2 ^ | 5 m "
54 U ! ig 1
1 8
rn I it 35
13c tX 12 1
? 10 c 20 23
it e m S.S
2Q 5 10 4
1C Mi 1C1 12.T
Fig. 1. Strength-to-mass ratio of different teams
To follow up on their written assessment, students took an online post-test titled Beams,
Materials, and Forces." Although the questions in the two tests were the same, the order of
questions was different and the tests had different titles." Almost all the students thought
they were taking different tests. This is another advantage of using SchoolFusion. As
teachers, we can create these tests, besides mid-term or end-of-term tests, easily using
the Online Quiz feature from a repository of questions we have created throughout the
semester. Moreover, I found it was easier to cope with common classroom challenges
associated with students tardiness, truancy, absence, and desire to improve their grades
with make-up tests. The instructions, tests, and students responses are all online and easy
to monitor.
I have embraced this pre- and post-test approach for all my classes since Fall 2005. In the
following paragraph, I report results from my Applied Technology classes because I could
perform meaningful statistically analyses. In the other classes, with less that 30 students, I
have noticed similar trends on improvement in students' performance but they were
smaller sample sizes.
I printed the pre-test and post-test scores from SchoolFusion and analyzed them using the
Statistical Package for the Social Sciences (SPSS). The results are summarized in Fig. 2
below.
Group "V Protest Mean {%} Protest SD i V> Port-test Mean (5o; Post-test SD t-value df p- value
Entire Class 34 45.0 23.4 6S.9 20,9 4.553 33 < 0001
Caucasian M. IS 47? 24.3 69.1 21 J 3 621 17 002
Minorities 16 430 22, ? 6S 8 21.0 i.m 15 .01
Fig. 2. Summary of two-tailed, paired sample t-tests for beams, materials, and forces
Even for such small sample sizes, the two-tailed, paired sample, t-tests show that the
mean test scores increased significantly from the pre-test to the post-test, irrespective of
gender and ethnicity, across the entire class. Even with disaggregated data, by ethnicity
and minority students (Girls, Hispanics, African Americans, and American Indians), the
mean test scores increased significantly for the two groups. Clearly, this study cannot be
construed as a valid scientifically based research because there was no control group
(Slavin, 2003). However, p = .01 means that there is 1% probability that the observed
difference among minority students happened by chance.
48


To complete the Designing Beams activity, students reflected on their learning and
completed a confidential self-assessment using another grading inventory and rubric. They
also reported on team members individual contributions during the design activity and
assigned percentages. Using money, it was easy for them to express percentages,
because they were asked How would you divide $100 between the members of your team
based on each individual's contributions? I was pleasantly surprised by one team in
particular. The individual and the other two in the team reported 45%, 45%, and 10%.
Although I was moving between the five teams, I had not noticed that this one student was
doing little work.
Although contentious, recently Jonassen (2005) argued that the only legitimate goal of
education is problem solving." Students repeatedly hear that the most important concept
they will learn in my class throughout the semester is creative problem solving using a
systems approach. Students receive a blank grading inventory and rubric at the end of
each activity. A sample grading inventory and rubric, illustrating exemplary student
responses is available at
http://www.innathansworld.com/technology/SampleGradingRubriclnventoryNGradingScale.
htm The grading inventory reinforces the problem-solving process by requiring students to
reflect on their learning while completing their self-assessment.
So far, I described briefly how students learning might be assessed through multiple
measures using written tests, drawing, problem-solving, presentations, peer-assessments,
and self-assessments. The following section illustrates how students work might be
assessed though observations, questioning, research products, practical investigations,
creative writing, and bundling activities, using examples from a Grade 8 science class.
The Alignment Question
How does one ensure that objectives, instruction, and assessment are consistent with one
another?
The previous sections illustrated how an outcomes-oriented approach of identifying desired
learning goals and then working backwards to develop meaningful learning opportunities
and assessments could be used to promote meaningful learning. This backwards design
approach (Wiggins and McTighe, 1998) is one way to align assessment with the
curriculum.
With high-stakes testing, the slogan what gets taught is what is tested" is common.
Learners do not readily access numerous available online resources (like online
discussions and website references). Reeves (2002) observed that often, the learners do
not see a relationship between assessment and online resources because they are
focused instead on other activities that might help them obtain the highest scores in
traditional course assessments.
In my classes, students pay attention to classwork because it counts toward 50% of their
grade. The quizzes are 30% and homework is 20%. The homework uses questioning and
49


creative writing assessment approaches using standards-based online discussions. For
exampie, the Grade 8 science students had to provide thoughtful online responses to the
scenario illustrated in Fig. 3. This example illustrates how teachers can use information
gathered through essential questions to plan, inform, and modify instruction using feedback
from authentic assessments. Authentic assessments must be contextualized, be public,
require collaboration with others, enable students to show off what they can do, and
replicate the actual challenges that typically face a person in the field: conduct original
research, analyze the research of others, argue critically, and synthesize divergent
viewpoints (Wiggins, 1989).

f-x X.-' s'-.-T.-1 'i hi'A
\* v f?K 'I'-Vvr*. rnUx-i ',.'1 ^
Hi M'&h.-J'i-U.:; hi? JtiTiVahS.r. t >
rro . l V>& tn*li l*' ?-'r. h0-.
V ,L JiU Cit. , tr
Why dy yd <: * .-dr* wtYi- r,fi* c-f fh.-rvT
You- >n-yA Justify y< .>y vt.fh :Q:uvx.h,x /v
H 'X-J'i't 1.4* V'.-u

Created By Name* g*H*ubr*rnanior
?:an
Fig. 3. Sample standards-based online discussion
Our school nurse typically made over 85 ice-packs everyday. Students use these ice-packs
often and could engage with the dilemma presented. They had to demonstrate their
understanding of the concepts of density, melting, evaporation, condensation, and closed
systems through their thoughtful responses. Students individual responses are available in
their class website at www.angevine.groupfusion.net Here I illustrate how their thinking
evolved over time using two sample student responses. Ryan and Leahs initial responses
are illustrated in Fig. 4.
Byant. w.3t*.osctti*AK
I §<* eecause 1 cont i-<*.*>* thin; test *& in the bag ftii 6ut and k ?.£ve }usl
smaller feeraust; fYe \os all treitsr
Lceb o. its:%a 56--afe-e£
1 Ekhai sresauts if yc* ar tcs £*--*. o.;5 a? this ?****. ther, r\r,zt U-* *'! Trail
waifei, and .fc hs* the! wsier, *v lit* tee padH warms: t a-wf htayrer.
Fig. 4. Sample students initial responses
After reading their responses, I asked them to divide themselves into three groups based
on their beliefs about the ice-pack becoming lighter, staying the same, or heavier in their
second class. There were six, nine, and two students in the three groups, respectively.
They had to plan their experiments, and record their discussions and observations on
Cornell notes. I used these as the prompts for the following class. They discussed the
prompts in pairs and subsequently posted their replies (Fig. 5).
50


Ryan L.
or ic
1 + ujr. thr i'v: -hi t'js serstss rsne d **- . ater >-.*s :e--:rs tre Li; t-.s ,t ;?
;S=; ; *V4C s: 3UT - ^ vup a cietted c tU> i.e- ? fai.it, uzk a-a is-*1* n-ss j-j-t vj*i
atfrersr i from. Cu'i
F2r3* r*ory
i -Jv-f ;> 'chi* 2'cup ii* d'e.sdr-g ard l agrte *tt.'- c>n cc.iciut
i eah O --*? :n ui t;v it ; r. ,,v
Am a
1 T n C g:Ofpt -rtud r>* ifrg t?55: g frs'r
t.t-uKf t.-' tes-i mji - r- l tfe i*H*
Fig. 5. Sample students response in their third class.
Following this, we discussed the relevant concepts as a class and students then had to
post their final responses using the necessary vocabulary (Fig. 6).
By engaging constructively with the multiple perspectives, students could "confront
stereotypes and simplifications about the subject matter (Gardner, 1991, p. 244) and
demonstrate significant learning and development of their analytical and critical thinking
skills.
Leah O. Ccstac PS 06,C5 CJ.SMw
[IEbH)
All cf the measurmemi ricuid'vc baen the came, but j-clt A £ became hjhter, s'cup £ £ stayed the
same, arid group C s became heavier.
I thin* that group A a ccuiCve teccme lighter because sc-me watt* *r one cf Lags ould've
evaporated foster than ir Hit ether beg-
Group 6 6 could ve craved me tame because them bag d-1 t evaporate ii^ct group A $ die
Group C's could've getter, heavier because the bag could've beer buoyant cr filed ,v.tji ccndersabcr,.
Ryan E. Ptatec CS/i*,;'G5 Cfc 44AM
I thir>< group A got a lighter measurement bevause the bag rrnght have beer epen and some of tf e Aster
evapcr.ited cr spilled cut
1 tltrtk that group B {our group) measured the water first before arty could get cut.
I tinnk that Group C had a heavier fneaturemtnt because condensation caused mcasture to gather cr the
outside cf the bag.
Fig. 6. Sample students response in their fourth class
In conclusion: This paper described how students and teachers receive timely feedback on
students' performance in pre-tests and think-writes before engaging in formal classroom
learning. Although extremely powerful for formative evaluation and informing instruction,
the online learning management system does not support graphics during test design.
Besides, this exploratory study was not scientifically based because there was no control
group. Nevertheless, it demonstrates how all students assumed responsibility for learning
and I had evidence of individual students improved conceptual understanding.
51


Chapter 5
GAMES AND SIMULATIONS
Abstract
This overview examines the challenges and opportunities afforded by games and
simulations to enrich teaching and learning. It presents the preliminary findings
from a classroom study that used the promising educational games and
simulations developed by the Nobel Foundation. Middle school students from all
groups, disaggregated by gender and ethnicity, showed significant learning gains
after playing these challenging Nobel games. We recommend five guidelines that
are necessary for games and simulations to be meaningfully integrated into
classrooms.
Introduction
Improving schools internationally is the greatest challenge of our generation -
Clark Aldrich (2004, p. 229)
Early studies show that gamers perceive the world more clearly, are more creative
problem solvers, are more confident, and are more social Steven Johnson
(2005)
We juxtaposed these two divergent comments because games and simulations
offer tremendous promise to help us cope with the current challenges in education and
training. The current technology-sawy generation of students are cognitively more
sophisticated and want learning to be fun, engaging, hands-on, challenging, interactive,
empowering, and thought provoking. However, some educators continue to think of
knowledge and learning in terms of textbooks sequential, fact-based, and immutable.
Students varied interests and habits of inquiry conflict with traditional textbook-centered
classroom instruction, and often result in discipline issues in the classroom.
Simultaneously, the problems facing the world and the workplace are becoming
more and more complex. Employers wonder if their employees can be better prepared at
schools and universities to cope with todays unique challenges, both nationally and
globally. Could simulations and gaming environments stimulate competence, creativity and
problem-solving through active collaboration, interactions, dialogue, and shared interests
between individuals?
The purpose of this introduction is to examine the challenges and opportunities
afforded by games and simulations to enrich teaching and iearning. In the latter half of the
paper, we assume Nathan's voice as he provides a practitioners perspective on classroom
52


realities. We conclude with five guidelines that we believe are essential for educational
games and simulations to be integrated into classrooms.
Why Games and Simulations?
History and Definitions
Although the idea of using computer games to facilitate learning is being
resuscitated with new technologies and fresh thinking, a visit to the library at a local
university revealed a shelf-load of textbooks from the late 1950s until early 1970s, centered
on using games and simulations in classrooms to facilitate learning. Klietsch's (1969)
curriculum guidelines elaborate on the underlying behavioral-learning systems theory
behind games and simulations. Klietsch details various unique characteristics of behavior-
based simulations and games (Unit A, pp. 4-5), which include: goals, capabilities,
resources, means, interactions, strategy, engagement, decision-making, and problem-
solving requirements. The commercial gaming industry has capitalized on these
characteristics and continues to design games that satiate gamers interests world-wide.
The latest snapshot study by the BBC Audience Research (BBC News, 2005) in the UK
reported that 59% of the 26.5 million individuals surveyed in the age groups 6-65 are
gamers 48% of them women. They concluded that gaming is enjoyed by both genders
across all ages in all walks of life.
With numerous genres available, the word game has been elusive to define and
holds various denotations and connotations. Glazier (1973), Prensky (2001), and
Rasmusen (2001) have described the presence of the following basic components in
games: 1) Player Roles, 2) Game Rules, 3) Goals and Objectives, 4) Puzzles or Problems
(Challenges), 5) Narrative or Story, 6) Players Interactions, 7) Payoffs and Strategies, and
8) Feedback and Outcomes. We define a game as an engaging interactive learning
environment that captivates a player by offering challenges that require increasing levels of
mastery. The Laser Challenge Game
(http://nobelprize.org/physics/educational/laser/challenge.html) designed by the Nobel
Foundation exemplifies this definition.
Similarly, with wraparounds or scaffolds to advance learning outcomes, simulation-
based environments also engage students and promote learning. Aldrich (2004) defines
simulations as tools that facilitate learning through practice in a repeatable, focused
environment. Additionally, simulations are safe, flexible, resource-efficient, globally
accessible when web-based, and effective in helping students develop visual and
conceptual models. SimCity, a popular simulation, is a good example. This simulation
challenges players strategic thinking and building abilities as they cope with resource
constraints to design a harmonious city. Players can see how well their city evolves based
on the decisions they make. According to Chaplin and Ruby (2005), the designer of
SimCity, Will Wright, had deliberately left the criteria of winning and losing to the players to
make their experience personal and compelling.
53


Even though researchers are constantly trying to define and differentiate games
from simulations, there are more commonalities than differences between them. Aldrich
(2004) attenuates the distinction further by recommending that educational simulations
should incorporate applied pressure situations that tap users emotions and force them to
act (p. 9). He argues that simulations can promote full cycles of learning starting with goal,
plan, experiment, feedback, update, and understanding. In a book published a year later,
Aldrich (2005) prefers defining educational simulations as something that happens when
simulation elements, game elements, and pedagogical elements converge. Although we
are not there yet. Aldrich (2004) predicts that the development and adoption of games and
simulations will have the greatest impact on teaching and learning in schools.
Educational Strengths
When designed well, both simulations and gaming environments can facilitate
students learning of both specific domain knowledge and concepts, and several cognitive
skills like pattern recognition, decision-making and problem-solving. From their review of
literature covering a period of 28 years, Randel et al. (1992) concluded that gaming could
be used effectively to provoke interest, teach domain knowledge, and shore up retention in
math, physics, and language arts when specific instructional objectives were targeted.
Funk (2002) cites studies which found that games strengthened students engagement,
information processing, problem-solving, social development, and academic abilities. Other
educational strengths of using games and simulations include developing a variety of
cognitive objectives, transferable process skills, student-centered learning, initiative,
creative thinking, affective objectives, sense of completion, and knowledge integration
(Ellington, Gordon & Fowlie, 1998).
Additionally.
Exploratory interactive games are useful for instruction in math and science,
particularly when concepts are difficult to visualize or manipulate with concrete
materials (Mitchell & Savill-Smith, 2004).
Students dialogue and decision-making while engaged with multi-level games
provokes experimentation, discovery learning, and perseverance as science,
technology, engineering, mathematics (STEM) principles are distorted and
explored in the games (Kirriemuir, 2002).
Students develop expert behaviors such as pattern recognition, problem solving,
qualitative thinking, and principled decision-making as their individual expertise
with games increase (VanDeventer & White, 2002).
Student effectiveness increases when they are afforded opportunities to contribute
to the game design and create new games (Mitchell & Savill-Smith, 2004).
Students motivation, skills, and ability to explore, experiment and collaborate
increased by playing computer games (BECTA, 2001).
With realistic games, students not only become smarter and intellectually engaged
but also realize their desire for hard fun, delayed gratification, rewards, making
right decisions, participation, depth of understanding, challenge, and using their
pattern recognition and problem-solving skills (Johnson, 2005).
54


Both resource-deprived and resource-affluent students, make significant learning
gains after piaying weli-designed games (Herselman, 1999, cited by Mitchell &
Savill-Smith, 2004).
Students spatial abilities and cognitive development increases after playing with
simulations and games among both genders (Mitchell & Savill-Smith, 2004).
Possible Explanations
Computer games embody good principles of learning (Gee, 2003) and motivate
players by providing them with appropriate levels of challenge, curiosity, control, and
fantasy (Malone & Lepper, 1987). More specifically, what might make games and
simulations so powerful for enhancing students learning?
1. Is it gamers familiarity with the powerful visual media and gaming environments?
Kafai (1996) noted that playing video games was often students first interaction
with technology in their homes.
2. Is it gamers active engagement in structured learning environments? Rendel et al.
(1992) observed that students active participation during play could account for
their better integrated cognitive structures, retention, and subsequent transfer.
3. Is it gamers engaging experience as they interact with the different levels of
game? Swartout and van Lent (2003) highlight the interplay of the three levels:
short-term, medium-term, and long-term goals in facilitating compelling
experiences for gamers during play.
4. Is it gamers increased self-efficacy as their proficiency develops? Although
temporary, Roe and Muijs (1998, cited by Mitchell & Savill-Smith, 2004) observed
an increased sense of mastery, control and achievement in players as their
individual gaming proficiencies improved.
5. Is it gamers improved knowledge and conceptual understanding due to
meaningful computer-based dialogue? Ravenscroft and Matheson (2002, cited by
Mitchell & Savill-Smith, 2004) found that 30 minutes of collaborative learning
through dialogue games (including exploratory talk, constructive conflict, and
collaborative argumentation) produced significant improvements in students
knowledge and conceptual understanding about the physics of motion.
6. Is it gamers ongoing learning from the immediate feedback, both successes and
failures, embedded in games? According to Prensky (2001), individuals' learning
through games is primarily due to the instant feedback gamers receive during play.
Challenges
While powerful and promising, the use of games and simulations present several
challenges. Aldrich (2004) discusses 17 challenges related to games and simulations,
including cost, delivery, time constraints, evaluation, and extent of guidance in simulations.
Both the case study by Yeo et al. (2004) and our personal experiences show that
interactivity and dynamic graphics in simulations, by themselves, do not promote transfer,
reflection, or understanding. Meanwhile, finding and using engaging educational games
continue to remain a challenge. Is this because games efficacy and usefulness have been
55


suspect (Wolfe & Crookall, 1998)? Are games perceived as frivolous diversions in this era
of increased accountability (Balasubramanian, 2003)? Has educational computer games'
limited use of sound pedagogical principles and reliance on drill and practice resulted in
their being ignored in educational research (Gredler, 1996; Reiber, 1996, cited in MIT
Games-To-Teach Research Team, 2003)?
Other concerns with using computer games include: difficulty of integrating games
with traditional instruction, mismatch between level of game and students abilities cr
needs, fear of some students not participating or cooperating, and exposing teacher
vulnerabilities amidst technology-savvy students (Ellington et at., 1998). Above all,
although several studies have shown the merits of playing computer games, none has
addressed the classroom challenges of matching the games to the standards-based
curriculum, justifying its use during premium instructional time, aligning game activity with
content understanding, customizing off-the-shelf games to the learning needs of culturally
diverse populations, designing authentic open-ended learning scenarios, and furthering
humane values of acceptance, trust, and citizenship.
Using Games and Simulations in the Classroom Nathans Experience
The preceding paragraphs highlight the challenges that need to be addressed
before games and simulations can become ubiquitous in classrooms. In this section, I
reflect on my 16 years of teaching science and technology in middle and high schools
when I have used computer games and simulations.
Four Critical Questions
What should a classroom teacher look for in games and simulations? Malone
(1980) made a compelling argument organized around challenge, fantasy, and curiosity for
designing intrinsically motivating computer games. Additionally, I would examine the
content, quality, usability, and age-appropriateness of the game. I believe well designed
games are a great asset in helping students engage and explore the core concepts in a
safe learning environment, prior to formal instruction. Egenfeldt-Nielsen (2005) makes a
case for using them to introduce theory and provide some concrete experience for the
students. We made a similar case for using games and simulations as the first step in our
conceptual framework for promoting STEM learning (Balasubramanian, Wilson, & Cios,
2005). Whether it is learning about systems and models, or examining cause-and-effect
relationships, or figuring out choices and consequences, students can be quickly exposed
to the big ideas in a topic by using well-engineered simulations. For example, I use the
Circuit Construction Kit designed by the Physics Education Technology Group (PhET) at
the University of Colorado (http://www.colorado.edu/physics/phet/web-pages/simulations-
base.html) to introduce the concepts of an electrical circuit, current, voltage, and
resistance. I have students explore these concepts by posing a challenge: Can you
construct an electrical circuit to light a bulb with just one wire, one battery, and one light
bulb and not burn the battery or your fingers? Students have opportunities to do this both
online using the simulation (and not have the battery burst into flames) and hands-on with
the three objects (and not have their fingers burnt). Although students are immediately
56


engaged because they know they should be able to do it, you will be surprised by the
number of students (and adults) who find this challenging.
Where should a classroom teacher look to find useful games and simulations?
This has been my major concern because there is no place teachers can go to find the
different games and simulations available by topic or age-appropriateness. In this era of
National Digital Libraries, it would be good to have one place where teachers can access
available games and simulations resources easily. I have used the Physics 2000
simulations (http://www.colorado.edu/physics/phet/web-pages/simulations-base.htmi) while
teaching modern physics for the International Baccalaureate program at the Emirates
International School in Dubai, United Arab Emirates. I vividly recall students fascination
with this resource fortesting their ideas, for example on interference and polarization, and
learn more about 20th century science and technology. I also used the Physlets, physics
applets (http://webphysics.davidson.edu/Applets/Applets.html), which are small flexible
Java simulations designed for science education as a resource. Physlets are used by
several physics teachers around the world for classroom demonstrations, peer instruction,
and media-focused homework, and just-in-time teaching of introductory and modern
physics. The PhET website at http://phet.colorado.edu hosts over 50 sims that are
designed to increase student engagement and learning (Perkins et al., 2006) on common
physics topics such as motion; work, energy, and power; sound and waves; heat and
thermodynamics; electricity and circuits; light and radiation; quantum phenomena;
chemistry; mathematics tools; and cutting edge research.
How should a classroom teacher use games and simulations? Recently I heard a
counselor chuckle about a student who whined about the social studies class, asking,
Why should we study about dead people? Researchers have used commercial games
like SimCity and Civilization III to enrich their social studies classes. For example, Squire
(2004) used Civilization III to explore its usefulness in the classroom and found that,
although useful, it led to several contradictions because of the complexity of the game,
extended time commitments required, students having varied difficulty learning how to use
it, and different levels of students personal motivation.
The most promising educational games and simulations I know, based on prize-
winning achievements, are those designed by the Nobel Foundation
(http://nobelprize.org/games_simulations.html). Students enthusiasm for learning and
playing well-designed games is captured in their rich descriptions available at
http://www.innathansworld.com/technology/GamesNSimulations.htm Students repeatedly
used words like learning, figuring out, paying attention, scoring, thinking, decision-making,
multiple game levels, fun, challenge, interactive, strategy, hands-on, and choices in their
descriptions. In my 16-years of teaching in middle and high schools, I have not seen such
widespread enthusiasm for learning and sharing.
How should a classroom teacher evaluate the use of games and simulations? I
was keen on finding out the ability of these games to promote student learning with
minimal teacher intervention. McDonald and Hannafm (2003) noted that web-based games
promote higher-order learning outcomes because they increase meaningful dialogue.
57


Before students played the games from a list of six, I administered a 25-question pretest
electronically that provided immediate feedback (Balasubramanian, 2006) to students.
Then students played the games for about an hour and in the last five minutes of class, I
debriefed them about their experience. I gathered feedback on what they liked about these
games or games in general. I consider this a brief after action review (AAR), recommended
by Bonk and Dennen (2005).
In their next class, students took a post-test, with the same 25 questions. However,
the order of questions was different and the tests had different titles. I analyzed their
results using the Statistical Package for the Social Sciences (SPSS). The results are
tabulated in Fig. 1 below.
Group J X Entire Class f 4f< Pretest Mean iV Pretest $01%) Post-test Mean Post-test t- 1 p* SO t i value 1 value
n.: "?.$ 1M l 8.541 j .0001
Caucasian M. \ 18 51.8 19 J 80.? i6.2 nniitt7 mm.
Minorities j 22 42J 16.2 ^5,5 21." 5.S24 < .0001
Girts j 11 41.8 11.6 fl\1 12.1 1 MSS I * .0001
Fig. 1. Summary of two-tailed, paired sample t-tests after playing the Nobel games
Clearly, despite small sample sizes and minimal teacher intervention, the two-tailed, paired
sample, t-tests show that the mean test scores increased significantly from pretest to post-
test for the entire class, even with disaggregated data by gender, ethnicity, and minority
students (Girls, Hispanics, African Americans, and American Indians) classification.
Clearly, my study is not a valid scientifically based research because there is no control
group. However, with larger sample sizes, I could have examined whether groups of
students with after action review (AAR) did better than those without AAR.
Games and Simulations are not Teacher-Proof
Evidently, designing wraparounds can be challenging. The computer games in
education project (BECTA, 2001) concluded that although the benefits of using games was
clear, a teacher s role in structuring and framing activities around games was critical. In the
case of games and simulations designed by the Nobel foundation, it was easier for me to
personally justify their use in the classroom and design quizzes to find out what students
were learning. Although the games designed by the Environmental Protection Agency,
(http://www.epa.gov/OGWDW/kids/gamesandactivies.html) directly related to water and
the filter design activities at that time, students were quick to point out that they liked the
Nobel games better. The Nobel games were useful, exciting, fun, active, challenging,
engaging, interactive, interesting, hard, and designed with very good graphics, the
students wrote.
Contrary to Schank (Green, 2000), who claims that interactive software would
make teachers redundant, I would argue that even with well-designed games, a teachers
role in facilitating a meaningful learning environment will remain pivotal. I would concede
though that a teachers subject expertise, understanding of pedagogy, comfort level using
58


technology, and easy access to technology would contribute significantly to games and
simulations becoming used more often in the classroom.
Recommendations
The findings of numerous researchers in this article illustrate that well designed
games and simulations can prepare our students to learn critical problem-solving and
decision-making skills necessary for the real world. Student endorsements that the Nobel
games and simulations actually teaches you about the subject, uses harder questions and
better graphics, along with results from their pretests and post-tests showing significant
gains, illustrated how students are not averse to learning in the classroom. Further studies
might explore what makes these Nobel games and simulations interesting.
Evidently, games are firmly entrenched among youth and adults alike, as the
recent BBC Audience Research study reported. When designed well, games can truly be
an important teaching tool (Shreve, 2005). They promote numerous cognitive benefits in
learners, including a facilitation of increased interactions, motivation for learning,
visualization, experimentation, self-efficacy, self-monitoring, pattern recognition, problem-
solving and critical thinking abilities that we want all our students to graduate with from
our schools.
Yet, several educators continue to view the use of games and simulations in the
classroom with apprehension. If games and simulations are to be meaningfully integrated
into classrooms, the following five guidelines should inform the design of educational
games in the future.
1. The design of games and simulations should be sophisticated and challenging
enough for students to be cognitively engaged with the game.
2. The content of games and simulations should be aligned with the standards and
viable curriculum in schools.
3. The logistics and usability of the games should reflect classroom realities and time
constraints in schools.
4. The feedback and assessments embedded in the games should embody
measurable learning outcomes.
5. The teacher guides accompanying the games should provide sufficient ideas,
activities and resources to enhance students learning.
The papers that follow provide more examples of how games and simulations might be
used to enhance learning in classrooms.
59


Chapter 6
NURTURING TEACHER EXCELLENCE USING THE
LEARN BY DESIGN MODEL (LBDM)
How might schools with large populations of students from low-income, migrant,
and international families ensure that every student reaches proficiency on challenging
State academic achievement standards and state academic assessments (NCLB, p.
1439)? Traditionally, schools across the country have tended to cope with this challenge
by offering a narrower range of curriculum and focused oniy on improving students' low
level literacy skills (Bransford et al., 2000) reading, writing and mathematics because
currently only these results get reported in the Federal School Accountability Report (CDE,
2006).
In this article, we share preliminary results from our systemic school-wide
curriculum reform effort using an evidence-based instructional model to cope with
challenges at Overland High School (Overland) located in the Cherry Creek School District
in suburban Denver. Overland is a comprehensive public, suburban, college-oriented high
school with a total enrollment of 2,153 students in the 2007-2008 school-year. Over 2000
students have been enrolled in the school each year since 2000-2001. Over the past 6
years, the school has undergone major demographic changes. Specifically, the student
community has changed from a predominately Caucasian, middle-class to an international,
minority-majority school. Students at Overland represent over 60 countries and speak over
54 different languages. The student community includes a diverse population from
different social, economic, ethnic, and racial backgrounds, with 37.1% identifying
themselves as African-American, 33.7% Caucasian, 22.1% Hispanic, 6.3% Asian, and
0.8% American-lndian. In addition to the ethnic diversity, the school now has a poverty rate
of 41 %. While college preparedness and academic excellence are hallmarks of the school
district, student performance at Overland has continued to decline over the past six years
when compared with the state and district performance (Exhibit 1).
Overlands campus also includes a feeder middle school, Prairie. The
demographics of the two schools are similar as both schools have students from the
community that surrounds our campus. Given these overall trends, the achievement of the
2000 cohort (Exhibit 2) and 2001 cohort (Exhibit 3) of students, during their five years as
they move from middle to high school, shows a stagnant and declining trend across the
two schools.
DiMartino, Clarke, and Lachat (2002) have written about the futility of making
students learn factual knowledge where students are merely listening to lectures, waiting,
taking tests, and doing seat work (p. 45). These students learn fewer life-skills because of
limited intentional opportunities to develop their higher-level literacy skills (HLS).
Consequently, they will continue to perform poorly on state assessments that test students
HLS. This leads to lower faculty, student and parent morale. Yet, research on how students
learn has shown us that using an explicit PLAN TEACH > MONITOR ADJUST
instructional model with rigorous curricula that provide opportunities for teachers to learn
60


effective instructional strategies, have structures in place for their mentoring, use active
monitoring and have accountability measures in place, can raise the achievement of every
student (Baiasubramanian, Wilson & Cios, 2006; CCSD, 2007; Grier, 2002).
Learn by Design Model (LBDM)
In this section, we describe our conceptual framework for developing and nurturing
teacher excellence to increase student achievement. The Leam by Design Model (LBDM)
is an evidence-based instructional intervention that is grounded in cognitive and
neuroscience theories on learning and motivation (Bransford et al., 2000; Goleman, 2006).
The model has two components (Exhibit 4). First, this model operationalizes Wiggings and
McTighes (2005) backwards design by using an embodied theory a specific template of
activities and protocols to align curriculum, assessment and instruction to promote
student-centered learning.
Second, to develop students' higher-level literacy skills (HLS), this model
operationalizes five HLS critical thinking, problem-solving, mathematical reasoning,
inference-making and visualization/modeling (see examples across the four core subjects
at http://www.doers.us/HLS_Defined.pdf) so teachers can explicitly plan, teach and
monitor student learning of these essential life-skills.
In order to create a Professional Learning Community (PLC) for this model, we
asked for teacher-volunteers who would teach the freshman class during the 2007-2008
school-year. Two administrators and two faculty members were the lead-trainers for this
PLC. The lead-trainers then worked with these teachers to provide them with 25 hours of
face-to-face professional development (available on 5 DVDs). 13 teacher-leaders across
four subject areas English, mathematics, science and social studies were trained on
the LBDM in summer 2007. To demonstrate their understanding of LBDM, these teachers
were asked to develop curriculum plans for the first quarter of the 2007-2008 school-year
and submit them to the Principal. These curriculum plans were then graded independently
by the lead-trainers using a 100-point grading rubric. The mean was 83% and the Kuder-
Richardson 20 coefficient was 0.9752, showing the close agreement between graders. The
pretest(47%)-post-test(73%) gains with a Pearson's r correlation of .799 showed that 89%
of the variance in the post-test scores could be accounted from the LBD training. Further
analyses of these summary results are available at http://doers.us/LBD_FAQs.html The 13
teacher-leaders started implementing their curriculum plans in August 2007 and now teach
over 670 students (approximately two-thirds of the combined freshman-sophomore classes
at Overland). We plan on continuing with the implementation of LBDM with the freshman
class this year and then scale the project to include the other three grade levels by adding
one grade level each year over a four year period.
Unique Features of LBDM
Although the implementation of LBDM is in its early stages, we want to share the
unique features and results of our systemic curriculum reform initiative at Overland
because it could help increase student achievement in other schools. As we describe these
61


features, we quote extensively from teacher-reflections at our training sessions to illustrate
how we developed teacher excellence as they became co-creators of knowledge with us
and emerging as teacher-leaders.
Emphasis on Writing
Even before teachers formally instruct their students, LBDM requires students to think and
respond to real-world scenarios. This writing activity not only challenges students but it
also gives them an opportunity to demonstrate their learning of standards-based content in
their own words. After acknowledging the importance of acknowledging and deflating
student misconceptions, one teacher wrote that these prewrites identify incomplete
understanding, false beliefs and naive rendition of concepts, prior to formal instruction.
While the prewrites give students a purpose for learning, at the end-of-a-unit when
students are given a similar but different scenario for their post-write (see Overland Unit
Planner exemplar for an example http://www.doers.us/Sample_Unit_Planner.html),
teachers know how well students can generalize and transfer their learning. Besides, the
explicit focus on writing prepares our students for college (Conley, 2005) as they
demonstrate their communication, reasoning, personal interaction, and quantitative
thinking skills (p. 135).
Growth Model
While all our teacher-leaders acknowledged the importance of pretests to show
measurable student growth and progress in their reflections, they articulated other benefits
for students including: helping students know where they are and where they need to be
by the end of the unit or chapter, could help with increasing self-esteem, be more self-
guided, be more accountable for their learning targets, and help motivate the students.
Teachers also said the pretests are a tool that is integral in differentiating the instruction
for the unit, the pre-test data can help drive my instruction for the unit by utilizing existing
student strengths and weaknesses, and assess student learning and the effectiveness of
teaching methodology." These reflections are consistent with essential learning goals that
are personal and relevant to students as described in the third core area of Breaking
Ranks II (NASSP, 2004).
Intentionally
Hands-on guided-inquiry learning, as valuable as it is, must be connected to the
established content in the standards. The revised two-dimensional Bloom's Taxonomy
(Anderson & Krathwohl, 2001) is useful to plan and organize the cognitive elements of
instruction so students could easily see the transition from simple to complex levels of
thinking. Reflecting on the purpose of planning their learning outcomes using the revised
taxonomy, teachers wrote: The 2D Bloom's Taxonomy forces you to decide the type of
knowing that your students are doing in addition to their level of thinking, Students will
learn if they know what it is that they are expected to learn, This is important because
students should not be able to only draw on factual knowledge. They should have to draw
upon other types of knowledge, To make sure that the frameworks are being addressed,
62


but also, that you are designing this for more complex levels of Bloom's taxonomy for your
test and unit design, To make sure that you are asking the students to think and know
the content in multiple dimensions, some at the lower levels and some at the higher levels.
It will help ensure that I get to HLS and use multiple assessment formats," This allows me
to more effectively analyze the different levels of thinking going on in my classroom.
Developing this epistemological understanding, helping students understand the value of
creative and higher-order thinking, is a valuable learning outcome in its own right
(Balasubramanian & Wilson, 2007).
Value-added
While it is important to have students know and be able to do things, it is critical that our
students value learning and its connection to the real-world. The initial inquiry scenario
through simulations and/or hands-on activities is designed to engage and motivate
students as they begin their formal study. Reflecting on its importance, teacher-volunteers
wrote: It grounds the facts and skills in a real-world application that allows students to see
the necessity of what they are learning. It also sets up the direct instruction that might
follow, Students may discover something new on their own Also, the inquiry scenario
may help students develop their own questions they want to explore further, Teenagers
are naturally competitive, we should use this desire in our favor, this will increase
engagement throughout the course, and students gain more ownership of the content.
Any step that helps students to THINK is a valuable tool.
Formative Assessment
This is a significant part of the embodied theory and the driving question: what evidence
will you accept that students value, know and are able to do in your class led one teacher
to reflect on this new understanding because it now has opened up the whole idea of
assessments as learning tools. Others said: It will help me to create better assessments
and to use my assessment scores to adapt my teaching, and I now understand the
difference between "assessment for learning" and "assessment as learning".
Backwards Design
An emphasis on targeted and intentional teaching of curriculum is meaningless without well
designed diagnostic, formative and summative assessments. Our teacher-volunteers
received extensive training in aligning assessments with instruction and how to use the
assessment resuits. Initially, they were asked to bring in a current assessment for a unit.
They then looked at these assessments in many ways. First, they examined the amount of
time spent on each topic within each unit and how this correlated with their assessment.
We also asked them to look at the quality of their tests. Incorporating higher-level literacy
skills (HLS) into instruction is a fundamental component in LBDM. If HLS is taught, they
should be assessed as well. Simple comparative matrices showed that most tests that
teachers brought were written at the factual and recall level After instruction on how to
write quality assessments, the items on diagnostic, formative and summative assessments
changed dramatically. With this paradigm shift, teachers then spent a considerable
63


amount of time rewriting exams, aligning instruction and incorporating feasible higher-leve!
thinking questions for the schools common assessments. Finally, the teacher-leaders used
simple item analysis rubrics to align test items with their instruction. A good understanding
of backwards design is foundational to the best practices discussion that teachers must
have as they continue to improve classroom instruction based on student performance in
the common assessments.
Metacognition
Throughout the LBD Model, both teachers and students use an the iterative metacognitive
cycle STOP -* REFLECT THINK ACT to actively promote teaching for transfer,
where students use the knowledge gained in one subject to apply it to not only that subject
but also to other subject-disciplines. One teacher summarized: The teacher must have
clear goals as to WHAT and HOW the kids are going to learn. It is important to think of
assessment as three dimensional and ongoing. Assessment is for the students too.
Students need to learn how to assess themselves and how to grow in their own learning.
This is the metacognitive piece that is essential to the LBD Model. Another wrote: As you
move towards LBD instruction, you are creating a learner-centered classroom that is
positive, engaging, and one in which students receive feedback everyday in different
forms. It also allows teachers to assess in many different ways.
Challenges and Next Steps for Measuring Effectiveness
It takes a huge amount of resources to implement the explicit PLAN - TEACH -
MONITOR -> ADJUST instructional model. Teacher training, curriculum development and
reflection take time and require financial and human resource support from the school and
the district. It has been a challenge helping teachers move away from a teacher-centered
to more student-centered learning in their classrooms. Although our inter-grader reliability
was very high, to make sure the lead-trainers knew what they were looking for as they
evaluated teachers' curriculum plans took time. Keeping up with all the communication and
follow-up required, amidst the lead-trainers normal work schedules has been difficult. The
huge expectations, including reporting pretest data and monitoring progress on student
learning every three weeks, although valuable, is very time intensive. As we look forward to
our next steps, we want to analyze these pretest results and share it with our emerging
teacher-leaders. We want to see how these pretest results and classroom instruction
impact student performance and how they correlate with our state academic assessments.
Using these results we would modify not only teaching but also the implementation of
LBDM. Additionally, we would like to include more faculty from the school across the
freshman and sophomore classes. We will continue collecting data and use it to evaluate
instructional effectiveness. Articulating with the feeder middle school is one of our next
goals. Despite all these challenges, we gain strength from the preliminary results of our
faculty training. The commitment, ownership and enthusiasm of these early-adopter
teacher-leaders in implementing this school-wide systemic intervention is inspiring.
64


% Proficient & Advanced
SUPPLEMENTARY MATERIAL
Summary Results compiled from Federal School Accountability Reports.
Prairie & Overland Students' Performance on CSAP
2000-2006 Compared with State & District Data
RWMRWM RWMRWM RWMRWMRWMRWM RWM RWM RWMRWM
COO- ( 00 ( 00- COO- ( 00- C00- ( 01 C0t- ( 01 ( 01- ('0V ( 01- ( 02- ( 02- {'02- ( 02- ( 02 ( 02- ( 03- ( 03- ( 03- ( 03- ( 03 ( 03- ( 04- ('04- CM- ( 04- ('04- ('04- ( 05 ( 05- ('05- ( 05- ( 05- ( 05
01) 01) 01) 01) 01) 01) 02) 02) 02) 02) 02) 02) 03) '03) 03) 03) 03) 03) 04) 04) '04) 04) 04) '04) 05) '05) 05) 05) 05) 05) Ofl) 08) 08) 08) 05) 03)
6-10 Grades Level Performance in Reading, Waiting & Mathematics
[ Prairie/Overland State------District ]
Exhibit 1: Student performance in reading, writing and mathematics since 2000


O)
O)
2000 Cohort's Performance on CSAP
*o
o
o
c
(TJ
>
x
<
00
c
o
s
from Prairie through Overland
* Reading
-* Writing
j---Math
6th 7th 8th 9th 10th
(00- (01- (02- C03- ('04-
01) '02) 03) '04) '05)
Grade Level
Exhibit 2: 2000 Cohort of Students' Progress over time
2001 Cohorts Performance on CSAP
from Prairie through Overland
u
0)
6th 7th 8th 9th 10th
('01- ('02- ('03- ('04- ('05-
'02) 03) 04) '05) 06)
Grade Level
Exhibit 3: 2001 Cohort of Students' Progress over time
Reading
Writing
|--Math J


o>
-J
STOP
Learn by Design Model (LBDM) Focuses on TWO components as
chi
Teachers and Students become Co-Creators of Knowledge
First component
Embodied Theory (aka 7-step Model)
Second component
Higher-Level Literacy Skills
REFLECT
Metacognition
An Iterative
Metacognitive Cycle
Used by BOTH
Teacher & Student
throughout the
LBD Model
Student
Achievement
THINK
1. Critical Thinking
2. Problem Solving
3. Mathematical Reasoning
4. Inference-Making
5. Visualization/Modeling
Diagnostic
Assessment
Assessments for Learning
[^Cognition
Formative
Assessment
Assessments as Learning
Motivation Cognition
Summatne
Assessment
Assessments ot Leamirvg
Motivation Cognition MotfaaSen t
2007 Nathan Balasubramanhn
Exhibit 4: Learn by Design Model (LBDM)


BIBLIOGRAPHY
Aldrich, C. (2004). Simulations and the future of learning: An innovative (and perhaps
revolutionary) approach to e-learning. San Francisco: Pfeiffer.
Aldrich, C. (2005). Learning by doing: A comprehensive guide to simulations, computer
games, and pedagogy in e-learning and other educational experiences. San
Francisco: Pfeiffer.
Allen, P., Strathern, M., & Baldwin, J. (2007). Complexity and the limits to learning. Joumai
of Evolutionary Economics, 17(4), 401-431.
American Association for the Advancement of Science. (AAAS). (1993). Benchmarks for
Science Literacy. New York: Oxford University Press.
American Association for the Advancement of Science. (2001). Atlas for Science Literacy.
Washington, DC: AAAS and the National Science Teachers Association.
American Association for the Advancement of Science (AAAS). (2001). Designs for
science literacy. New York: Oxford University Press.
Anderson, L. W., & Krathwohl, D. R. (Eds.). (2001). A taxonomy for learning, teaching, and
assessing: A revision of Blooms taxonomy of educational objectives. New York:
Longman.
Avgerou, C., & McGrath, K. (2007). Power, rationality, and the art of living through socio-
technical change. MIS Quarterly, 31(2), 295-315.
Balasubramanian, N. (2002). Student perceptions on career education: a case study in an
international secondary school. Unpublished master's thesis, University of
Sheffield, Sheffield, United Kingdom. Retrieved October 23, 2007, from
http://www.innathansworld.com/aboutme/thesispdf.pdf
Balasubramanian, N. (2003, June 2-9). Smart education: Blending subject expertise with
the concept of career development for effective classroom management. Retrieved
October 23, 2007, from University of Georgia, Instructional Technology Forum
(ITFORUM) Web site: http://itech1.coe.uga.edu/itforum/paper73/paper73.hfml
Balasubramanian, N. (2003). Colorado MESA Evaluation Aggregate Results from 72
Schools. Retrieved October 23, 2007, from
http://www.innathansworld.com/technology/MESA%20Evaluation.pdf
Balasubramanian, N. (2004). Response from Nathan Balasubramanian on 12/26/2004 in
response to U. S. Math and Science Skills: Improving or Not? Retrieved October
23, 2007, from http://www.edweek.org/tb/2004/12/14/82.htmi
68


Balasubramanian, N. (2005a). Where I should learn: Interactive worksheet. Retrieved
October 23, 2007, from
http://doers.us/laser_challenge/PDF_Laser_Challenge/learnwhere.pdf
Balasubramanian, N. (2005b). Learning outcomes for the filter project. Retrieved October
23, 2007, from
http://www.innathansworid.com/technoiogy/NathansRevisedBloomQuestionsForFi!
ters.html
Balasubramanian, N. (2005c). Recording students' monetary monitoring in Mr. Balas
classes. Retrieved October 23, 2007, from
http://doers.us/laser_challenge/PDF_Laser_Challenge/microeconomy_data_coliect
ion_student_version.pdf
Balasubramanian, N. (2005d). Nathan Balasubramanians response to Assignment 17.
Retrieved October 23, 2007, from Design, Learning, and Collaboration SWIKI at
the University of Colorado at Boulder httD://webguide.cs.Colorado.edu:3232/dlc-
2005/339
Balasubramanian, N. (2005e). # 12. Designing beams. Retrieved October 23, 2007, from
Movies from the Classroom at
http://www.innathansworld.com/technology/Macromedia%20Flash%20Movies.htm
Balasubramanian, N., Wilson, B. G., & Cios, K. J. (2005). Innovative methods of teaching
and learning science and engineering in middle schools. In F. Malpica, F. Welsch,
A. Tremante, & J. Lawler (Eds.), The 3rd International Conference on Education
and Information Systems: Technologies and Applications: Vol. 1. Proceedings,
174-178, Orlando, FL. July 14- 17, 2005. Retrieved October 23, 2007, from
http://doers.us/EISTA.pdf
Balasubramanian, N. (2006a). Increasing Student Achievement Through Meaningful,
Authentic Assessment. In C. Crawford et al. (Eds.), Proceedings of Society for
Information Technology and Teacher Education International Conference 2006
(pp. 3-8). Chesapeake, VA: AACE. Retrieved October 23, 2007, from
http://www.innathansworld.com/KART/MeaningfulAuthenticAssessment.pdf
Balasubramanian, N. (2006b). The filter project. Retrieved October 23, 2007, from
http://doers.us/filter_project_lesson_plan.html
Balasubramanian, N. (2006c). Movies from the classroom. Retrieved October 23, 2007,
from http://www.innathansworld.com/technology/macromedia_flash_movies.html
Balasubramanian, N. (2006d). Water filtration vocabulary. Retrieved October 23, 2007,
from http://doers.us/filter_vocabulary.htm
Balasubramanian, N. (2006e). Water filtration study: Students perspective. Retrieved
69


October 23, 2007, from http://doers.us/water_filtration_students_perspective.html
BaSasubramanian, N. (2006f). Online resources for the filter project. Retrieved October 23,
2007, from http://doers.us/more_data_for_filter_project.html
Balasubramanian, N. (2006g). Wet sponge and dry sponge inquiry. Retrieved October 23,
2007, from http://doers.us/sponge_inquiry.pdf
Balasubramanian, N., & Muth, R. (2006). Simon, Herbert Alexander (1916-2001). In F. W.
English. (Ed.), Encyclopedia of Educational Leadership and Administration,
Thousand Oaks, CA: Sage, pp. 930-931.
Balasubramanian, N. & Wilson, B. G. (2006). Games and Simulations, In C. Crawford et al.
(Eds.), ForeSITE (http://site.aace.org/pubs/foresite/), Volume One, 2005,
Proceedings of Society for Information Technology and Teacher Education
International Conference 2006. Chesapeake, VA: AACE (p. 2).
Balasubramanian, N., Wilson, B. G., & Cios, K. J. (2006). Innovative methods of teaching
science and engineering in secondary schools. Journal of Systemics, Informatics
and Cybernetics, 4(6), 41-46. Retrieved October 23, 2007, from
http://www.iiisci.org/Journal/SCI/Past.asp
Balasubramanian, N. & Wilson, B. G. (2007). Learning by design: Teachers and students
as co-creators of knowledge. In K. Kumpulainen (Ed.). Educational Technology:
Opportunities and Challenges, Oulu, Finland: University of Oulu, 30-51. Retrieved
October 23, 2007, from
http://herkules.oulu.fi/isbn9789514284069/isbn9789514284069.pdf
BBC News (2005). At-a-glance: State of play. Retrieved October 23, 2007, from
http://news.bbc.co.Uk/1/shared/spl/hi/sci_nat/05/state_of_play/html/1.stm
BECTA. (2001). Computer games in education project. British Educational
Communications and Technology Agency (BECTA). Retrieved October 23, 2007,
from http://www.becta.org.uk/page_documents/research/cge/report.pdf
Bellamy, R. K. E. (1996). Designing educational technology: Computer-mediated change.
In B. A. Nardi (Ed.). Context and consciousness: Activity theory and human-
computer interaction. Cambridge, MA: The MIT Press.
Bonk, C. J., & Dennen, V. P. (2005). Massive multiplayer online gaming: A research
framework for military education and training. Technical Report 2005-1 of
Advanced Distributed Learning. Retrieved October 23, 2007, from
http://www.adlnet.org/downloads/189.cfm
Borges, A. T., & Gilbert, J. K. (1999). Mental models of electricity. International Journal of
Science Education, 21{ 1), 95-117.
70


Bransford, J. D., & Schwartz, D. L. (1999). Rethinking Transfer: A Simple Proposal With
Multiple implications. In A. Iran-Nejad & P. D. Pearson (Eds.). Review of Research
in Education, 24, Washington, DC: American Educational Research Association
(AERA). 61-100.
Bransford, J. D., Brown, A. L., Cocking, R. R., Donovan, M. S., Bransford, J. D., &
Pellegrino, J. W. (2000). How People Learn: Brain, Mind, Experience, and School
(Expanded Ed ). Washington, D.C.: National Academy Press.
Bransford, J. D., & M. S. Donovan, M. S. (2005). Scientific inquiry and How People Learn,
In M. S. Donovan, & J. D. Bransford (Eds.), How Students Learn: History,
Mathematics, and Science in the Classroom, Washington, D.C.: National Academy
Press.
Bybee, R. W., Buchwald, C. E., Crissman, S., Heil, D. R., Kuerbis, P. J., Matsumoto, C., &
Mclnemey, J. D. (1989). Science and Technology Education for the Elementary
Years: Frameworks for Curriculum and Instruction. Washington, DC: National
Center for Improving Science Education.
Carey, S. (1999). Sources of conceptual change. In E. K. Scholnick, K. Nelson, S. A.
Gelman, & P. H. Miller (Eds.), Conceptual development: Piaget's legacy. Mahwah,
NJ: Lawrence Erlbaum.
Cios, K. J., Pedrycz, W., & Swiniarski, R. W. (1998). Data Mining Methods for Knowledge
Discovery. Boston: Kluwer Academic Publishers.
Champagne, A. B., Gunstone, R. F. & Klopfer, L. E. (1985). Cognitive structure and
conceptual change. New York: Academic Press.
Chaplin, H., & Ruby, A. (2005). Smartbombs: The quest for art, entertainment, and big
bucks in the videogame revolution. Chapel Hill, NC: Algonquin Books.
Cherry Creek School District (CCSD). (2007). Cherry Creek's Guaranteed and Viable
Curriculum. Retrieved October 23, 2007, from
http://www.ccsd.k12.co.us/gvc/index.html
Chevalier, R. (2003). Updating the behavior engineering model. Performance
Improvement, 42(5), 8-14.
Cobb, P., Confrey, J., diSessa, A., Lehrer, R., & Schauble, L. (2003). Design Experiments
in Educational Research. Educational Researcher, 32(1), 9-13.
Cognition and Technology Group (CTG) at Vanderbilt. (1992). The Jasper series as an
example of anchored instruction: Theory, program description and assessment
data. Educational Psychologist, 27(3), 291-315.
71


Colorado Department of Education (CDE), (2005). School Accountability Reports.
Retrieved October 23, 2007, from
http://reportcard.cde.state.co.us/reportcard/CommandHandler.jsp
Colorado Department of Education (CDE). (2007). Colorado's Mode! Content Standards for
Grades K-12. Retrieved October 23, 2007, from
http://www.cde.state.co.us/cdeassess/documents/olr/k12_standards.html
Committee on Science, Engineering, and Public Policy (COSEPUP). (2007). Rising above
the gathering storm: Energizing and employing America for a brighter economic
future. Washington, D.C.: National Academies Press.
Conley, D. T. (2005). College knowledge: What it really takes for students to succeed and
what we can do to get them ready. San Francisco: Jossey-Bass.
Cozzens, M. B. (1997). Foundations: The Challenge and Promise ofK-8 Science
Education Reform. Arlington, VA: Division of Elementary, Secondary, and Informal
Education. National Science Foundation.
Covey, S. R. (2004). The 8th habit: From effectiveness to greatness. New York: Free
Press.
Davis, J., & Farbman, D. (2004). Rethinking time: The next frontier in education reform.
Education Week, 24(14), 52 & 40.
DiMartino, J., Clarke, J., & Lachat, M. A. (2002). Creating student-centered high schools.
Principal Leadership, 2(5), 44-50.
Douglas, J., Iversen, E., & Kalyandurg, C. (2004). Engineering in the K-12 classroom: An
analysis of current practices & guidelines for the future. Washington, DC: American
Society for Engineering Education.
Downey, C. J., Steffy, B. E., English, F. W., Frase, L. E., & Poston, W. K. (2004). The
three-minute classroom walk-through: Changing school supervisory practice one
teacher at a time. Thousand Oaks, CA: Corwin Press.
Driscoll, M. P. (2005). Psychology of learning for instruction (3rd ed.). Boston: Allyn &
Bacon.
Echevarria, J., Vogt, M., & Short, D. J. (2000). Making content comprehensible for English
language learners: The SIOP Model. Boston: Allyn and Bacon.
Egenfeldt-Nielsen, S. (2005). Beyond edutainment: Exploring the educational potential of
computer games. Unpublished doctoral dissertation. University of Copenhagen,
Denmark. Retrieved October 23, 2007, from http://www.it-
c.dk/people/sen/egenfeldt.pdf
72


Ellington, H., Gordon, M., & Fowlie, J. (1998). Using games & simulations in the classroom.
London: Kogan Page.
Etheredge, S. & Rudnitsky, A. (2003). Introducing Students to Scientific Inquiry: How do we
Know What we Know. Boston: Allyn and Bacon, 2003.
Fischer, G., Giaccardi, E., Eden, H., Sugimoto, M., & Ye, Y. (2005). Beyond Binary
Choices: Integrating individual and Social Creativity. Computer support for
creativity (Special issue). International Journal of Human-Computer Studies, 63(4-
5), 482-512.
Funk, J. B. (2002). Electronic Games. In V. C. Strasburger and B. J. Wilson. Children,
adolescents, & the media. Thousand Oaks, CA: Sage Publications. 117-144.
Gardner, H. (1991). The unschooled mind: How children think and how schools should
teach. New York: Basic Books.
Gee, J. P. (2003). What video games have to teach us about learning and literacy. New
York: Palgrave Macmillan.
Giest, H., & Lompscher, J. (2003). Formation of Learning Activity and Theoretical Thinking
in Science Teaching. In A. Kozulin, B. Gindis, V. S. Ageyev, & S. M. Miller (Eds.),
Vygotskys Educational Theory in Cultural Contexts. New York: Cambridge
University Press. 267-288.
Gilbert, T. F. (1978). Human competence: Engineering worthy performance. New York:
McGraw-Hill.
Glazier, R. (1973). How to design educational games (4th ed.). Cambridge, MA: ABT
Associates.
Godfrey, N. S., Edwards, C., & Richards, T. (2006). Money doesn't grow on trees: A
parent's guide to raising financially responsible children. New York: Simon &
Schuster.
Goleman, D. (2006). Social intelligence: The new science of human relationships. New
York: Bantam Books.
Gredler, M. E. (1996). Educational games and simulations: A technology in search of a
(research) paradigm. In D. H. Jonassen, (ed ), Handbook of Research on
Educational Communications and Technology. New York: Macmillan. 521-539.
Green, J. (2000, August 10). No lectures or teachers, just software. New York Times.
Retrieved October 23, 2007, from
http://www.nytimes.com/iibrary/tech/00/08/circuits/articles/10prof.html
73


Grier, T. B. (2002). Advanced placement: Access to excellence. Principal Leadership, 2(8),
16-19.
Hake, R. (2007). Should we measure change? Yes. In R. Hake. Evaluation of Teaching
and Student Learning in Higher Education. Retrieved October 23, 2007, from
http://www.physics.indiana.edu/~hake
Heavner, B., Yakachi, C., Zarske, M. S., & Carlson, D. (2004). TE activity: Breaking
beams. Teach Engineering: Resources for K-12. Retrieved October 23, 2007, from
http://www.teachengineering.eom/view_activity.php7urNhttp://www.teachengineeri
ng.com/collection/cub_/activities/cub_mechanics/cub_mechanics_lesson07_activit
yl.xml
International Baccalaureate Organization. (2000). Middle Years Programme Technology,
Book One. Geneva, Switzerland: International Baccalaureate Organization.
James, K. T., Mann, J., & Greasy, J. (2007). Leaders as lead learners: A case example of
facilitating collaborative school leadership learning for school leaders.
Management Learning, 38(1), 79-94.
Jensen, E. (1998). Teaching with the Brain in Mind. Alexandria, VA: Association for
Supervision and Curriculum Development.
Johnson, S. (2005). Everything bad is good for you. New York: Riverhead Books.
Jonassen, D. H. (2005, September 19). Let us learn to solve problems. Retrieved October
23, 2007, from University of Georgia, Instructional Technology Forum (ITFORUM)
Web site: http://it.coe.uga.edu/itforum/paper83/paper83.html
Kafai, Y. B. (1996). Software by kids for kids. Communications of the ACM, 39(4), 38-39
Kirriemuir, J. (2002). The relevance of video games and gaming consoles to the higher and
further education learning experience. In Techwatch report TSW 02-01of The Joint
Information Systems Committee, Retrieved October 23, 2007, from
http://www.jisc.ac.uk/uploaded_documents/tsw_02-01.rtf
Klietsch, R. G. (1969). An introduction to learning games & instructional simulations: A
curriculum guide. Newport, MN: Instructional Simulations.
Kuutti, K. (1996). Activity theory as a potential framework for human-computer interaction
research. In B. A. Nardi (Ed.). Context and consciousness: Activity theory and
human-computer interaction. Cambridge, MA: The MIT Press.
Linn, M. C., Shear, I., Bell, P., & Slotta, J. D. (1999). Organizing principles for science
74


education partnerships: Case studies of students learning about rats in space"
and deformed frogs. Educational Technology Research and Development, 47(2)
61-85.
Lompscher, J. (1999). Learning activity and its formation: Ascending from the abstract to
the concrete. In M. Hedegaard, & J. Lompscher (Eds.). Learning activity and
development. Oxford: Aarhus University Press.
Lyman, R. (2006, June 22). In School's Waning Days, a Focus on the Focus: June Tempts
With Distractions, but There Is Still Learning to Be Done. The New York Times, p.
B1.
Malone, T. W. (1980). What makes things fun to learn? A study of intrinsically motivating
computer games. In Cognitive and Instructional Sciences Series CIS-7. Palo Alto,
CA: Xerox Palo Alto Research Center.
Malone, T. W., and Lepper, M. R. (1987). Making learning fun: A taxonomy of intrinsic
motivations for learning. In R. E. Snow & M. J. Farr (Eds.), Aptitude, learning, and
instruction: Cognitive and affective process analyses: Vol. 3. Hillside, NJ:
Lawrence Erlbaum Associates. 223-254.
McDonald, K. K., & Hannafin, R. D. (2003). Using web-based computer games to meet the
demands of todays high-stakes testing: A mixed-methods inquiry. Journal of
Research on Technology in Education, 35(4), 459-472.
Merrill, M. D. (2002). First principles of instruction. Educational Technology Research and
Development, 50(3), 43-59.
MIT Games-to-Teach Research Team. (2003). Design principles of next-generation digital
gaming for education. Educational Technology, 43(5), 17-23.
Mitchell, A., & Savill-Smith, C. (2004). The Use of Computer and Video Games for
Learning: A Review of Literature. London: Learning and Skills Development
Agency.
NASSP. (2004). Breaking Ranks II. Reston, VA: National Association of Secondary School
Principals.
National Research Council. (1996). National Science Education Standards, Washington,
D.C.: National Academy Press.
Newsleader (NL). (2005, January). PISA results support need for high school reform.
Newsleader, 52(5), 1,10
Nobel Foundation. (2005). Laser Challenge Game, Retrieved October 23, 2007, from
http://nobelprize.org/physics/educationaI/laser/challenge.html
75


No Child Left Behind Act of 2001, (2002). Public Law 107-110, 107th Congress, Retrieved
October 23, 2007, from http://www.ed.gov/policy/elsec/leg/esea02/index.html
Perkins, D. (1998). What is understanding? In M. S. Wiske (Ed.), Teaching for
Understanding: Linking Research with Practice. San Francisco: Jossey-Bass, 39-
57.
Perkins, K., Adams, W., Dubson, M., Finkelstein, N., Reid, S., Wieman, C., & LeMaster, R.
(2006). PhET: interactive simulations for teaching and learning physics. The
Physics Teacher, 44(1), 18-23.
Physics Education Technology Project (PhET). Circuit Construction Kit, Retrieved October
23, 2007, from http://www.colorado.edu/physics/phet/simuiations-base.html
Pintrich, P. R., Marx, R. W. and Boyle, R. A. (1993). Beyond cold conceptual change: the
role of motivational beliefs and classroom contextual factors in the process of
conceptual change. Review of Educational Research, 63(2), 167-199.
Popham, W. J. (2003). Test better, teach better: The instructional role of assessment.
Alexandria, VA: Association for Supervision and Curriculum Development.
Posner, G. J., Strike, K. A., Hewson, P.W., & Gertzog, W. A. (1982). Accommodation of a
scientific conception: Toward a theory of conceptual change. Science Education,
66(2). 211-227.
Prensky, M. (2001). Digital Game-Based Learning. New York: McGraw-Hill.
Randel, J. M., Morris, B. A., Wetzel, C. D., & Whitehill, B. V. (1992).The effectiveness of
games for educational purposes: A review of recent research. Simulation &
Gaming, 23(3), 261-276.
Rasmusen, E. (2001). Games and Information: An Introduction to Game Theory (3rd ed.).
Malden, MA: Blackwell.
Rose, E. (2004). Is there a class with this content? WebCT and the limits of
individualization. Journal of Educational Thought, 38(1), 43-65.
Reeves, T. C. (2002). Keys to successful e-iearning: Outcomes, assessment, and
evaluation. Educational Technology, 42(6), 23-29.
Roth, K. J. (2002). Talking to understand science. In J. Brophy (Ed.), Social Constructivist
Teaching: Affordances and Constraints. Oxford, UK: Elsevier Science, 197-262.
Russell, H. H. (2004). Connections among factors in education. Curriculum Inquiry, 34(3),
267-282.
76


Schbn, D. A. (1983). The Reflective Practitioner: How Professionals Think in Action. New
York: Basic Books.
Scrogan, L. (2006). Compelling stories. EdTech: Focus on K-12. Retrieved October 23,
2007, from http://edtech.texterity.com/article/20060607/25/
Secretarys Commission on Achieving Necessary Skills (SCANS). (1993). Teaching the
SCANS competencies. U. S. Department of Labor. Retrieved September 3, 2007
from http://wdr.doleta.gov/SCANS/teaching/teaching.pdf
Shreve, J. (2005). Let the games begin. Edutopia. 28-31. Retrieved October 23, 2007, from
http://www.edutopia.org/magazine/ed1 article. php?id=art_1268&issue=apr_05
Skinner, J. (n.d.). Brief biography of B. F. Skinner. Retrieved October 23, 2007, from
http://www.bfskinner.org/briefbio.html
Slavin, R. E. (2003). A readers guide to scientifically based research. Educational
Leadership, 60(5), 12-16.
Simons, D. J., & Chabris, C. F. (1999). Gorillas in our midst: Sustained inattentional
blindness for dynamic events. Perception, 28(11), 1059-1074. Movie retrieved
October 23, 2007, from http://viscog.beckman.uiuc.edu/grafs/demos/15.html
Squire, K. D. (2004). Replaying history: Learning world history through playing civilization
III. Unpublished doctoral dissertation. Indiana University, Bloomington. Retrieved
October 23, 2007, from
http://website.education.wisc.edu/kdsquire/dissertation.html
Swartout, W., & van Lent, M. (2003). Making a game of system design. Communications of
the ACM, 46(7), 32-39.
The Design-based Research Collective (2003). Design-based research: An emerging
paradigm for educational inquiry. Educational Researcher, 32(1), 5-8.
VanDeventer, S. S., & White, J. A. (2002). Expert behavior in childrens video game play.
Simulation and Gaming, 33(1), 28-48.
Weiss, I. R., Pasley, J. D., Smith, P. S., Banilower, E. R., & Heck, D. J. (2003). Looking
Inside the Classroom. Chapel Hill, NC: Horizon Research Inc.
West, L. H. W.. & Pines, A. L. (Eds ). (1985). Cognitive structure and conceptual change.
Orlando, FL: Academic Press.
Wilson, B. G. (1997). The postmodern paradigm. In C. R. Dills & A. J. Romiszowski (Eds.),
Instructional Development Paradigms. Englewood Cliffs, NJ: Educational
Technology Publications, 297-309.
77


Wiggins, G. (198S). A true test: Toward more authentic and equitable assessment. Phi
Delta Kappan, 70(9), 703-713.
Wiggins, G., & McTighe, J. (1998). Understanding by Design. Alexandria, VA: Association
for Supervision and Curriculum Development.
Wilson, B. G., Parrish, P., Balasubramanian, N., & Switzer, S. (2006). Open Learning
Meets the Business of Education: Trends in Distance Learning. In A. M6ndez-
Viias, A. Solano Martin, J. A. Mesa Gonzalez & J. Mesa Gonzalez (Eds.), Current
Developments in Technology-Assisted Education (2006) Vol. I: General Issues,
Pedagogical Issues (pp. 125-129). Badajoz, Spain. Retrieved October 23, 2007,
http://www.formatex.org/micte2006/pdf/125-129.pdf
Wilson, M. R., & Bertenthal, M. W. (2005). Systems for state science assessment.
Washington, DC: National Academy Press. Retrieved October 23, 2007, from
http://www.nap.edu/books/0309096626/html/
Wolfe, J., & Crookall, D. (1998). Developing a scientific knowledge of simulation/gaming.
Simulation and Gaming, 29(1), 7-19.
Yeo, R. K. (2007). Change interventions to organizational learning: bravo to leaders as
unifying agents. The Learning Organization, 14(6), 524-552.
Yeo, S., Loss, R., Zadnik, M., Harrison, A., & Treagust, D. (2004). What do students really
learn from interactive multimedia? A physics case study. American Journal of
Physics, 72(10), 1351-1358.
78