EFFECTS OF DESIGN BASED SCIENCE INSTRUCTION ON SCIENCE PROBLEM SOLVING COMPETENCY AMONG DIFFERENT GROUPS OF HIGH SCHOOL TRADITIONAL CHEMISTRY STUDENTS by Cobina Adu Lartson B.S. (Hon) Biochemistry University of Ghana, Legon, Accra 1991 Postgraduate Diploma in Education University of Cape Coast, Ghana 1997 M.A. Environmental Leadership, Naropa University, Boulder, 2004 A thesis submitted to the Faculty of the Graduate Scho ol of the University of Colorado in partial fulfillment of the requirements for the degree of Doctor of Philosophy Educational Leadership & Innovation 2013
2013 COBINA ADU LARTSON ALL RIGHTS RESERVED
ii This thesis for the Doctor of Philosophy degree by Cobina Adu Lartson has been approved for the Educational Leadership and Innovation Program by Geeta Verma Chair Alan Davis Heather Johnson Carole Basile, Advisor April 15 2013
Lartson, Cobina Adu (Ph.D., Educational Leadership and Innovation) Effects o f Design Based Science Instruction on t he Science Problem Solving Skills among Different Groups o f High School Traditional Chemistry Students Thesis directed by Associate Professor Geeta Verma ABSTRACT Recent trends indicate a significant decline in the number of students graduating from S cience, T echnology, E ngineering and M ath (STEM) programs in the US The under representation of students of color, females and low income students in STEM programs has also been documented. Design Based Science (DBS) instruction has been suggested to improve the problem solving skills of students of color. The present s tudy employed a quasi experimental pre post test research study. Four equivalent parallel high school traditional Chemistry classes of eighty two (82) 10 th and 11 th grade students was invited to participate in this study. The treatment group comprised of 3 6 students while the control group was made up of 46 students. The purpose of this study was to investigate whether DBS affects student problem solving competency and chemistry achievement across student demographics (gender, race and SES). The re search questions were: 1) Does DBS have an y effect on the problem solving competencie s of students in a high school traditional c hemistry class ? 2) Does the effect of DBS on problem solving competency depend on gender? 3) Does the effect of DBS on problem solving competency depend on race ? 4) Does the effect of DBS on problem solving competency depend on SES? 5) Do es DBS have any effect on the c hemistry achievement of students in a high school traditional c hemistry class ? 6) Does the effect of DBS on chemistry achievement vary depending on gender ? 7) Does iii
the effect of DBS on chemistry achievement vary depending on race ? 8) Does the effect of DBS on chemistry achievement vary depending on SES? 9) Is the problem solvi ng co mpetency of students in a traditional c hemi stry class predictive of their c hemistry achievement? The findings are as follow: a) DBS significantly improved the problem solving competency of students in the study, b) DBS significantly improves the problem s olving competency of both males and females, with a slight urge among females, c) the differences in the effects of DBS in improving problem solving competency among Black and Hispanic students in this study was not statistically significant, however, Blac k students and Hispanic female students showed significant improvement in problem solving competency after the DBS instruction, d) DBS did not statistically significantly improve the problem solving competency of students of particularly SES group(s), and e) Problem solving competency is a strong predictor of higher chemistry concepts score among students in both treatment and control groups. The form and content of this abstract are approved. I recommend its publication. Approved: Geeta Verma iv
DEDICATION I dedicate this work to loving memory of my parents, Alexander Lartson and Charlotte Love Lartson, who are still very much a part of my life; and to my lovely wife, Angela Lartson and my children Alexander Lartson, Charlotte Lartson and Mary Street Awura Abena Lartson. v
vi ACKNOWLEDGMENTS I would like to express my heartfelt gratitude to my wife, Angela Lartson, for her steadfast support and encouragement throughout this worthwhile journey and for being a constant source of joy in my life. I would also like to thank my parents for their fai th in me and for the love and support they have shown me throughout my life. The completion of this milestone in my lifelong journey is a testimony to the great care and sacrifice they have made for me and my brothers and sister. Thank you to my Aunt Marga ret Price for presence and encouragement. She together with my parents have been a constant but extend my appreciation to Anna Schoettle, who has been there for my family and I, in diverse ways, since I arrived in the United States. Her presence has been an additional sometimes wild journey. Sincere thanks go to Dr. Anne Zonne Park er, Marjorie McCurtain and Cheryl Barbour for the diverse ways in which they supported me and helped create a springboard for the academic life in the United States. I am very appreciative of Dr. Mike Marlow for his encouragement, guidance and direction through his research lab and for the connections and experiences made available to me by fellow students of his research lab. A special thank you to all the teachers who held my hand and walked me through these tough times I consciously chose to endure, pa rticularly: Dr. Alan Davis, Dr. Deanna Sands, Dr. Connie Fulmer, Dr. Mark Clarke, Dr. Nancy Leech, and Dr. Honorine Nocon. I am indebted to Dr. Carole Basile for serving as my initial dissertation advisor, chair of my dissertation committee, and for mentoring me. I thank her for remaining on
vii my dissertation committee despite her new location miles away. I will be forever grateful for her guidance, support and encouragement of good quality work. I admire the significant contributions she has made over the years to the teaching of Math and STEM in Colorado. I will forever cherish her advice to focus on completing the program realizing that my life work does not end at graduation. My admiration and appreciation also go to Dr. Geeta Verma for willingly and without the slightest hesitation, accepting to be chair of my dissertation committee. I am thankful for the high standards she set for me to produce excellent sc holarly work. I am very appreciative of the sacrifices she made for me and for being available, even on call for advice. I will forever remember her desire for excellence combined with flexibility and enthusiasm to see her students succeed. I am also highl y indebted to Dr. Alan Davis being such a wonderful and passionate teacher. I enjoyed being a student in his Quantitative and Research Methods courses. I am thankful to him for serving on my dissertation committee and providing me with the direction and su pport I much needed. I extend a big thank you to Dr. Heather Johnson for her willingness to join my dissertation committee on such short notice. I will forever remember her valuable comments and contribution to my work. Finally, I would like to express my deep admiration for the many dedicated science teachers in Colorado and elsewhere, especially Ghana, who truly make a difference in the lives of children. They contribute daily to building future leaders by helping them acquire not only science content but also skills that will determine the ability of these children to be successful in our fast changing, technology driven new world. I express much admiration for those who have gone before me to become doctors of philosophy in academia and research for I ha ve derived my innermost motivation and
viii drive their work and ingenuity. I pray that the almighty God guides me to be fruitful for the common good, with the knowledge and skills I have acquired from the generosity of all the people mentioned above.
ix TABLE OF CONTENTS CHAPTER I. Relationship between Problem Based Learning, Project Based Learning, Design Based Science and Inquiry Potential of DBS for Low SES and Low PISA and Prob Definition of II. Design Review of Literature on Design Expert Cognitive Theory.....................................
x Models of Problem Solving Instruction and Problem Solving Skills...................45 ..51 III. Teacher as IV. D Effects of Problem Solving Competency across Gender, Race and SES..77 Research Question One: Comparison of Treatment and Control
xi Research Question Two: Problem Solving Co mpetency across Research Question Three: Problem Solving Competency across Research Question Four: Problem Solving Competency across Research Question Five: Comparison of Treatment and Control Research Question Six: DBS and Chemistry Achievement across Research Question Seven: DBS and Chemistry Achievement Research Question Eight: DBS and Chemistry Achievement Correlatio n between Problem Solving Competency and Chemistry V. Findings and Interpretation
xii Effects of DBS on Problem Solving Competency across Effects of DBS on Problem Solving Competency across Race...116 Effects of Effects of DBS on Chemistry Achievement across Gender, Race Problem Solving Competency as Predictor of Chemistry Problem Solvin Implications for Res APPENDIX A. B. PISA C. D.
xiii E. F. Chemistry Con G. H. I. J. Parent Co K.
xiv LIST OF TABLES Table I. Adjusted and Unadjusted Group Means and Variability for Problem Solving Competency Using Problem Solving Pretest S II. Analysis of Covariance for Problem Solving Competency as a Function of Group, Using Problem Solving Pretest Scores as Covariate III. Adjusted and Unadjusted Group Means and Variability for Decision Making Competency Using Decision Making Pret 81 IV. Analysis of Covariance for Problem Solving Competency as a Function of Group, .................. ..............................81 V. Adjusted and Unadjusted Group Means and Variability for System Analysis Competency Using System Analysis Pretest Scores as Covari VI. Analysis of Covariance for System Analysis Competency as a Function of Group, Using System Analysis Pretest Scores as Covariate VII. Adjusted and Unadjusted Group Means and Variability for System Analysis Compe tency Using Troubleshooting Pretest Scores as Covariate VIII. Analysis of Covariance for System Analysis Competency as a Function of Group, Using Troubleshooting Pretest Scores as Covariate IX. Adjusted and Unadjusted Gen der Means and Variability for Problem Solving Competency Using Problem Solving Pretest Scores as Covariate
xv X Analysis of Covariance for Problem Solving Competency as a Function of Gender, Using Problem Solving Pretest Score s as Covariate XI. Adjusted and Unadjusted Group, Race and Gender Means and Variability for 8 9 XII. Analysis of Covariance for Problem Solving Competency as a Function of G roup, Race and gender Using Problem Solving Pretest XIII. Adjusted and Unadjusted SES Group Means and Variability for Problem Solving Competency Using Problem Solving Pretest XIV. Analysis o f Covariance for Problem Solving Competency as a Function of SES Group, Using Problem Solving Pretest Sc XV. Adjusted and Unadjusted Group Means and Variability for Chemistry Concepts Inventory (CCI) Using CCI Pretest Score 8 XVI. Analysis of Covariance for CCI as a Function of Group, Using CCI Pretest Scores XVII. Adjusted and Unadjusted Gender Means and Variability for Chemistry Concepts Inventor y (CCI) Using CCI Pretest Sco XVIII. Analysis of Covariance for CCI as a Function of Gender, Using CCI Pretest XIX. Adjusted and Unadjusted Race Means and Variability for Chemistry Concepts Inventory (CCI) Using CCI Pretest Sco
xvi XX. Analysis of Covariance for CCI as a Function of Race, Using CCI Pretest Scores as XXI. Adjusted and Unadjusted SES gro up Means and Variability for Chemistry Concepts Inventory (CCI) Using CCI Pretest Scores XXII. Analysis of Covariance for CCI as a Function of SES Group, Using CCI Pretest Scores as Covaria X XIII. Simple Linear Regression Analysis for Problem Solving Competency Predicting Chemistry Concepts Score
xvii LIST OF FIGURES Figure II. Relationship between Project based Learning, Problem based Learning, Design Based III. Design Based Science Learning Cycle .36 IV. 4 6 VII. .....79 XI. Mean Problem Solving Scores (Pretest and Posttest) for T reatment and Control Groups XII. Effects of DBS on Problem Solving Competency across Gender XIII. olving Scores in Treatment and Control Groups by Race XIV. Solving Scores in Treatment and Control Groups by Race
xviii XV. Comparison of Problem Solving Competency Scores of Black Female and Male XVI. Comparison of Problem Solving Competency Scores of Female and Male Hispanic XVII. Graph of Problem Solving Posttest Scor es for Treatment and Control SES XVIII. Graph of CCI Posttest Scores for Female and Male Students in Treatment and
1 CHAPTE R I INTRODUCTION The total number of students graduating from math, engineering and physical science majors has been on the decline since the mid 1980s (Mooney and Laubach 2002). Recent trends suggest a significant decline in the number of students interested in Science, Technology, Engineering and Mathematics (STEM) careers (van Langen and Dekkers, 2005) in K 12 T he under representation of STEM Bachelor degrees earn ed by targeted minority student groups, namely, African Americans, Latino/as, South East Asians and Native Americans (ALANAS) ( The Center for Education and Work, 2008) and girls ( Fadigan & Hammrich, 2004; Gilbert & Calvert, 2003; Scantlebury & Baker, 2007) has also been reported. D espite the similarity in the intentions of ALANA and White students to major in STEM fields the former are less likely to major and more likely to drop out of STEM programs ( The Center for Education and Work, 2008). A number of factors have been identified as accounting for these findings. One of the reasons advance for t he decline in the number of US students participating in STEM programs in general and in particular among female and minority students is the view among these students that science and technology i s uninteresting, Technology PCAST, 2010). Hanushek & Rivkin, (2003) also point to teacher quality and effectiveness as important factors deter classroom s The results of the PISA 2003 problem solving competency assessment (OECD, 2003) suggest a strong correlation between problem solving competency and socioeconomic status (SES). With a high proportion of ALANA S falling within low SES
2 in the United States, the PISA 2003 problem solving assessment results also suggest low problem solving competency among this group of students. The current study focuses on problem solving ability since problem solving has been id entified as a 21 st century skill needed for students to be successful in and out of school ( Partnership for 21 st Century Skills, 2009) Problem solving is a focus of the current study also because problem solving through scientific reasoning (as proposed b y John Dewey) remains a primary goal of science education (Atkins & Black, 2003). Various related pedagogies such as Problem based learning (PlBL), Project based learning (PjBL), Design Based Science (DBS) and Inquiry based learning (IBL) have been used in an attempt to improve problem solving skills. These pedagogies contextualize learning by making content relevant to student life experiences. Such modes of teaching have a potential of closing the achievement gap (Wisely, 2009). During Problem Based Learn intended to help students build skills and content knowledge. PjBL is similar to PlBL except that student projects involve a culminating artifact in the problem solving lesson. DBS is a type of Pj BL, which incorporates inquiry (the use of the scientific method in problem solving). In his review of research on PjBL, Thomas (2000) notes that most of the research on PjBL took place between 1995 and 2000. Thomas (2000) references a number of studies s uch as those by the Expeditionary Learning Outward Bound (ELOB), Co nect schools and the Academy for Educational Development (AED), relating PjBL and student achievement. Although significant improvements in student achievement were reported by the studies for students in PjBL programs, Thomas (2002) points out that the
3 results may be attributable in part to features other than those of PjBL (e.g. portfolios, flexible block scheduling for ELOB; technology in the case of Co nect schools). He also posits that since technology and expeditions do not target basic skills (reading, writing and computation), the reported effects of these PjBL achievement may be the result of a generalized effect associated with the whole school re form effort or, perhaps, the motivational effect of project based instruction may lead to increased student attendance, attention, and engagement during the (non project) periods students spend learning basic skills. Also, only one study of PjBL effectiveness was found that used a longitudinal experimental design, with pre post assessments. As far as the effects of PjBL on problem solving skills, studies conducted between 1995 and 2000 relate primarily to PlBL (not PjBL) (Thomas, 2000). Furtherm ore, there are improved methods for assessing problem solving capabilities (OECD, 2003). Recent empirical studies suggest that middle school students, particularly African American students, involved in DBS improve both in their science content knowledge a nd problem solving skills ( Fortus, Dershimer, Krajcik, Marx and Mamlok Naaman, 2005; Mehalik, Doppelt and Schuun 2008) It is clear from the preceding overview that while more research is needed in the effectiveness of PjBL (and consequently DBS) in impr oving student achievement and problem solving capacities, more of this work is needed at the high school level, where achievement gaps, particularly that between genders widen ( Ingels & Dalton, 2008; Bacharach, Baumeister & Furr 2003; Jones, Mullis, Raize n, Weiss, &Weston, 1992) Also, in the last ten years, just a few studies have been conducted comparing the effectiveness of DBS in improving science achievement across race/ethnicity. The focus of this study is thus to investigate the relationships betwee n
4 DBS on the one hand, and problem solving competency and science knowledge gain on the other, among different genders, race and socioeconomic groups. The interactions between DBS problem solving skills, science achievement and race ethnicity, elicited fro m some of the above referenced studies, are summarized in Figure I below. Mehalik, Doppelt and Schunn 2008; Fortus, Dershimer, Krajcik, Marx and Mamlok Naaman, 2004, 2005 and Chang, 2001a, 2001b) posit that different modes of problem solving associated ins truction such as DBS solving ability and consequently science achievement. The strong correlation between problem solving ability and science achievement reported by OECD (2003) also speaks to the latter relationship. DBS is o ne of such science pedagogies that has been found to show promising results in science education, by improving both science achievement (Kolodner, Camp, Crismond, Fasse, Gray, Holbrool, Puntambekar, Ryan, 2003; Mehalik, Figure I. Interactions between DBS, Science Knowledge and problem solving skills Design Based Science (DBS) ( Science inquiry through engineering design) Development of Real World Proble m Solving Skills SES Gende r Science knowledge gain/Achievement Race/Ethnicity Can improve Any effects? High or low correlation ? What differential effects of DBS exist (in achievement & problem solving competency) between high school Chemistry students in these groups ?
5 Doppelt and Schunn, 2008; Fortus, Dershimer, Krajcik, Marx and Mamlok Naaman, 2004, 2005; Silk, Schunn & Cary (2009). DBS has also been described as pedagogy in which scientific know ledge and problem solving skills are constructed. The extent to which the achievement gap contributes to the potential loss of the economy cannot be overlooked. The McKins ey consulting firm released a report on the of the report revealed a national decline in productivity and jobs. According to the report, if the U.S. had been able to clos e the gap in science and math achievement between 1983 and 1998 and raised its performance to the level of such nations as Canada, Finland and South Korea, the U.S. Gross Domestic Product (GDP) in 1998 would have been approximately $2 trillion higher. If t he achievement gap had been closed between Black and Hispanic students on the one hand and white and Asian students by 1998, the GDP in low income students and the remaini ng students had been similarly narrowed, GDP in 2008 would have been $400 to $670 billion higher. In terms of PISA math and science output and the amount of money the U.S. spends on each student, which is among the highest in the world, the report conclude s that the United States gets 60% less for its education dollars in terms of average test score results than do other wealthy [industrialized] nations. This chapter presents the trend of declining science achievement among U.S. students as well as the pote ntial of DBS in the construction of new scientific knowledge It also reviews the potential of DBS in the development of problem solving skills in
6 secondary science education It describes the need for a study of the effects of DBS on different groups of s tudents, vis vis the over a decade long decline in the number of US students in STEM programs (particularly girls, and minority students) as well as poor problem solving abilities This decline is presented not only as a national issue but also in relati on to other industrialized countries with its implications for the U.S. economy. A number of contributory factors to the decline are discussed, with emphasis on the problem solving abilities of students across gender, racial ethnic and socioeconomic groups both in the U.S. and on the international scene. Also, findings from empirical studies are presented in exploring the benefits of innovative science pedagogies on the development of problem solving skills The relationship s between such pedagogies as DBS, P l BL, project based learning, and inquiry that highlight problem solving and the development of problem solving skills are presented The goal of this chapter is therefore to propose the need to study the effects of DBS (an integration of project based le arning and inquiry based learning) on high and science knowledge Problem solving skills must be studied across gender, racial ethnic and SES in an attempt to meaningfully contribute to efforts aimed at closing the a chievement gap in science. Closing the achievement gap has become increasingly important in a struggling U.S. education system in which racial ethnic and socio economic differences are becoming more and more pronounced. Trends in the Performance of U.S. S tudents in Science The decline in U.S. education has been attributed to inadequate preparation and experiences in math and sciences (ACT, 2006), family characteristics and educational support variables, attitudes toward math and science, differences in ap titude (Benbow &
7 Arjmand, 1990) and inadequate problem solving skills (Ornstein, 2010). Although the United States can identify individual schools and school districts that have been successful, the U.S. education system as a whole has been on the decline for at least a couple of decades nationally and internationally. The Department of Education (2004) and Helpman (2004) reported that several indicators of the performance of U.S. students in science and mathematics education at the pre college level reveal a mixed picture of successes and shortcomings. A discussion of data from the National Assessment of Educational Progress (NAEP), the Trends in International Mathematics and Science Study (TIMSS) and the Program for International Student Assessment (PISA) will elucidate the performance of students in high stakes science assessments. On NAEP less than one third of U.S. eighth graders show proficiency in mathematics and science, and science test scores have improved although very little over the past few dec ades. According to the US Department of Education (2012), t he overall average score for the nation at grade 8 was 2 points higher in 2011 than in 2009. Score gaps between White and Black students and White and Hispanic students narrowed from 2009 to 2011. Sixty five percent of eighth graders performed at or above the in 2011, 32 percent performed at or above Proficient, and 2 percent performed at the Advanced level. The percentages of students at or above Basic and at or above Proficient were higher in 2011 than in 2009. Of the 47 states/jurisdictions that participated in 2009 and 2011, public school students in 16 states scored higher in 2011 than in 2009. In 2011, students in 29 states scored higher than the national average, and in 16 states they scored l ower. T he NAEP 2005 science assessments revealed that 12 th graders, showed no change in performance from the administration of the assessment in 2000. However, the 2005
8 average scores were lower than those in 1996. Also, at this grade level, the percentage of students performing at or above the basic level, at or above the proficient level, and at the advanced level all declined compared to 1996 data. In addition, the number of students who scored below basic increased since 1996 (Department of Education, 2 006). In the U.S. achievement gaps in science and math between white or Asia n/Pacific Islander students and minorities ( traditionally underrepresented in STEM ) exist at all levels, including significant gaps among the highest performing students. For exam ple, a recent analysis of both NAEP and state assessment data shows that large achievement gaps in mathematics performance continue to persist between white and underrepresented minority high achievers (Plucker, Burroughs & Song, 2010). I nternational compa mathematics place the United States in the middle of the pack or lower. TIMSS measures what students know and can remember in science and math. In 2007 U.S. fourth graders and eighth graders placed about average among industrialized and rapidly industrializing countries. However, U.S. students in fourth, eighth, and twelfth grades drop progressively lower on international comparisons of science and mathematics ability as their grade level increases. The U. S. Department of Education (2009), reports that in 2007, the average science scores of both U.S. fourth graders (539) and eighth graders (520) were higher than the TIMSS scale average (500 at both grades). The average U.S. fourth grade science score was hi gher than those of students in 25 of the 35 other countries, lower than those in 4 countries (all of them in Asia), and not measurably different from those in the remaining 6 countries. At eighth grade, the average U.S. science score was higher than the av erage scores of students in 35 of the 47 other countries, lower than those in 9
9 countries (all located in Asia or Europe), and not measurably different from those in the other 3 countries. PISA is a triennial survey of knowledge and skills of fifteen year old students in countries that form the Organization for Economic Co operation and Development (OECD). The areas PISA focuses on during each six year cycle are mathematics, science, reading and problem solving. However, within each cycle the surveys emphas ize different areas. The U.S. average score in science literacy in 2009 was higher than the U.S. average in 2006, the only time point to which PISA 2009 performance can be compared in science liter acy. While U.S. students scored lower than the OECD average in science literacy in 2006, the average score of U.S. students in 2009 was not measurably different from the 2009 OECD average. However, in 2009 while 1% of U.S. students were proficient at level 6 the percentage was higher (1.5%) in 2006. At Level 6, st udents can consistently identify, explain and apply scientific knowledge and knowledge about science in a variety of complex life situations. U.S. students scored below most other nations tested in 2006, and the U.S. standing dropped from 2000 to 2006 i n b oth math and science. It is generally accepted that most high school graduates do not enroll in science and mathematics courses. In fact, of all ninth graders in the United States in 2001, for example, only about 4 percent are predicted to earn college de grees in STEM fields by 2011 (PCAST, 2010). Likewise, many studies note the importance of achievement in science during high school for determining later persistence in the science pipeline through college and early career years (Carmichael, 2007; Hanson, 1996; Kaufman, 1991). Other predictors of persistence in the science pipeline include classroom climate,
10 classroom pedagogy, faculty attitudes and behavior and financial aid (Carmichael, 2007). Thus, pre college years represent a critical period for encour aging students to enter the science pipeline. Indeed, the level of science preparation in secondary school, and specifically pre college science achievement, is generally noted to be among the most ersistence in postsecondary STEM fields (Griffith, 2010; Hanson, 1996; Davis, Ginorio, Hollenshead, Lazarus & Rayman., 1996). The decline in the stream of high school graduates into undergraduate science, technology, engineering and mathematics trend mani fests on college campuses all over the country. In fact, the number of U.S. college students majoring in science, math and engineering is flat, and the percentage of graduates in these essential areas in Western European and especially Asian countries have increasingly outpaced the U.S. (Ornstein 2010). The achievement gaps in science and math between white or Asian/Pacific Islander students and minorities as established by NAEP and state assessment data underscore a system ic problem: the lac k of opportunities and support for underrepresented minority students, inadequate teaching, and an absence of both real life, hands on experiences with STEM materials and positive role models of STEM professionals (Hanushek & Rivkin, 2009; Reardon, 2008; G ndara, 2005; Donovan & Cross, 2002). In addition, many stu dents who academically qualify for postsecondary studies in science and math fields at both two and four those programs due to a number of reasons. These reasons inc lude: dissuasion by disappointing postsecondary experiences, high tuition or demanding curricula and courses of study, relatively low salaries in STEM fields compared to other professions, or
11 the lack of role models with whom they can identify (American As sociation of State Colleges and Universities, 2005). As varied as the causes are for a struggling U.S. education system, so must the solution. The under representation of women, people of color, and the poor decreases not just the quantity, but the quality and breadth of the talent of persons in STEM fields (Drew, 1996). Just by their sheer numbers, women, people of color and the poor participate significantly in the struggling U.S. education system. Solutions for the declined U.S. education system must add ress the achievement gap in science and math between races, socio economic classes and gender. The achievement gap between Asian and white students compared to Hispanic and black students remain alarmingly high, granted that by the year 2015 the latter gro up of students will represent the majority enrollments in U.S. public schools (Ornstein 2010). It is typically noted in the literature, the primary cause of attrition of minority students from scientific fields is often poor academic preparation prior to c ollege (Oakes, 1990; Petersdorf, 1991). In addition, racial ethnic differences in science achievement are generally larger throughout all grades than are gender differences (Hanson, 1996). Research studies show that race ethnicity explains much more of the variance in scie nce achievement scores than does gender, and females and males within racial ethnic categories are much more similar with regard to achievement than are females across racial ethnic categories ( Muller, Stage & Kinzie, 2001; Clewell & Ginor io, 1996; Creswell & Houston, 1980). Research on racial ethnic differences shows that Asian American and White students show higher science achievement scores, as well as disproportionately greater science achievement gains, during middle school and high s chool than their Latino/a and African American
12 counterparts ( Bacharach, Baumeister & Furr 2003; Scott, Rock, Pollack, Ingels, & Quinn, 1995), and are similarly overrepresented in high school and college science courses; ST E M college majors; and scientific and technical careers (Peng, Wright, & Hill, 1995). In general, these racial ethnic differences on standardized science tests appear much earlier than gender differences ( Bacharach, Baumeister & Furr 2003; Mullis, Dossey, Owen, & Phillips, 1993), and the racial ethnic differences tend to increase with age ( Bacharach, Baumeister & Furr 2003; Gross, 1988, 1989). Although less prominent, gender gaps in science must be addressed to ensure that all students reach their potential. Gender Gap in STEM Education Over the last 30 years the gender gap in science has narrowed, however, girls and women remain underrepresented and marginalized in physics, engineering and technology (Fadigan & Hammrich, 2004; Gilbert & Calvert, 2003; Scantlebury & Baker, 2007). The ge nder gap favoring males does not only appear consistently across all racial ethnic groups, but also the pattern appears to be consistent throughout middle school and high school, with the differences widening sharply by Grade 12 ( Ingels & Dalton, 2008; Bac harach, Baumeister & Furr 2003; Jones, Mullis, Raizen, Weiss, &Weston, 1992). There are a number of pointers that may explain some of these observations. Girls of all ethnic groups have negative science attitudes and fewer science experiences than boys (C atsambis, 1995). According to Miller, Blessing & Schwartz (2006) girls like biology, chose people oriented majors, and chose science majors to help people or animals and that girls perceive science as uninteresting, passionless, or leading to an unattracti ve
13 lifestyle. This finding partly explains the under representation of women in STEM professions. In 2006, women earned only 28% of Ph.D.s in physical sciences, 25% in mathematics and computer science, and 20% in engineering in the United States (NSF, 2008). Although women made up 47% of the US workforce in 2009, the percentage of nd mathematical 14%, respectively (US Bureau of Labor Statistics, 2009). The gender gap in STEM disciplines goes beyond the limited representation of women. In college physic s women earn lower exam grades and lower scores on standardized tests of conceptual mastery Pamela, 2010). Various factors have been identified as accounting for the gender gap in STEM. identified as a major contributor to performance in introductory physics (Hazari, Tai & Sadler, 2007). In their review of literature Scantlebury and Baker (2007) identified other factors such as science attitudes, classroom environments and the impact of policies such as high stakes testing that contribute to the gender gap. Brotman & Moore (2008), in reviewing science education literature from 1995 to 2006, develo ped four themes underlying t he gender gap a) equity and access b) curriculum and pedagogy c) the nature and culture of science and d) identity. Strategies related to gender responsive curricula are found in PjBL (and DBS) and is therefore worth exploring. Brotman & Moore (2008) noted a variation in what different
14 researchers describe d as gender inclusive curriculum and pedagogy. However, some common features emerge d from the interventions attempted by these researchers Specifically, a gender inclusive science curriculum experiences, interests, and preconceptions; prioritizes active participation; incorporates long term, self directed projects; includes open ended assessments that take on diverse forms; emphasizes collaboratio n and communication; provides a supportive environment; uses real life contexts; and addresses the social and societal relevance of science. It also pays attention to issues of sexism and gend er bias in curriculum materials. Roychoudhur Tippins, & Nichols (1995) found that among majority of prospective elementary teachers most of whom were female, situated, collaborative learning and long term, open ended projects in a physical science class triggered feelings of empowerment, competence, ownership, and a n appreciation for the connection between science and their lives The implications of the achievement gap to the United States make it imperative for serious systemic corrective actions. G ender inclusive science curricula are in many ways consistent with recommendations made by science education reform efforts in general, which attempt to improve science education for all students through constructivist approaches (American Association for the Advancement of Science [AAAS, 1993]; National Research Council [NRC, 1996]) promoted by John Dewey, for example, DBS John Dewey and Problem Solving In 1910 Dewey proposed problem solving through scientific reasoning as a goal of science education. A review of science education reform in the U.S. and other developed n ations (Atkin and Black 2003) clearly indicate that this goal was never really
15 l ate 1980s the Deweyan problem solving approach to science education, revived in the U.S., was evident in other developed nations such as France, Japan, Scotland, Canada, Australia, Germany and Spain. As the 20 th century drew to a close the Deweyan goal of science education was broadened to include inquiry. During the last fifteen years inquiry activities have become increasingly integrated with the design process, which is also a problem solving process (Fortus, Dershimer, Krajcik, Marx & Mamlok Naaman (200 4). This is in line with recommendations by the International Technology Education Association (ITEA) (ITEA, 2002). The development of problem solving skills as a long standing goal of science education is well documented (Atkin and Black 2003, Stewart, 1982; Wavering, 19 80; Champagne and Klopfer, 1977 ). In furtherance of this goal however, s chool science has traditionally been taught around well defined problems, such as predicting an ideal ed by the ignition of given amounts of hydrogen and oxygen. On the other hand, real world scientific inquiry focuses on ill defined problems as aptly described by the American Association for the Advancement of Science (AAAS, 1990), xed set of steps that scientists always follow, no one path Many school curricula and teaching practices have been criticized because they do not give students experience in real world problems, in situations where decisions are not clear cut, where requirements can conflict, and
16 A number of researchers and organizations have recommended restructurin g school science so that science, in the classroom is taught around real world problems world problem solving, should be incorporated into education in general and in science education in particular (AAAS, 1990; Chiapetta, Koballa, Jr., & Collette, 2002; Davis, 1998; ITEA, 2002; Layton, 1993; NRC, 2002). These recommendations have led to crucial actions by various stakeholders including the U.S. federal government, industry an d foundations to improve science, technology, engineering and mathematics (STEM) education. Real world problems are ill defined, lacking some required information, and not necessarily having a known correct or the best solution (Nickerson, 1994; Roberts, 1 995). The Government Accountability Office (GAO, 2005) catalogued and assessed the impact of federal programs designed to improve educational programs particularly STEM curricula. The analysis also included the impact of such programs on the number of stu dents pursuing S TEM careers. Industries and firms dependent upon a strong science and math workforce pipeline have launched a variety of programs that target K 12 students and undergraduate and graduate students in STEM fields. Industry associations that i nclude the Society for Manufacturing Engineers, the American Chemical Society, the American Physical Society, the National Association of Manufacturers, and the National Science and Technology Education Partnership invest in STEM education initiatives that involve curricular improvements, career focused websites, mentoring programs, scholarships, and other incentives and supports. Individual firms and their corporate foundations, including Raytheon, Bayer, and General Electric, have created
17 outreach efforts of their own (Delaware Valley Industrial Resource Center and National Council for Advanced Manufacturing, 2006). Project Lead the Way (PLTW) operates in over 4,000 middle and high schools in the 2011/12 school year in all 50 states of the U.S., bringing t hem STEM programs (PLTW, 2011). For instance, The PLTW Gateway To Technology (GTT) program features a project based curriculum designed to challenge and engage the natural curiosity and imagination of middle school students ( http://www.pltw.org/our programs/middle school engineering program ) Another example is showcases math (and) in action as students design and experience t heir own thrill ride using math fundamentals ( http://mathalive.com/raytheon mathmovesu/#raytheon ). STEM Education attempts to transform the typical teacher centered classroom by encour aging a curriculum that is driven by problem solving, discovery, exploratory learning, and require students to actively engage a situation in order to find its solution ( Fioriello, 2010) The above initiatives, though worthwhile, they are optional to schoo ls and not available to schools nation wide. The Next Generation Science Standards to be released in April 2013 sets the stage for the application of engineering design in K 12 science lesson nationwide. Draft II ( http://www.nextgenscience.org ) of the Next Generation Standards incorporates engineering design in science lessons in K 12 classrooms. Some of the well known STEM approaches are PlBL, inquiry, problem based learning and DBS. These approaches are howeve r similar and intertwined. Relationship between PlBL, P j BL, DBS and Inquiry Problem based learning, as it is generally known today evolved from innovative health sciences curricula introduced in North America over 30 years ago at McMaster
18 University in Ca nada and became an accepted instructional approach in medical institutions across North America and in Europe in the 1990s (Boud and Feletti, 1997). Hmelo Silver (2004) described P roblem based learning as an instructional method in which students learn thr ough facilitated problem solving that centers on a complex problem, which does not have a single correct answer. Torp and Sage (2002) described P roblem based learning as focused, experiential learning organized around the investigation and resolution of me ssy, real world problems. They describe d students in a P roblem based learning classroom as engaged problem solvers, seeking to identify the root problem and the conditions needed for a good solution and in the process becoming self directed learners. Savery (2006) described problem based learning as a learner centered i nstructional approach that empowers learners to conduct research, integrate theory and practice, and apply knowledge and skills to develop a viable solution to a defined problem. The Pro blem based Learning Institute has developed curricular materials and teacher training programs for all core disciplines in high school (Barrows & Kelson, 1993). P roblem based learning is now used in multiple domains such as pre service teacher education (H melo Silver, 2004) and chemical engineering (Woods, 1994). Project B ased Learning (P j BL) has a long historical background (Grant, 2002). It (Wrigley, 1998). Since, John (Oguzkan, 1989). P l BL can thus and According to the definitions found in P j BL handbooks for teachers, P j BL involves complex tasks, based on challenging questions or
19 problems, that involve students in design, problem solving, decision making, o r investigative activities; give students the opportunity to work relatively autonomously over extended periods of time; and culminate in realistic products or presentations (Jones, Rasmussen, & Moffitt, 1997; Thomas, Mergendoller, & Michaelson, 1999). Inquiry based learning just like P j BL is grounde d in the philosophy of John Dewey, wh o believed that education begun with the curiosity of the learner. Inquiry based learning is a student centered, active learning approach focused on questioning, critical thinking, and problem solving (Savery, 2006) In quiry based learning activities begin with a question followed by investigating solutions, creating new knowledge as information is gathered and understood, discussing discoveries and experiences, and reflecting on new found knowledge. Inquiry based learni ng is frequently used in science education and encourages a hands on approach where students practice the scientific method on authentic problems ( or questions). DBS was designed around a stepwise description of the design process (Davis, Hawley, McMullan & Spilka, 1997) and a social constructivist perspective of learning (Blumenfeld, Marx, Patrick, Krajcik, & Soloway, 1997). Silk, Schunn & Cary (2009) define design based learning in general as a type of project based learning, which engages students in t he process of developing, building, and evaluating a product they have designed. According to Krajcik, Blumenfeld, Marx, Bass, Fredricks, & Soloway (1998) DBS is an inquiry based pedagogy that grew out of Project Based Science, which is similar to P roblem based learning Design based science (DBS) is a science pedagogy that aligned with the goals of STEM education and inquiry based science education. DBS, according to Fortus, Dershimer, Krajcik, Marx and Mamlok Naaman (2005) is an
20 inquiry based science peda gogy in which new scientific knowledge and problem solving skills are constructed in the context of designing artifacts. In a PjBL environment learners are usually provided with specifications of a desired end product achieved by following correct procedur e. However, learners are likely to encounter several problems that The relationship between PlbL, PjBL, DBS and IBL are presented in Figure 2. The primary difference between P lBL and in quiry based learning relates to the role of the F igure II. Relationship between PjBL, PlBL, DBS and IBL PROJECT BASE LEARNING (PjBL) PROBL E M BASE LEARNING ( Pl BL) INQUIRY BASE LEARNING ( I BL) Authentic Problem to solve centered making practice Students seek root problem & conditions for resolution of problem Teacher is purely a facilitator DBS: Developing, building & evaluating student designed product or presentation provided Begins with questioning & critical thinking Directed by scientific method & experimental design Teacher as facilitator & provider of information
21 tutor (Savery, 2006) In an inquiry based approach the tutor is both a facilitator of learning (encouraging/expecting higher order thinking) and a provider of information. In a PjBL approach the tutor supports the process and expects learners to make their thinking clear but the tutor does not provide information related to the problem that is the responsibility of the learners. DBS, however, incorporates inquiry and the design process. Common to these three approaches is certainly the presence a problem to be solved w ith the learner as an active player and the teacher as a facilitator hence their inter relatedness. It must be noted that the choice of DBS for the current study was made due to t he promising results reported in previous research efforts in the areas of L earning by Design (Kolodner, Camp, Crismond, Fasse, Gray, Holbrool, Puntambekar, & Ryan, 2003), project based learning (Prince, 2004; Thomas, 2000) and problem based learning 2007). T he choice of DBS over other approaches was influe nced by the fact that it combines inquiry and project based approaches both of which are consistent with problem solving as a goal of science education. Potential of DBS for Low SES and L ow P erforming Students The results of preliminary studies by Fortus Dershimer, Krajcik, Marx and Mamlok Naaman, (2005) and Puntambekar & Kolodner, 2005 imply that DBS and other inquiry based pedagogies have the potential of helping students develop science knowledge Engaging students in design based learning or problem based learning within a science classroom has the potential of helping students develop problem solving skills and scientific inquiry skills (Kolodner, Camp, Crismond, Fasse, Gray, Holbrool, Puntambekar, and Ryan, 2003; Silk, Schunn and Strand, 2007). In their study of the effect s of DBS on science achievement among middle school students by gender, socio
22 economic status (SES) and race ethnicity, Mehalik, Doppelt and Schuun (2008) report ed that low achieving African American students benefited the most fro m DBS The above findings provide an excellent opportunity to study the effects of innovative pedagogies such as DBS on the improvement of science achievement gaps in secondary education. This is because they are primarily focused on middle schools. More r esearch is thus needed in subjects such as Physics, Chemistry and Biology in high school settings. Efforts to draw attention to the importance of problem solving in science are exemplified not only by the increasing number of project /problem solving progr ams but also by PISA. The potential of DBS in improving student achievement and problem solving competencies in science may emanate not only from its student centered, hands on approach but also from it contextualization of science. The effects of context ualization, such as provided by DBS were studied by Wisely (2009). He investigated the hypothesis that low skilled students can learn more effectively and advance to college level programs more readily through contextualization of basic skills instruction. The results of his study which was however not in science, showed that participation in contextualization was associated with the completion of developmental education courses and the speed of entry into, and performance and completion of, college level courses. These positive effects were however, limited to non white students: no effects for contextualization were found for white students. There is an that contextualiz ing science problems through real world problem based instruction, aligned with lower class students' preferred ways of thinking. This overlap is the relationship between race and socio economic status : the idea that low skilled and lower
23 class students be nefited more from contextualization Indeed, LaVeist (2005) describes the correlation between race and SES as substantial, with Whites and Pacific Asians having a high SES while African Americans and Hispanics tend to belong to the low SES group According to Baker, Hope, and Karandjeff (2009), contextualization has been defined in numerous ways. For the purposes of this study I refer to the proposal by Mazz eo, Rab, and Alssid (2003) that, designed to more seamlessly link the learning of foundational skills and academic or occupational content by focusing teaching and learning squarely on concrete applications in a specific context tha (p. 3) The above findin gs therefore present another opportunity to investigate the effects of DBS on the development of problem solving skills among various groups of students If the relationship between DBS and science achievement among low class African American students and the correlation between science literacy and problem solving skills are real then DBS may contribute to fighting the downward trend of minority PISA and Problem Solving capabilities. Problem solving will next be measured in 2012. The PISA 2003 problem solving assessment measured the capability of fifteen year olds to apply knowledge to solving cross disciplinary tasks, which approximate real life situations. While U.S. s tudents showed improved science results between 1995 and 2003 on their TIMSS average science score, their 2003 PISA average score in science and problem solving were below the international average
24 (National Science Board, 2006). This confir ms the observat ion by OEDC (2004 ) that U.S. students solve d problems at the basic level ( http://www.oecd.org/dataoecd/25/12/34009000.pdf ) These students were consistently able to understand the nature of a problem and the relevant data associated with a dealing with multi faceted problems involving multiple data sources or requiring analytical reasoning with the information provided. Data from the PISA 2003 also reveals a high correlation of 0.8 between problem solving competency and science achievement. The PISA 2003 data also describe the relationship betwe en international socio economic index (largely determined by parental occupational status) and problem solving capabilities. Fifty percent of the variance in problem solving performance in the U.S. results was explained by international socio economic index. The disparity in problem solv ing performance between the top and bottom socio economic index for the U.S. is approximately 90 score points, close to one proficiency level in problem solving performance (OECD, 2003). Improving this relationship between problem solving performance and s ocio economic status may therefore contribute significantly to closing the achievement gap b etween socio economic classes. Earlier r esearch studies by Clewell & Ginorio (1996); Creswell & Houston (1980) suggest that race ethnicity and gender explain a sig nificant por tion of the variance in science achievement scores to varying degrees. If problem solving is significantly correlated to science achievement and SES, then any instructional approach that improves the problem solving competencies of students cou ld provide leverage in closing achievement gaps between genders, SES and race. Also, o n the premise that almost
25 everyone naturally engages in problem solving (Nickerson, 1994) and design activities (Roberts, 1995; Baynes, 1994) it can be inferred that desi gn based lessons have the potential to address the basic capacity of all students. The current study therefore attempts to ascertain the impact of DBS on the development of problem solving skills among students across gender, racial ethnic and socio econom ic status (SES). T he integration of science, mathematics, and technology has been a common expectation in science education reform to encourage problem solving, particularly in Project 2061. According to Project 2061, ce, mathematics, and technology They are ideas that transcend disciplinary boundaries and prove fruitful in explanation, It is therefore anticipated that science education will help stud ents learn to integrate cross disciplinary principles and knowledge in solving problems. Numerous documents stress the importance of technology in science learning. Some of these efforts include: The National Science Education Standards [NSES] (National Re search Council [NRC], 1996), and Project 2061 (American Association for the Advancement of Science [AAAS], 1989, 1993). Technology is quite often used in a wide variety of meanings. However, among many definitions of technology, the above documents consist ently use technology to refer to engineering, design, or engineering and design interchangeably (Raizen, Sellwood, Todd, & Vickers, 1995; Roth, 1998). The component of technology most closely allied to scientific inquiry and mathematical modeling is engine ering. In its broadest sense, engineering consists of construing a problem and designing a solution for it. According to NSES,
26 difference in goal: The goal of science is to und erstand the natural world, and the (NRC, 1996, p. 24). The integration of science and technology thus allows students to use scientific knowledge to design and solve real world p roblems. Despite its importance in making science relevant and practical in everyday life, technology as engineering and design has been largely ignored in school science (Raizen, Sellwood, Todd, & Vickers, 1995). However, the situation is changing. There have been increasing efforts by corporate bodies and the U.S. government to improve STEM programs in schools (Delaware Valley Industrial Resource Center and National Council for Advanced Manufacturing, 2006; PLTW, 2011; GAO, 2005). The result is an increas ing popularity of pedagogies such as DBS, P l BL and PjBL all of which have problem solving in common. These science instructional pedagogies (in science) enable the transfer of science knowledge to solving real world problems. A number of empirical studies link real world problem solving associated pedagogies to science achievement. Statement of the P roblem Student achievement in science ( and math ) in the United States has been on the decline over the past couple of decades, both nationally and internatio nally (Ornstein, 2010; U. S. Department of Education 2004, 2006; NEAP, 2005) particularly in secondary education The achievement gaps between males and females (NSF, 2008; Ornstein, 2010) and race (Ornstein, 2010; Clewell & Ginorio, 1996; Creswell & Hous ton, 1980) have indeed not improved. An observable direct consequence of this decline is the
27 decreasing of competitive edge by the United States in the global market place and the leveling of college enrollment into STEM programs while the opposite is the case in other industrialized nations (Ornstein, 2010). The decline in student academic achievement in science ( and math ) has been attributed to a myriad of factors including pre college educational preparation and high school test scores in math and scienc e (Griffith, 2010), family characteristics and educational support variables, attitudes toward math and science, and differences in aptitude (Benbow and Arjmand, 1990) and problem solving skills (Ornstein, 2010; OECD 2003). Problem solving associated science pedagogies such as DBS and PlBL have the /critical thinking, problem solving ability, science process skills and consequently science achievement (Mehalik, Doppelt and Schunn 2008; Fortu s, Dershimer, Krajcik, Marx and Mamlok Naaman, 2004, 2005; OECD 2003; Chang, 2001a, 2001b). Other factors that have been verified to improve science achievement, particularly among African American students in particular, include their experience/conflict with science discourse (Brown, 2004; Fang, 2004; Bergin & Cooks, 2002). Systemic problems of teaching African American students, including problems of teachers lacking science knowledge, poorly trained teachers, and poor expectations continue to exist and can be seen as a social inequity issue for African American learners (Atwater, 2000) as well as other low income urban students. The current study identifies real world problem solving ability for study for a number of reasons, namely, a) it has been a long standing goal of science education, b) it has been identified as one of the 21 st century skills that will help students be successful in life and at their workplaces (Partnership for 21 st Ce ntury Skills, 2009), c) it correlates
28 highly with science achievement ( OECD 2003), d) a number of studies suggest that DBS, a project based inquiry approach can improve real world problem solving skills as well as improve science achievement among one of the target groups of this study namely, African Americans (Mehalik, Doppelt and Schuun, 2008; Kolodner, 2002; Rivet & Krajcik, 2004), and that e) boys and girls tend to favor hands on activities (such as in DBS), which result in better science attitudes (Jovanic & Steinbach King, 1998). Rational/Purpose of the S tudy A number of factors have been identified to account for the decline in science achievement in the U.S. locally and on the international scene. Of these factors real world problem solving ski lls will be the focus of this study for reasons explained above. While there have been signs of improvement in science scores during the last few years, achievement gaps (between race/ethnicity, gender and SES) remain a concern. Some studies have identifie d the direct effect of DBS on science achievement among African American students, while observing a high correlation between real world problem solving skills and science achievement. The direct effect of real world problem solving skills and science achi evement needs to be studied across race/ethnicity, gender and SES. Such a study will not only confirm the efficacy of DBS in helping close the achievement gap but also in preparing students both for academics and their future workplaces. Despite the findin gs stated earlier however, not enough studies have been done to demonstrate the effects of DBS on the development of real world problem solving skills across gender, racial ethnic and SES and the consequent improvement in science achievement among high sch ool students Shepardson and Pizzini (1994), in one of a few such studies, reported that there is no significant difference in science achievement
29 between middle school boys and girls who learned science concepts through problem solving approaches. There i s the need to study this relationship among high school students where the gender gap widens even further. Similarly any differences in the effect of design based learning across gender, racial ethnic and socio economic groups thus needs to be established and subsequently understood. The purpose of this study is thus to investigate the differences in the real world problem solving abilities (and science achievement) of high school student s of different gender, race and SES after treatment with DBS in Tradit ional Chemistry Classes The achievement will be described by group means, variances, correlation ratios, etc. Results from this study will contribute to responses to calls by Ault (1994) for research into the nature of learning and problem solving in the area of science education, and by Thomas (2000) for more research on the effectiveness of PlBL. Research Q uestions In order to achieve the above objective the research questions th is study seeks to address we re: 1. Does DBS have an y effect on the problem solving competencie s of students in a high school traditional c hemistry class ? 2. Does the effect of DBS on problem solving competency depend on gender? 3. Does the effect of DBS on proble m solving competency depend on race ? 4. Does the effect of DBS on problem solving competency depend on SES?
30 5. Do es DBS have any effect on the c hemistry achievement of students in a high school traditional c hemistry class ? 6. Does the effect of DBS on chemistry achievement vary depending on gender ? 7. Does the effect of DBS on chemistry achievement vary depending on race ? 8. Does the effect of DBS on chemistry achievement vary depending on SES? 9. Is the problem solvi ng competency of students in a traditional c hemi stry class predictive of their c hemistry achievement? Definition of T erms Problem : A problem exists when there is an imbalance (referred to by Festinger, 1962 as roblem situation and the conceptual schema of the individual, which motivates the individual to find a solution. Ill structured problem : This is problem that addresses complex issues and thus cannot easily be described in a concise, complete manner. Furthe rmore, competing factors may suggest several approaches to the problem, requiring careful analysis to determine the best approach. Well defined problem : A well defined problem is identified by a testable goal state reachable from an initial state via one o r more possible paths. Problem solving competency : This is to confront and resolve real, cross disciplinary situations where the solution path is not immediately obvious and where the literacy domains or curricular areas that might be applicable are not within a single domain of mathematics, science and other domains.
31 Design Based Science (DBS) : Design Based Science is an inquiry based project based science pedagogy in which new scientific knowledge is c onstructed in the context of designing artifacts. Inquiry based learning (IBL) : A student centered, active learning approach focused on questioning, critical thinking, and problem solving. It involves making observations, gathering, analyzing and interpreting data in an attempt to answer a question/problem through the scientific process Problem Based Learning (PlBL) : An instructional method in which students learn through facilitated problem solving that centers on a complex problem, which does not have a single correct answer. Project based Learning (PjBL) : In project based learning learners are usually provided with specifications of a desired end product achieved by following correct problem solving procedure. Contextualization : A diverse family of instructional strategies designed to utilize particular situations or events that occur outside of science class or are of particular interest to students to motivate and guide the presentation of science ideas and concepts. Contextualizing often takes the form of real world examples or problems that are meaningful to students personally, to the local area, or to the scientific community. Technology : Technology as used in this work involves making modifications in the world to meet human needs. Real wor ld problem: an ill defined problem that calls on individuals to merge knowledge and strategies to confront and resolve a problem readily identifiable as arising
32 from real life. It is a situation where decisions are not clear cut, requirements can conflict Traditional Chemistry: This is an introductory chemistry taught to students taking chemistry for the first time and usually as a science requirement for high school graduation.
33 CHAPTER II REVIEW OF THE LITERATURE Introduction In the following sections, a critical overview of the current state of research in the areas of design based learning, and the development problem solving skills are provided. The overview also includes trends in gender and race ethnicity differences in science achievement Another section is dedicated to an analysis of the research findings on the impact of project science performance in terms of knowledge acquisition attitudes and metacognitive development. A brief overview of the foundations and rationale for using DBS and the models of DBS are presented. Studies of t he relationship between DBS and science achievement are elucidated. Theories of problem solving are reviewed as a lens through which the results of this study will be analyzed and interpreted. Foundations of Design Based Learning: Constructivism DBS was developed over the course of the 1999 2000 school year, by the Center for Highly Interactive Computing in Education (hi ce) at the University of Michigan DBS (like other project based pedagogies) is a detailed instructional model rooted in inquiry and which is consistent with the principles of instruction arising from constructivism (Savery & Duff, 1995; Krajcik, Czerniak, & Berger, 2002) A review of constructivism will therefore be helpful in order to appreciate the potential impact of DBS on knowledge acquisition and the development of problem solving skills. Constructivism is a philosophical view about how people come to understand or know. Each of us builds our own key to knowing by making sense of the world.
34 Constructivist theory in education comes primarily from the work of John Dewey (1938 ) and Jean Piaget (1977). Working from the idea that learne rs construct their own knowledge, both Dewey and Piaget contended that the stimulus for learning is some that learning should prepare a person for life, not simply for w ork. He proposed that learning should therefore be organized around the interests of the learner and that learning is an active effort by learners interested in resolving particular issues. Piaget, similarly, proposed that cognitive change and learning tak thinking, or scheme, leads to puzzlement instead of producing what the learner expects. Such puzzlement then leads to accommodation (cognitive change) and a new sense of equilibrium. Learners bring their own suppositions to learning experiences based on what fits their experiences. also thinking skills such as problem solving, reasoning, and knowing how to learn manifests in aspirations of recent educational policy directions (U.S. Department of Labor, 1991). During the Progressive Era, Dewey (1916) promoted the tackling of significant problems by students, as the ultimate way to engage learners in meaning making and the development of problem sol ving ability. Dewey (1943) believed that learning should be situated within the context of the community. In this perception knowledge acquired is meaningful and relevant. Therefore, cognitive change often results from interactions with other learners who may hold different understandings (Vol et, McGill & Pears, 1995). The associated social interactions may challenge current
35 views as well as allow them to test their current understandings to see how well they help them make sense of and function i n their world (Savery & Duffy, 1995). The idea that knowledge is constructed in the minds of learners has been extensively written about. For instance, Rorty (1991) described knowledge not as a ion and reality, but rather as a key fits a lock (Bodner, 1986). D esign B ased S cience Framework The presumption associated with DBS is that students need opportunities to construct knowledge by solving problems through asking and refining questions; designing and conducting investigations; gathering, analyzing, and interpreting information and data; drawing conclusions; and reporting findings (Rivet & Krajcik, 2004) In DBS, the design of the artifacts is not a culminating activity at the end of the curriculum, but rather it is the framework around which all the learning activities are organized. Any DBS instruction is characterized by five learning features (Singer, Marx, Krajcik, & Clay Chambers, 2000) These five features are a) active construction, b) situated cognition, c) community, d) discourse, and e) cognitive to ols. engaging students with the task in thought demanding ways such as explaining, gathering evidence, generalizing, representing, and applying ideas (Perkins, 1993). Situated cognition refers to student s making meaning through interactions between the world and others, and their interpretations of these interactions (Lave & W enger, 1991) within the
36 contexts of the discipline. These interactions engage students with a community of practitioners in the dis cipline (Perkins, 1993) It enables students learn ways of knowing what counts as evidence, and how ideas are shared within the culture of the discipline. Participation also brings students into the language and discourse of the community of practice ( Sing er, Marx, Krajcik, & Clay Chambers, 2000 ). Cognitive tools can extend what students can do and learn (Solomon & Perkins, 1989), in that they provide opportunities for students to visualize and explore phenomena that would not otherwise be possible in class rooms through manipulating multiple dynamic representations (Novak & Krajcik, 2005). The DBS learning cycle (Figure III) provides the framework for how classroom activities are structured. The cycle involves five stages. The first stage is contextualizatio n. Context supplies significance for the tasks the students will be facing and provides trigger points for action things the students can immediately begin to investigate (Kimbell, Stables & Green, 1996). The second stage involves background research, wh ich can be in the form of searching and gathering relevant information, Figure III. The Design based science learning cycle ( Fortus, Dershimer, Krajcik, Marx and Mamlok Naaman, 2005)
37 benchmark lessons in which the teacher presents new scientific concepts, reading selected materials, sharing on a whiteboard of data collected in group experiments and then collectively analyzing the complete database, teacher led demonstrations, computer based simulations of relevant phenomena, and virtual expeditions to examine appropriate primary sources. During the third stage every student generates their solution to the design problem and presents it to their group members. The group decides which of the suggested solutions they prefer or they might combine the solutions. The group then writes a justification for their decision. In the fourth stage, each design team constructs a model or modifies an existing model based upon the design solution they d ecided upon in the former stage. For example, they might construct a three dimensional model of a house or a cell phone antenna shield, or a cut away drawing of an electrochemical cell. In the final henever possible, and they are presented to the entire class in a pin up session (Kolodner, Stables, K., & Green, 1998; Schn, 1985). The models are laid out or hung up and the entire class moves from model to model, listening to the student offering their own critique. Review of Literature on D esign B ased S cience As the U.S. and other nations search for ways to close the achievement gap in STEM education in an effort to become more competitive in a global economy, the prospects of DBS should be seriously st udied Studies suggest that the adoption of DBS in science classrooms provide contextualized instruction, which benefits almost all students (Mehalik, Doppelt and Schuun, 2008 ; Rivet & Krajcik, 2004 ) particularly non
38 white students as well as students fro m low SES families. The subsequent improvement in science achievement has been documented. The effect of DBS on the improvement of problem solving skills, for the 21 st century, has also been suggested although more empirical studies need to be conducted. S ome of these studies are presented in the following paragraphs. http://www.nextgenscience.org ), a national initiative for new science standards, is due to the potential of DBS improving science achievement and problem solving skills. One of the most recent studies on project based inquiry instruction like DBS as a contextualizing instruction was conducted by Rivet & Kra jcik (200 4, 2008). Their study involved sixth through eighth grade students within the Detroit Public School System, and investigated the effects of project based curriculum materials on science achievement over a 10 week period. The curriculum materials used conte xtualized the learning of science in meaningful real world problems and engage students in science inquiry. Students in these schools were representative of the district, which was over 91% African American, with over 70% of students receiving free or redu ced price lunches, and 85% of students were below grade level on standardized eighth grade science assessment. The results show a strong and significant correlation between contextualizing score and all measures of learning. Similar suggestions have been m ade by other researchers, namely, calls for using Lee & Songer, 2003), making science ( Fusco, 2001), and promoting community connections, and building from local contexts ( Bouillion & Gomez, 2001). Also, Wisely (2009) investigated the hypothesis that low skilled students can learn more effectively and advance to college level programs more
39 readily through contextualization of basic skills instruction. The results of his study showed that participation in c ontextualization was associated with the completion of developmental education courses and the speed of entry into, and performance and completion of, college level courses. These positive effects were however, limited to non white students; no effects for contextualization were found for white students. Studies that are specific to DBS, but which were not focused on contextualization are consistent with the above findings. For instance, Mehalik, Doppelt and Schuun (2008) contrasted overall performances and by gender, ethnicity, and socioeconomic status (SES) for middle school students learning science through traditional scripted inquiry versus a design based inquiry (DBS). In their study the treatment group (DBS group ) had a higher proportion of students f rom schools in the low SES range (53 percent of 587 students v ersus 32 percent of 466 students in the control or scripted inquiry group ). The four lowe st SES schools in the district we re in the DBS group. SES categories we re based on the proportion of stud ents considered by the district to be economically disadvantaged, with the low group having schools with more than 66 percent of their students economically disadvantaged In terms of gender, the design group had a slightly higher proportion of female stud e nts (54 percent vs. 51 percent). As far as ethnicity, there was twice as much African American students (66 percent vs. 33 percent) The results suggested that the DBS approach for teaching science concepts had superior performance in terms of knowledge g ain achievements in core science concepts, engagement, and retention when compared to a scripted inquiry approach. The DBS approach was most helpful to low achieving African American students.
40 Since the 1970s, there has been an increasing emphasis on the use of the problem solving approach in science teaching. Some research evidence has shown that explicit sol Heyworth, 1998). Sternberg (1985) and Simon and Simon (1978) stressed that students meaningfully learn problem solving skills through concrete experiences. Visser (2002) c ompared the effects of problem based and lecture based teaching on student problem solving and attitudes in a high school genetics class. She found statistically significant differences (p<.05) in learning outcomes and motivation for students in the P l BL a nd Lecture/Discussion treatments. Problem solving skills remain vital to the success of students even today ( Partnership for 21 st Century Skills, 2009 ). Teachers can help students by providing explicit strategies that are procedurally structured to encour age students to become involved in their own learning and undertake the steps necessary to solve problems in science (Pizzini, Shepardson & Abell, 1989). It is suggested that DBS (and other project based inquiry approaches) can improve problem solving skil ls ( Fortus, Dershimer, Krajcik, Marx and Mamlok Naaman, 2005) however, the above studies do not show that the effects of DBS on science achievement is associated with a corresponding improvement in problem solving skills, nor are there enough recent empirical studies to establish this relationship. In addition, maj ority of studies on DBS that compare student demographics have been limited to elementary and middle schools and not high schools where achievement gaps tend to be wider. The current study therefore seeks to investigate the link between DBS and real world problem solving skills of students in high school.
41 Theories of Problem Solving Any given problem has at least three components: the givens, goal(s) and operations. The Givens are the facts or pieces of information presented to describe the problem. The Goal(s) is the desired end state of the problem while the Operations are the actions to be performed in order to reach the desired goal (Newell and Simon, 1972). To Con ceptually, there are two kinds of problem solving knowledge (Gagn, Yekovich, and Yekovich, 1993) : d eclarative knowledge, which is knowledge that something is the case, and procedural k nowledge, which is knowledge of how to do something. While declarative knowledge is knowledge of facts, theories, events, and objects, procedural knowledge includes motor skills, cognitive skills, and cognitive strategies. Both declarative and procedural knowledge are activated in working memory as problem solving occurs. Dec larative and procedural knowledge interact in a variety of ways during problem solving (Gagn, Yekovich, and Yekovich, 1993). theories that may be used to explain how differen t problems may be solved. An overview of some of these theories and how they relate to the current study may provide some insight in understanding and perhaps explaining differences that may be observed between the different groups of students being invest igated in this study. These theories include the a) constructivism, b) expert novice theory, and c) cognitive theory. Constructivism Learners bring their own suppositions to learning experiences based on what fits their experiences. Thus, constructivism ph ilosophy explains that knowledge is actively
42 constructed by an individual by comparing new ideas and concepts with their current knowledge (schema or mental models). Constructivist theory in education comes primarily from the work of John Dewey (1938 ) and Jean Piaget (1977). Dewey and Piaget contended that the stimulus for learning is some experience of cognitive conflict, Even the motivation to resolve the cognitive conflict may be spurred among others, by curiosity and personal or community needs. Hence, De wey argued that learning should prepare a person for life, not simply for work. He proposed that learning should therefore be organized around the interests of the learner and that learning is an active effort by learners interested in resolving particular issues. During the Progressive Era, Dewey (1916) promoted the tackling of significant problems by students, as the ultimate way to engage learners in meaning making and the development of problem solving ability Dewey (1943) believed that learning should be situated within the context of the community. In this perception knowledge acquired is meaningful and relevant. Therefore, cognitive change often results from interactions with other learners who may hold dif ferent understandings (Vol et, McGill & Pears, 1995) of a given situation. The associated social interactions may challenge current views as well as allow them to test their current understandings to see how well they help them make sense of and f unction in their competence in not only basic skills and personal qualities but also thinking skills such as problem solving, reasoning, and knowing how to learn are encouraged by recent educational policy di rections (U.S. Department of Labor, 1991).
43 In the current study students had to overcome specific constraints in order to meet needs that they had identified as relevant to their lives. In solving their problem, if the science concepts students were expec ted to learn were new to them, it was anticipated that students would tap into their prior knowledge (misconceptions or otherwise). If the problems were indeed relevant to their lives (as was the case), students would have the motivation to actively constr uct new knowledge as they solve the problem. Expert Novice Theory A review of the differences between expert and novice problem solvers may help understand differences that may be observed during the current study. Three attributes are commonly used to differentiate expert from novice problem solving characteristics (Muller, 1996) These attributes are a) conceptual understanding, b) basic, automated skills and c) domain specific strategies. Conceptual understanding refers to both the actual information in memory and the organization of that information in memory. Conceptual understanding is closely related to schema theory in which information is considered to be stored in memory as frameworks or structures that, once instantiated, provide a lens throug h which to view new information. Having a conceptual understanding of a domain means that an individual can make meaning of domain specific situations or problems, based on prior knowledge of that domain. Students may approach a problem with this attribute already acquired or should develop it during the second or third stages of the DBS learning cycle: background research and the development of personal and group ideas. Basic, automated skills in any domain are those that allow an individual to perform nec essary and routine operations without much thought. These skills are learned
44 to the extent that they become habitual and even unconscious, enabling individuals to operate quickly and accurately without over burdening their short term memories. This form of a utomaticity allows individuals to focus their attention on the more complex tasks associated with a specific domain and is a general attribute associated with experts During a DBS unit students who do not already have such skills will need sufficient time to develop some automaticity, hence the importance of project duration. Unlike basic, automated skills, which occur unconsciously and thus do not tax short term memory, domain specific strategies remain under conscious control. They are the processes and procedures in a domain that an individual, even an expert, must consciously think about in order to solve a problem. They are, in other words, the procedural knowledge associat ed with a domain. Expert novice differences have been studied and described within the context of these three attributes: Experts (a) exhibit better conceptual understanding of their domain; (b) use more automated skills and domain specific strategies; and (c) have a conceptual understanding that is declarative, while basic skills and strategies are procedural (Miller, 1996). During the current study, in following the DBS framework, students are expected to develop conceptual understanding of the problem they intend to solve. It is also anticipated that students will develop an appreciation for the power of eliciting both decla rative and automated skills in order to resolve a problem. In other words beyond the problem being resolved in the current study, students would recognize the need to look inward and elicit knowledge and skills required to solve any problem on hand. These students would move toward becoming expert problem solvers.
45 Cognitive Theory The cognitive theory is consistent with constructivism, in proposing that an inconsistency between behavior and beliefs motivates change. C ognitive psychologists, Wallas and Poly a, separately developed four stage models of problem solving. The four stages of problem solving identified b y Wallas were: a) preparation defining the problem and gathe ring information relevant to it, b) incubation thinking about the problem at a sub conscious level, c) inspiration having a sudden insight i nto the solution of the problem, and d) verification checking to be certain that the solution was correct solving p rocess included: a) understand the problem, b) devise a plan, c) carry out the plan, and (d) look backward (Ormrod, 1987). These two processes are very similar to each other and consistent with the five steps of the DBS learning cycle. Hence, assuming that the design challenge provides ample cognitive dissonance, the DBS learning cycle will provide the necessary paths needed by students in the current study to solve their problem of interest. It can therefore be anticipated that students who go through a unit by following the DBS learni ng cycle would ultimately become better problem solvers. Models of Problem S olving Instruction and Problem S olving Skills solving skills will improve after a treatment c knowledge of problem solving processes. In other words if a student does not have a clue how to approach the solution of real world problems even a strategy that has a potential of improving problem solving skills may not have a fair chance of successful ly solving the problem. The selection of a problem solving model of instruction is however one of the critical choices a teacher
46 must make as s/he prepares students for real world problem solving. Since the current study includes the provision of basic ins truction in problem solving strategies, a review of problem solving instruction models will provide a great reference for the model chosen and why. Three models commonly used are : a) Parnes Creative Problem Solving (CPS) process Define, Explore, Act, and Look (IDEAL) model and, c) the Search, Solve, Create, and Share (SSCS) model created by Edward Pizzini (1987), and Pizzini, Abell, & Shepardson (1988). The CPS model is a hierarchical pr ocess, with each step depending on the preceding step. The five steps of the CPS model are: a) fact finding, b) problem finding, c) idea finding, d) solution finding, and e) acceptance finding. The IDEAL and SSCS Models of Teaching Problem Solving The IDE AL model is also a five step hierarchical model which involves: a) identifying the problem, b) defining and representing the problem, c) exploring alternative strategies, d) acting on the strategies, and e) looking back and evaluating the effects. The SSCS model on the other hand is a four step cyclical model allowing for re entry into the various states of the model during the p roblem solving process (Figure IV ). The SSCS model reduces the other problem Fig ure IV The SSCS Problem Solving Cycle.
47 solving models into fewer steps; thereby, simplif ying the process (Figure IV ). Additionally, the SSCS model provides students with a n opportunity to communicate their results (Figure V) something that is missing in other problem solving models of instruct ion. Although the SSCS model is not a pre packaged curriculum, it can easily be incorporated into science instruction, providing a successful and creative way for students to learn science concepts and problem solving skills in science (Pizzini, Shepardson & Abell, 1989). The use of problem solving models in instruction calls for purposefulness on the part of the teacher and student. In any problem solving model of instruction the first level of learning includes problem recognition, the determination of in formation needed to solve the problem and where to obtain the information (Presseisen, 1985). Johnson, Ahlgren, Blout & Petit (1981) stressed the importance of how students search for an idea (concepts within the problem) that will assist them in understan ding the problem. Glatthorn and Baron (1985) emphasized the importance of the search process, as well as setting goals, searching for question asking ability is an essential aspect of problem solving. Through the above processes, students derive meaning from the problem (Anderson & Smith, 1981; Winne & Mark, 1977) and can take ownership of the problem Pizzini, Shepardson & Abel (1989) found that student ownership of the pro blem is one of the most essential variables resulting in successful problem solving. Providing students with the opportunity to select and pursue problems of concern and interest to them increases their motivation, persistence, and intensity to learn.
48 The SSCS model was developed on the premise that students meaningfully learn problem solving skills and science concepts through concrete experiences in solving problems in science, as evidenced by the literature. The SSCS model requires students to utilize various problem solving thinking skills identified by Stemberg (1985) and Fig ure V The SSCS Model as Related to the IDEAL and CPS Models (Pizzini, Shepardson & Abell, 1989) Presseisen (1985) (Figure VI ). The four phases of the SSCS model according to Pizzini, Abell, & Shepardson (1988) are expl ained as follows. The Search phase of the SSCS (SSCS) SEARCH SOLVE CREATE SHARE ( IDEAL ) IDENTIFY DEFINE EXPLORE ACT LOOK Recognize the problem What? Who? When? Where? How? Seek out additional information What else is necessary to know? Where can it be found? Listing problems/ideas from the situation. In wha t ways might I ..? State the problem. Generate listing of approaches or ideas to use. The plan what is it? I mplement the plan Create products or ideas. Self evaluation of processes and/or solution. Communication and interaction. Articulation of thinking. Massive feedback. Evaluation of solution. Generate potential search questions. Brainstorming, Observing, Analyzing, Classifying, Measuring and Describing. Questioning, S earching literature and Inquiring. Brainstorming, Hypothesizing, Predicting, Evaluating, Testing and Questioning. Brainstorming, Focusing, Inquiring, Comparing, Combing and Analyzing. Decision making, Defining, Creating, Designing, A pplying, Synthesizing, Testing and Verifying Accepting, Rejecting, Modifying, Refining, Completing, Troubleshooting Communicating, Displaying, Displaying, Promoting and Evaluating. Promoting, Displaying, Reporting, Verbalizing, Questioning, Reviewing and verifying. PROBLEM SOLVING MODELS ( CPS ) SITUATION FACT FINDING PROBLEM FINDING IDEA FINDING SOLUTION FINDING ACCEPTANCE FINDING QUESTIONS/TASKS/ APPROACHES PROCESSES/SKILLS
49 model involves brainstorming and other idea generating techniques that facilitate the identification and development of researchable questions or problems in science. Demonstrations, magazines and newspaper ar ticles, field trips, and science textbooks can lead students to the identification of researchable questions. In addition to identifying and developing questions and problems during the Search phase, students identify criteria for problem selection and sta te the question or problem in a researchable format. The Search phase assists students in relating the science concepts inherent in the problem to the relevant, existing science concepts embedded in their schema. This initiates the development of the probl em space or mental representation of the problem. The problem then is identified and defined by the student, based on his/her existing conceptual schemata. The Solve phase requires students to generate and implement their plans for finding a solution to t he problem they identified in the Search phase. During the Solve phase, the student reorganizes the concepts derived from the Search phase into a new "higher order" that identifies the method for solving the problem and the desired solution, completing the development of the problem space. It is during the Solve phase that students apply operator(s) to solve the problem. If the operation is unable to solve the problem or creates an intermediate state, the student may re enter the Search phase or continue to implement their plan (apply additional operators). The application of science concepts in the Solve phase provides meaning to the concepts as the student experiences the relationship between the concepts inherent in the problem, the concepts of the solved
50 SEARCH SOLVE CREATE SHARE Figure VI. Problem Solving/Thinking Skills within the SSCS Model (based on Sternberg, 1985 and Presseisen, 1985) Recognizing the problem Defining the problem Forming Mental Representation Selecting Problem Solving Procedure Allocating Time and Resources Forming Mental Representation Monitoring Selecting Method Monitoring Using Solved Problem Feedback Selecting Method Using Solved Problem Feedback M E T A C O M P O N E N T S Inductive Reasoning Spatial Visualization Deductive Reasoning Reading Inferring Alternative Solutions Inductive Reasoning, Spatial Visualization, Deductive Reasoning, Reading, Testing Alternative Solutions, Assembling of Facts, Eliminating Discrepancy, Determinati on of Additional Information. Inductive Reasoning, Spatial Visualization, Deductive Reasoning, Reading, Checking Solution for Generalization, Checking Alternative Solutions. Inductive Reasoning, Spatial Visualization, Deductive Reasoning, Reading, Reducing Level of Explanation. P E R F O R M A N C E C O M P O N E N T S C O M P O N E N T S A C Q U I S I T I O N K N O W L E D G E Selective Encoding Selective Comparison Selective Combination Selective Encoding Selective Comparison Selective Combination Selective Combination Selective Combination
51 problem, and the concepts applied to the problem, which are all linked to the s conceptual schema. The Create phase requires students to create a product that relates to the problem/solution, compare the data to the problem, draw generalizations, and if necessary modify. Students employ skills such as reducing data to simpler levels of explanation or eliminating discrepancies. The Create phase enables students to evaluate their own thinking processes. The outcome of the Create phase is the development of an innovative product, which communicates the results of the Search and /o r Solve phase to others. Self evaluation (thinking about thinking) is the dominant activity throughout the Create phase. The basis of the Share phase is to involve students in communicating their problem solutions or question answers. The product cr eated becomes the focus of the Share phase. The Share phase goes beyond simply communicating to students and others. Students articulate thinking through their communication and interaction, receive and process feedback, reflect on and evaluate solutions a nd answers, and generate potential Search questions. The generation of new potential Search questions occurs when an accepted solution creates a new problem, or when faulty reasoning or errors in the problem solving plan are discovered through external eva luation of the shared product. This enables the problem solver to identify problem solving skills which are in need of refinement, as well as initiate new Search questions. F ramework for Assessing Problem Solving S kills The development of a problem solvin g framework for assessing student performance is not easy. One of the reasons for this is that both individual and
52 collaborative problem solving are important for future learning, effective participation in society and for conducting personal activities. H owever, while measurement of individual problem solving competency may be achieved with greater ease, measurement of collaborative problem solving competency is beset with numerous challenges ( Reeff, Zabal & Blech, 2006 ). Foremost among these challenges ar e : how to a) assign credit to individual group members if this is required, b) account for differences across groups that may bias individual performance, and c) account for cultural differences in group dynamics. Most researchers who study problem solving in practice or research based settings agree that in describing student problem solving, the major focus is on describing the cognitive acts students make in addressing, solving and reporting solutions (OECD 2003). Cognitive acts therefore form the corner stone of the PISA 2003 problem solving assessment framework. Based on the definition of problem solving advanced earlier, the task to be performed by the student is shaped by its context(s), domain specific knowledge and Figure VII. PISA 2003 Probl em Solving Assessment Framework ( OECD, 2003 ) Context Personal life Work and Leisure Community and society Disciplines Mathematics, Science, Literature, Social studies, Technology and Commerce etc. Problem Types Decision making System analysis design Trouble shooting Item Solution Problem Solving Processes Understanding, Characterizing, R epresenting, Solving, Reflecting, Communicating Reasoning Skills Analytic reasoning Quantitative reasoning Analogical reasoning Combinatorial reasoning
53 strategies or skills required for solution. The PISA 2003 problem solving assessment framework (Figure VII) adopted for the current study, include the following components: a) problem types, b) problem context, c) disciples involved, d) problem solving pro cesses and e) reasoning skills. PISA 2003 chose to assess decision making, system ana lysis and design, and trouble shooting as problem types because they are generic problem solving structures that capture important aspects of everyday, real life analytical reasoning. They provide the structure within which problem solving is assessed with out necessarily placing emphasis on domain knowledge but rather on the process and skills. Sample problems are provided in appendix B. Decision making problems enable one to determine whether a student understands different alternatives and constraints in a problem situation and if the problem is different from a decision making problem in the sense that the former, a) requires a student to analyze a system or design a soluti on to a problem instead of selecting from a set of alternatives, b) involves a complex system of interrelated variables and a solution is not clear cut. Solution of system analysis and design problems requires the ability to identify the different variable s and how they affect each other. In the case of designs such relationships must be considered in optimizing the desired goal. Trouble shooting tasks on the other hand involve diagnosing, proposing a solution and sometimes executing this solution. Trouble shooting requires a) an understanding of how a device or procedure works and b) the ability to identify the relevant features for the task and to create or apply a representation in order to successfully solve the problem at hand. In all the three types of
54 solution to another person(s), form integral aspects of the problem solving competency. The PISA 2003 problem solving assessment involved problems embedded in real life settings assoc iated with personal life, work and leisure or community and society. While it may be necessary to identify the processes used by students as they solve problems the endeavor can be cumbersome if all problem types are considered. In the PISA 2003 framewor k problem solving processes considered are those based on cognitive analysis of the three types of problems assessed. The selection of these processes was informed by the work of cognitive psychologist such as Mayer & Wittrock (1996), Baxter & Glaser (1997 ) and Bransford, Brown & Cocking (1999) as well as by the seminal work of Polya (1945). These problem solving processes include: a) understanding the problem, b) characterizing the problem, c) representing the problem, d) solving the problem, e) reflecting on the solution, and f) communicating the problem solution. The framework does not assume that these processes are hierarchical or even necessary for the solution of any problem. Indeed it recognizes that a student may solve a problem in a way that transc ends the narrow linearity of the above model. In characterizing the problem students identify the variables in the problem and their interrelationships; decide relevant and irrelevant variables; construct hypotheses; and retrieve, organize, consider and cr itically evaluate contextual information. Representing the problem involves tabular, graphical, symbolic or verbal representation. Solving the problems, however, involves finding a solution that meets or exceeds the constraints and goals of the problem. Re flection involves examination and evaluation of the solution from different perspectives in an attempt to make it more acceptable as well as justifying these solutions.
55 The ability of a student to effectively use a given problem solving process does not only depend on his/her domain knowledge but also on the reasoning skills s/he possesses. In the PISA 2003 framework four reasoning skills are identified as being related to the problem types assessed. These skills include: a) analytical reasoning, which in cludes the application of principles from formal logic in determining cause and effect relations in order to select strategies, b) quantitative reasoning, which involve the ability to apply principles related to number sense and number operations, c) analo gical reasoning, which involves the ability of the student to tap into his/her previous knowledge in order to venture into the unfamiliar, and d) combinatorial reasoning, which enables a problem solver to identify or rank all combinations of factors in ord er to achieve a set goal. The above mentioned skills are all expected to be demonstrated in SSCS, CPS and IDEAL framework (Figures IV and V). In summary, the PISA 2003 problem solving assessment framework measures student problem solving competency, using three types of problem: decision making, system analysis and design, and trouble shooting. The problems are cross disciplinary to the community and society. Soluti on of a given problem may require the use of problem solving processes each of which depends of the reasoning skills of students. The problems are presented so that problem solving and not knowledge is being assessed and necessarily have to be demonstrated at all and if demonstrated, not necessarily in any specific order.
56 CHAPTER III MATERIALS AND METHODS Introduction Chapter three presents the research methodology of the present study. It begins by summarizing the p roblem and restating the research questions. This is followed by a description of the subjects of the study and an overview or the research design. The chapter ends with data collection and data analysis procedures and statement of the hypothese s. Restatement of the Problem The purpose of this study is to investigate whether DBS affects student problem solving skills and science achievement across student demographics in a high school in Denver, Colorado. Effects of DBS will be studied across gende r, race/ethnicity and SES among students in a traditional chemistry class. A strong correlation between problem solving skills and science achievement has been reported (OECD, 2003). The effects of pedagogies such as project based learning on science achi evement and problem solving skills have not been extensively studied among different groups of students. For example, studies show that the use of pedagogies such as project based learning, such as DBS, is associated with higher achievement in science in m iddle schools. However, more studies need to be conducted to investigate whether these pedagogies directly improve problem solving skills, with a consequent improvement in science achievement. With the achievement gaps in science not getting any better esp ecially in high schools it will be helpful to establish the benefits of these pedagogies to high school students of different gender, race and SES. For the scope of this study DBS is chosen because of its potential
57 in improving science achievement among no n white students and its increasing presence in recent science education studies. The current study tries to achieve its purpose by attempting to answer the following research questions: 1. Does DBS have an y effect on the problem solving competencie s of s tudents in a high school traditional c hemistry class ? 2. Does the effect of DBS on problem solving competency depend on gender? 3. Does the effect of DBS on problem solving competency depend on race ? 4. Does the effect of DBS on problem solving competency depend on SES? 5. Do es DBS have any effect on the c hemistry achievement of students in a high school traditional c hemistry class ? 6. Does the effect of DBS on chemistry achievement vary depending on gender ? 7. Does the effect of DBS on chemistry achievement vary depend ing on race ? 8. Does the effect of DBS on chemistry achievement vary depending on SES? 9. Is the problem solvi ng competency of students in a traditional c hemi stry class predictive of their c hemistry achievement? Hypothesis as Null Hypothesis The reformulati on of the rese arch questions as null h ypotheses will facilitate the examination of the statistical analyses and are indicated as follows:
58 1. DBS has no effect on the problem solving competencie s of students in a high school traditional c hemistry class ? 2. The e ffect of DBS on problem solving competency does not depend on gender? 3. The effect of DBS on problem solving competency does not depend on race ? 4. The effect of DBS on problem solving competency does not depend on SES? 5. DBS has no effect on the c hemistry ach ievement of students in a high school traditional c hemistry class ? 6. The effect of DBS on chemistry achievement does not vary depending on gender ? 7. The effect of DBS on chemistry achievement does not vary depending on race ? 8. The effect of DBS on chemistry ac hievement does not vary depending on SES? 9. P roblem solvi ng competency of students in a traditional c hemi stry class is not predictive of their c hemistry achievement? Participants For the purposes of the current study, the treatment and control groups were derived through the non probability means of purposive sampling. Krathwohl (2004) describes non probable purposive sampling as a technique that is convenient to the researcher but which lessens questions about the representativeness of the sample. The sub jects for this study include d students in four traditional Chemistry classes in an urban high school in the State of Colorado. Majority of the students in these classes were low performing students with a high proportion of African American and Hispanic s tudents
59 Based on state tests (CSAP) these students were a fair representation of the Afr ican American and Hispanic high school student population in the state of Colorado. The Colorado Department of Education reports that in 2011 60.4% of students from th is high school in grades 9 and 10 were below proficient in science. Among the Black students while 89% were below proficient in math the number was 72% among Hispanics. Also, while 62% of Black students were below proficient in reading this number was 50% among Hispanic students. Only 23% of Black students were proficient in writing while 30% of Hispanic students were proficient. With a student population of 1610 in 2011, this school was 41.7% Black, 29.3% White, 23.3% Hispanic, 4.8% Asian and 0.9% American Indian. The percentage of students that was eligible for free and reduced lunch was 51.8%. In contrast to the preceding statistics 67% of Asian students and 72% of White students in the school were at or above proficient in math. In reading 84% of White a nd Asian students were at or above proficient while in writing 81% Asian students and 78% of White students were at or above proficient. Four equivalent parallel traditional c hemistry classes of ninety five (95) 10 th and 11 th grade students were invited to participate in the study. E ighty two (82) students participate d in this study. The treatment group comprised of 36 students (16 females and 20 males) while the control group was made up of 46 students (23 females and 23 males). The composition of the trea tment group by race was as follows: Asians 1; Black 19; Hispanic 14; Native American 2; White 0. The control group however consisted of: Asians 1; Black 21; Hispanic 20; Native American 1; White 3.
60 Research D esign A quasi experimental pre/posttest research study with non randomized sampling was conducted. Non randomized assignment of participants to groups deals with intact groups and thus does not disrupt the existing research setting. This reduces the reactive effects of the experime ntal procedure and, therefore, improves the external validity of the design (Dimitrov & Rumrill, 2003) In order to ensure that any treatment and control group created were similar in their chemistry knowledge, district chemistry assessment data were revie wed before these groups were formed. The Denver Public Schools District c hemistry assessment class average scores (ranging from 20% to 30%) showed that all four classes were similar. Subsequently two of the classes were randomly assigned to the treatment g roup while the other two classes are assigned to the control group. The treatment group learned anticipated chemistry concepts through DBS instruction (treatment) while the control group was expected to learn the chemistry concepts but through traditional methods of instruction. Pretests and posttests on problem solving competency and chemistry achievement were given to both groups to determine the effect of the treatment on problem solving competency and chemistry achievement. Problem solving competency wa s measured for both groups using the PISA 2003 problem solving assessment protocol. After 2003 the next focus on PISA problem solving was 2012, after science in 2006 and reading in 2009 (OECD, 2003). At the time of this study the PISA 2012 problem solving items were not available, hence the use of the 2003 learned from the chemical energy/heating and cooling unit was also assessed before and after treatment. The unit included th e three c ore chemical concepts: atomic interactions,
61 reactions and energy changes during reactions In this design the method of instruction (DBS) was the independent variable. The dependent variables on the other hand were problem solving competency and k nowledge of chemistry concepts (appendix F). Sample questions for the assessment of knowledge of chemistry concepts are shown in appendix E. Weekly assessment data were collected only as a way of monitoring student progress and to keep students focused. Th is is because DBS, like other forms of PjBL is self directed and students are likely to be at different stages of learning during the project Procedures The Treatment Before the implementation of the unit in both groups the teacher elicited ideas and understanding currently held by students about the chemistry and design of heating and cooling systems For meaningful learning to occur, instruction should begin with an of the science concepts t o be addressed (Taber 2003 ). The treatment provided opportunities for students to learn about the design process and presented the problem that needed to be resolved through scientific principles from the heating/cooling unit and the design process. The co ntrol group was neither engaged in the design process nor required to solve a design challenge problem. The treatment was the DBS Heating/Cooling System (Chemical Energy) unit used by Apedoe, Reynolds, Ellefson and Schunn (2008). As a first step, the tre atment group watch ed the video clip ed the latter video, they were expected to document what they learn ed thou ght he
62 World War II by allied forces. They were expected to discuss if and how he used the engineering design process as described in the first video. This activity enable d students to become familiar with key design ideas such as needs, requirements and functional decomposition. It also provide d a context for student work during the 12 week unit as well as help ed students see how scientific principles and engineering design go hand in hand during the search for solution to a problem. Students in the control group did not watch the video on engineering design. However, they watched the video on bombing ed to ending World War II. The second main step involve d students in the treatment groups working in groups to brainstorm their needs for a heating and cooling system in their own lives. This opportunity was expected to create a personal motiva tion for th e design work and mad e the topic relevant across ethnicity, gender, and other micro cultures and help ed students see the relevance of science and technology in their daily lives (Apedoe, Reynolds, Ellefson and Schunn, 2008). When groups had identified thei r needs they were then required to design a heating or cooling system that relies on chemical energy to meet the need(s), using concepts/knowledge from the unit. For example, they could create systems that would a) help keep them cool in the summer when th ey are playing sports, b) prevent them from having to sit on a cold toilet seat during cold weather and c) keep them cool Students also had the option of designing and building a prototype of a toy. ork involve d three aspects: a) planning the design, b) deciding and studying the chemical reactions that meet their specific needs as
63 well as how the reactions will be contained and c) making a three to five minute presentation to their classmates who repr esented the board of directors of a firm that was looking for ideas to invest in. During the next step students thought about other examples of heating and cooling systems from their everyday lives to consider the parts of these systems that make them work The students did this to develop suggestions for solution to the problem on hand. Students in the control group were not assigned a design challenge. Instead, they learned chemistry concepts (appendix F) through traditional methods such as lecture, word problems and scripted inquiry. The identification of other heating and/or cooling systems was intended to help students understand that systems are made of subsystems, which in turn can be broken down in order to understand how they function. Subsystem decomposition is critical for engineering design (Bradshaw 1992 ; Ulrich and Eppinger 2004 ). While students suggested solutions to the problem on hand the teacher ensured that students stayed with the use of required chemistry conc epts (appendix F). Although actual heating and cooling systems tend to have more than two subsystems, as a result of limited time for the unit and the emphasis on science concept learning students were encouraged to work towards a two subsystem design: the reaction subsystem where energy is produced and a container subsystem, which manages the transfer of energy in the system. Students were to spend more time on the reaction and container subsystems stage than the other two (planning the design and presenti ng the design) since most of the chemistry concepts will be learned during this stage. Figure VIII summarizes the process of planning the design. To illustrate the above process one of the student projects is described as follows. After deciding to design a Representing the Learning Cycle Properties of matter Kinetic Energy Energy Transfer Thermal conductivity 1. Needs & Requirements 2. System decomposition PLANNING THE DESIGN REACTION I CONTAINER PRESENTING THE DESIGN Represents the Learning Cycle
64 toilet seat warmer the group planned their design by making drawings of their design. As part of their design they identified their reaction sys tem, which would generate the heat for their warmer and their container system, which would comprise materials that will hold the chemical(s) and/or transfer the generated heat. During reaction I the group researched the source of chemical energy during ch emical reactions. The goal was to understand the effect of molecular shape, size and bond type as well as other factors in Figure VIII Aspects of Heating and Cooling Unit (Apedoe, Reynolds, Ellefson and Schunn, 2008 ) determining the amount of heat change in a chemical reaction. During reaction II the group applied knowledge gained in system I to generate chemical energy. For example, Attractive Forces Endo/Exo Reactions Particulate nature of matter Properties of matter Particulate rearrangement Q = mc T REACTION I REACTION I I CONTAINER SYSTEM 1. Needs & Requirements 2. System decomposition PRESENTING THE DESIGN Properties of Matter Kinetic Energy Energy Transfer Thermal Conductivity
65 the group investigated factors affecting the quantity of heat generated. The group the n used properties of materials (metals, plastics, leather, etc.) to decide where chemicals would be stored as well as what materials to insulate and transmit heat. According to Apedoe, Reynolds, Ellefson and Schunn ( 2008 ), the reaction subsystem (Reactio ns I and II) and the container subsystem address different, chemistry concepts and as a result if students went through them more times they gained a deeper understanding of the relevant chemistry concepts. As shown in appendix F, each of the subsystems in deed addresses one big idea. During each lesson students engaged in activities that challenge them to work with one or two of these key concepts as they discuss them with their teacher. The lessons build upon each other and culminate in a The Design Science Cycle Activities within each subsystem (Reactions I a nd II) are structured cyclically so that students move from design goals to science goals and back to design goals. This cycle, made room for whole class discussions, team activities, and individual activities intended to maximize the learning of science c ontent as well as design and science processes. Thus, this cycle can be called the design science cycle (or the learning cycle), similar to the legacy cycle (Brophy & Bransford 2001). Students start ed th e design science cycle (Figure IX ase node. They discuss ed reasons for their outcomes as a class or in group during During this stage, students address ed questions such as: a) W as our design successful? b)
66 W hat factors were important for the success of the design? c) W hat factors may have influenced the failed Figure IX. The Design Science Cycle ( Apedoe, Reynolds, Ellefson and Schunn, 2008) performance of the design? Students propose d ways to systematically test some of their d the results from their experiments and discuss ed their findings as a class /group during theory, or trend. Finally, students arrive d at Connect to Big Idea where they link ed their design to the key science concept(s) that can be used to improve its performance. Overall, the Design Science Cycle is structured to maintain a motivating design storyline while preventing students from wasting time floundering and encouraging them to focus attention on the selected core concepts. At the end of the reaction subsystem students built on their ideas developed by consideri ng the material(s) for the container, and the properties such as density, thermal conductivity, melting point, specific heat capacity, etc. of the material. Student groups presented their design to the class in ways
67 they found convenient and effective. App endix G summarizes the activities for the treatment and control groups. While students in the treatment group learned about heating and cooling through a DBS unit over a 12 week period ( Apedoe, Reynolds, Ellefson and Schun n 2008) the control group learned about heating and cooling through lecture, word problems and scripted inquiry, where students were given steps and guidelines to follow to complete the experiments. Data Sources Student biographical information (gender, race/ethnicity, SES) was the first set of information to be collected, before the start of the unit, by means of a survey from both determined using the BSMSS (Appendix A). The BSMSS incorporates a student 's parent's educational attainment and occupat ional prestige as well as hi s /her own educational attainment and occupational prestige (Barratt, 2006). Pretests of problem solving abili ty and chemistry concept knowledge in heating and cooling were administered to both the control and treatment groups. A sample of real world PISA 2003 problem solving items, that were use d for the pre test and post test can be found in Appendix B The prob lem solving items assess problem solving competency in the following areas: decision making, system design and analysis, and troubleshooting. The chemistry concepts inventory is a twenty two item test that assesses understanding in chemical and physical ch anges. Teacher as Researcher Whenever the teacher of a given class is also a researcher in the same class there are some advantages and disadvantages. An advantage of a teacher as researcher
68 arrangement is the fact that the research environment is kept in tact. The teacher student and student students relationships in the room are not interfered with. Students may more likely be open to sharing their challenges and inclinations with the teacher than an outsider. Another advantage is that the teacher researc her usually has a long term experience of the setting being studied and therefore know the history and information needed to understand what is going on in the setting (Hammersley, 1993). On the other hand, there is a risk of the teacher researcher being b iased in his/her judgments towards students and different situations. Therefore, in order t o ensure inter rater reliability and minimize any biases that may result from the teacher as researcher two peer observers were requested to visit two class periods each. They also reviewed sample student responses (problem solving assessment and Chemistry Concepts test) graded by the class teacher (researcher). Instrumentation The independent variable to be studied is participation in DBS unit (across gender, race and SES). The instruments that were used for data collection from the treatment and control groups are described below. The s ocioeconomic status of students was measured using the Barratt Simplified Measure of Social Status (BSMSS) shown in Appendix A The BSMSS is a version of the A. B. Hollingshead (1975) four factor index of social status, with a reliability of 0.85 and demonstrated to be a valid measure of SES ( Cirino, Chin, Sevcik, Wolf, Lovett & Morris 2002). The BSMSS accounts for an individual's pa rent's educational attainment and occupat ional prestige and combines them with the individual's own educational attainment and occupational prestige (Barratt, 2006) BSMSS scores range from 8 (lowest SES) to 66 (highest SES). This measure of
69 SES is more li kely to be accurate than student eligibility for free or reduced lunch since students living in single parent families may have an unfair advantage of being classified as belonging to free or reduced lunch program while support from the other parent may no t be considered in determining eligibility The score that results from the BSMSS is ordinal only and is sufficient for regression analysis or for creating SES groups based on the data collected. In order to answer the first research question, the effect o f DBS on the problem solving competency of a student was measured by administering a pretest and posttest using the PISA 2003 problem solving sample problems (Appendix B). The problems that were used in assessing problem solving competency include d 19 items as in PISA 2003: 7 decision making items, 7 system analysis and design items and 5 trouble shooting items. In this assessment, more emphasis is placed on decision making followed by system design, with troubleshooting being allocated the least sc ores. Across the three problem types more difficult problems are scored at the middle of the scale (Appendix D). Some questions require students to construct their own responses, either by providing a brief answer from a wide range of possible answers (sho rt response items) or by constructing a longer response (open constructed response items), allowing for the possibility of divergent, individual responses and opposing viewpoints. Other parts of the test are based on students constructing their own respons es, but based on a very limited range of possible responses (closed constructed response items), which are scored as either correct or incorrect. The remaining items are asked in multiple choice format, in which students either make one choice from among f our or five given alternatives (multiple choice items) or a series of choices by circling a word or short phrase (for
70 choice items). The reliability of these assessments is 0.87 (OECD, 2005) The features of each problem type (goals, processes and sources of complexity) compared in Appendix C, serve as the basis for establishing a scale to describe increasing student proficiency in problem solving. The problem solving items would be scored u sing the rubric associated with each item (Appendix B), while the skill level of students would be determined using the PISA 2003 problem solving scale (Appendix D). The PISA problem nd, characterize, represent, solve, reflect on and communicate their solutions to a problem. competency) of problem solving. The three levels of proficiency in problem solv ing are: a) level 1 bas ic problem solvers, b) level 2 reasoning, decision making problem solvers, and c) level 3 reflective, communicative problem solvers. Level 3 students score above 592 points on the PISA problem solving Scale. Typically, these s tudents are able to analyze a situation and make decisions, as well as think about the underlying relationships in a problem and relate them to the solution. These students are systematic problem solvers and construct their own representations to help them solve problems. They verify that their solution satisfies all requirements of the problem. These students communicate their solutions using accurate written statements and other representations. Students at the top of Level 3 can cope with multiple interr elated conditions that require them to work back and forth between their solution and the conditions laid out in the problem.
71 Students at Level 3 are also expected to be able to successfully complete tasks located at lower levels of the PISA problem solvi ng scale. The Students at Level 3 therefore possess the following skills: monitoring variables, accounting for temporal restrictions, and other constraints; troubleshooting, analytical, decision making, visualization, evaluation of their solution, effectiv e handing of the complexity of multiple interrelated conditions and effective communication. These are problem solving skills that are associated with all three levels (metacognitive, performance and knowledge acquisition) of the SSCS Model (Figure VI ). S tudents proficient at Level 2 score between 499 to 592 points on the problem solving Scale. These students use reasoning and analytic processes and solve problems requiring decision making skills. They can apply various types of reasoning (inductive and de ductive reasoning, reasoning about causes and effects, or reasoning with many combinations, which involves systematically comparing all possible variations in well described situations) to analyze situations and to solve problems that require them to make a decision among well defined alternatives. To analyze a system or make decisions, students at Level 2 combine and synthesize information from a variety of sources. They are able to combine various forms of representations (e.g. a formalized language, nume rical information, and graphical information), handle unfamiliar representations (e.g. statements in a programming language or flow diagrams related to a mechanical or structural arrangement of components) and draw inferences based on two or more sources o f information. Students at Level 2 are also expected to be able to successfully complete tasks located at Level 1 of the PISA problem solving scale.
72 Students proficient at Level 1 score between 405 to 499 points on the problem solving scale. They typically solve problems where they have to deal with only a single data source containing discrete, well defined information. They understand the nature of a problem and consistently locate and retrieve information related to the major features of the problem. Stu dents at Level 1 are able to transform the information in the problem to present the problem differently, e.g. take information from a table to create a drawing or graph. Also, students can apply information to check a limited number of well defined condit ions within the problem. However, students at Level 1 do not typically deal successfully with multi faceted problems involving more than one data source or requiring them to reason with the information provided. Students below level1, with scores of less t han 405 points, are weak or emergent problem solvers. They consistently fail to understand even the easiest items in the assessment or fail to apply the necessary processes to characterize important features or represent the problems. At most, they can dea l with straightforward problems with carefully structured tasks that require the students to give responses based on facts or to make observations with few or no inferences. They have significant difficulties in making decisions, analyzing or evaluating sy stems, and trouble shooting situations. The effects of DBS on student proficiency in the science concepts and knowledge gain intended by the Heating and Cooling System unit would be assessed by 24 questions taken from the Chemical Concept Inventory (CCI) ( American Chemical Society 2001 ) (Eubanks an Eubanks, 1993). Samples of these questions are shown in appendix E. The
73 reliability of CCI is 0.71 with proven validity (Krause Birk, Bauer, Jenkins, & Pavelich, 2004). Data Analysis In order to answer research questions one to eight, data collected were analyzed by ANCOVA, controlling for pretest scores. ANCOVA was used because it yields more powerful results, meaning there was a higher probability of finding group differences if indeed any difference existed and is not associated with an inflated level of significance (Dimitrov & Rumrill, 2003). Other analyses that could have been perf ormed, namely: a ) Analysis of varia nce (ANOVA) on gain scores, b) ANOVA on residual scores, or (d ) Repeated measures ANOVA. T he use of pretest scores in these methods helps to reduce error variance, thus producing more powerful tests than designs with no p retest data (Stevens, 1996). ANOVA on residual scores was not used because: a) when the residuals are obtained from the pooled within group regression coefficients, ANOVA on residual scores results in an inflated level of significance and b) when the reg ression coefficient for the total sample of all groups combined is used, ANOVA on residual scores yields an inappropriately conservative test (Maxwell, Delaney, & Manheimer, 1985). Also, repeated measures ANOVA was not used because according to Huck & McLe an (1975) and Jennings (1998) the results provided by repeated measures ANOVA for pretest posttest data can be misleading. Specifically, the F test for the treatment main effect (which is of primary interest) is very conservative because the pretest scores are not affected by the treatment. Hence for the analysis of pretest posttest designs Dimitrov & Rumrill (2003) recommend one way ANOVA on gain scores or, even better, ANCOVA with the pretest scores as a covariate.
74 Analysis of Cov ariance (AN COVA), was therefore performed (controlling for pretest scores) to determine if groups (including gender, race ethnicity and SES ) differ ed significantly in gains in problem solving competency and chemistry achievement in Chemical h eating/Cooling pursuant to the administration of the DBS instruction For the ANCOVA procedure, the effect sizes are noted and discussed using the Eta 2 value. The Eta 2 value is the proportion of variation in the dependent variable (problem solving competency or Chemistry concept) that is attributable to the DBS instruction and other factors such as gender, race and SES. The dependent variabl es (within group) in the study we re : a) Problem solving competency, and b) Chemistry achievement, considered one at a time. Both of these are s cale quantities. The main independent variable (between group) is the treatment (with 2 levels treatment and control). The other predictors are: gender race and SES. All predictors we re nominal variables The third research question sought to investigat e whether the problem solving competency of a student was predictive of his/her achievement in chemistry. This question was answered by performing a correlation for all participants to determine if students who did well on the problem solving assessment al so performed well on the chemistry concepts inventory and vice versa. Summary T he research methodology o f the study involved a quasi experimental pretest/posttest design with a non randomized sample, comprising 10th and 11 th grade students in four traditi onal chemistry classes. Two main groups were compared, namely treatment and control groups. The treatment was DBS instruction on chemical heating/cooling unit. The study sought to investigate: a) the effects of DBS on problem
75 solving competency across gend er, race and SES, b) the effects of DBS on chemistry achievement across gender, race and SES, and c) whether problem solving competency is predictive of chemistry achievement. Analysis of Covariance (ANCOVA) was the preferred analysis due to its reduction of systemic bias caused by group differences in pretest scores and the higher power associated with its results. The g eneralizability of the findings of this study is restricted to: a) students of traditional chemistry who are taking chemistry for the firs t time, b) students who are generally not highly motivated to study chemistry and c) schools in an urban school district with similar student characteristics as the one used in this study. This is because t esting one school makes generalization difficult since the individual school tested may generate better or worse results for students using that particular educational instruction The students in one school may be from a completely different soci oeconomic background or culture and therefore cannot be a representative sample of the population.
76 C HAPTER IV DATA ANALYSIS AND RESULTS Introduction The purpose of this study was to investigate the effects of DBS instruction on the problem solving competency and the concepts of chemical energy among different groups of students. Student groups used in the analysis included gender and race. The relation ship between SES and problem solving competency was also considered. A treatment group of 33 students were taught a twelve w eek c hemical energy unit through lessons in which students designed solutions to real world problems as well as design their own inv estigations. T Do the effects of DBS on problem solving competencies of students in a high school Traditional Chemistry class vary depending on gender, race of DBS on science achievement of students in a high school Traditional Chemistry class vary A control group of 41 students studied the same unit in traditional classroom settings. This chapter presents results from the quantitative methods used in the study. To investigate research questions one to eight a n Analysis of Covariance (ANCOVA) procedure was performed using pretes t scores as covariate, to compare the means between the treatment and control groups. The analysis was conducted to ascertain group differences attributable to DBS. For the ANCOVA procedure, the effect sizes are noted and discussed using the Eta 2 value. The Eta 2 value is the proportion of variation in
77 the dependent variable (problem solving competency or Chemistry concept) that is attributable to the DBS instruction and other factors such as gender, race and SES. Data Analysis and Results Effects o f DBS on Problem Solving Competency across Gender, Race and SES Research question one: Comparison of treatment and control groups. The first research question is as follow: Does DBS have any effect on problem solving competencies of students in a high sch ool traditional chemistry class? In order to answer the first part of this question, an ANCOVA was conducted to determine if there were significant differences between the means problem solving competency scores the treatment and control groups. The result s show that there was a statistically significant difference between the treatment and control groups in their problem solving competency after the treatment (DBS instruction). This significant difference was observed across all three aspects of problem so lving measured in the study. The ANCOVA results comparing the mean total problem solving scores of the treatment and control groups are presented as follow. The following assumptions were tested, a) independence of observations, b) normal distribution of t he dependent variable, c) homogeneity of variance, d) linear relationships between the covariate and dependent variable, and e) homogeneity of regression slopes. All assumptions were met. The results indicate that after controlling for the pretest scores o n problem solving competency, there was a significant difference between the treatment and control groups in problem solving competency. Table II shows that the problem solving competency of the treatment and control groups were significantly different, F 2 = .32. Thus
78 32 percent of the variance in problem solving scores is explained by the treatment. Table I presents the means and standard deviations for the two groups on problem solving competency, before and after controllin g for problem solving pretest scores. Table I Adjusted and Unadjusted Group Means and Variability for Problem Solving Competency Using Problem Solving Pretest Scores as Covariate Unadjusted Adjusted N M SD M SE Control group 41 3214.0 7 2056.72 2831.31 285.93 Treatment group 33 5139.88 1934.76 5305.72 313.18 Table II Analysis of Covariance for Problem Solving Competency as a Function of Group, Using Problem Solving Pretest Scores as Covariate Source df MS F p eta 2 Problem solving pretest 1 47604131.04 21.90 Group 1 52939811.61 24 Error 71 2174182.62
79 Figure X M ean Problem Solving Scores for T reatment and Control Groups Figure XI Mean Problem Solving Scores (Pretest and Posttest) for T reatment and Control Groups Figures X and XI depict the effect of DBS in improving overall problem solving competency. Figure X is compares the estimated mean problem solving competency ( 0 Control group 1 Treatment group) 0 Control group 1 Treatment group 0 Control group 1 Treatment group
80 scores of the control and treatment groups with pretest scores adjust ed (controlled for). Figure XI on the other hand compares the pre/posttest mean problem solving competency scores of the control and treatment group. The latter graph shows that while the pretest scores were similar for the two groups, the posttest scores for the treatment group are much higher than those of the control group. Thus the DBS instruction appeared to have affected the problem solving competency of students. The problem solving items comprised of three types of problems, namely, decision making (DM), system analysis and design (SAD), and troubleshooting (TS). It may be necessary to dig deeper to determine whether the above effect of DBS on problem solving competency was evident in all three aspects of problem solving. The ANCOVA results for the subtotals for each of the three areas of problem solving competency measured, controlling for pretest scores for each aspect of problem solving, indicated statistically significant differences between the treatment and control groups. For decision making c 2 = .21. Table III presents the mean and standard deviations for the control and treatment groups on decision making competency before and after controlling for decision making pretest scores.
81 Table III A djusted and Unadjusted Group Means and Variability for Decision Making Competency Using Decision Making Pretest Scores as Covariate Unadjusted Adjusted N M SD M SE Control group 41 1426.37 988.90 1415.57 108.62 Treatment group 33 2101.36 766.98 2114.78 121.06 Table IV Analysis of Covariance for Problem Solving Competency as a Function of Group, Using Decision Making Pretest Scores as Covariate Source df MS F p eta 2 Decision making pretest 1 23603294.30 48.81 Group 1 8934757.71 Error 71 483627.45 The ANCOVA results for system analysis competency, show that F (1, 71) = 2 = 14. Table V presents the mean and standard deviations for the
82 control and treatment groups on system analysis and design competency before and after controlling for system analysis and design pretest scores. Table V Adjusted and Unadjusted Group Means and Variability for System Analysis Competency Using System Analysis and Design Pretest Scores as Covariate Unadjusted Adjusted N M SD M SE Control group 41 1326.68 1020.32 1310.07 127.55 Treatment group 33 1937.73 986.69 1953.37 142.18 Table VI Analysis of Covariance for System Analysis Competency as a Function of Group, Using System Analysis and Analysis Pretest Scores as Covariate Source df MS F p eta 2 System Analysis pretest 1 25458677.26 38.19 Group 1 7677008.59 Error 71 666715.50
83 The ANCOVA results for troubleshooting also show a significant difference between the treatment and contr ol groups for troubleshooting competency, F (1, 71) = 2 = .18. The assumption of homogeneity of variances was violated. However, because the cell sizes (41 and 33) were similar this violation did not present an issue (Leech, Barrett and Morgan, 2008). Table VII presents the mean and standard deviations for the control and treatment groups on troubleshooting competency before and after controlling for troubleshooting pretest scores. Table VII Adjusted and Unadjusted Group Means and Va riability for System Analysis Competency Using Troubleshooting Pretest Scores as Covariate Unadjusted Adjusted N M SD M SE Control group 41 497.10 549.66 478.80 102.50 Treatment group 33 1076.42 749.82 1099.16 114.67
84 Table VIII Analysis of Covariance for System Analysis Competency as a Function of Group, Using Troubleshooting Pretest Scores as Covariate Source df MS F p eta 2 Troubleshooting pretest 1 433749.75 1.04 .30 .014 Group 1 6570140.63 Error 71 417497.77 The above analysis suggests that DBS improves all three aspects of problem solving competency measured in the study. This, thus provides an answer to research question one, thereby disproving the null hypothesis that the DBS has no effect on problem solving competency. Research question two: problem s olving c ompetency across g ender The preceding section suggests that there is a significant effect of DBS on problem solving competency. This section is dedicated to analyzing the data for gender differences. To search for any significant gender differences an ANCOVA was conducted to dete rmine whether there were differences between the average total problem solving scores of male and female students in the treatment and control groups, after controlling for problem solving pretest scores. Group and gender were used as fixed factors. The fo llowing assumptions were once again tested, a) independence of observations, b) normal distribution of the dependent variable, c) homogeneity of variance, d) linear
85 relationships between the covariate and dependent variable, and e) homogeneity of regressio n slopes. All the assumptions were met. The results indicate that the problem solving competencies of the male and female students in the treatment group on the one .05, eta 2 = .099. The means and standard deviations for the two groups are presented in Table IX. Table IX Adjusted and Unadjusted Gender Means and Variability for Problem Solving Competency Using Problem Solving Pretest Scores as Covariate Unadjusted Adjusted N M SD M SE Control group 41 3214.07 2056.72 2831.31 285.93 Female 22 2882.09 2066.21 2510.45 418.31 Male 19 3598.47 2031.94 3216.34 377.05 Treatment group 33 5139.88 1934.70 5305.72 313.18 Female 15 5731.07 2069.17 5802.91 446.13 Male 18 4647.22 1718.98 4709.09 443.54
86 Table X Analysis of Covariance for Problem Solving Competency as a Function of Gender Using Problem Solving Pretest Scores as Covariate Source df MS F p eta 2 Problem solving pretest 1 47604131.04 21.90 Gender 1 1679 23.86 .08 .782 .00 Group*Gender 1 Error 51 2174182.62 The treatment (DBS) therefore explained about ten percent of the variance in female and male problem solving scores between the treatment and control groups. Figure XI shows that both males and females benefited from the treatment, with females in the treatment group a slight edge over males. Males in the control group performed better than females on the problem solving compet ency protocol. The interaction therefore indicates a small benefit of DBS for female students over male students on problem solving competency
87 Figure XII Effects of DBS on Problem Solving Competency across Gender The ANCOVA results for the previous section indicate a significant difference between the treatment and control groups in problem solving competency. Thus, DBS appeared to have improved the problem solving competency of students. In an attempt to answer research question one, this section sought to investigate if this effect of DBS was significantly different when females and males were compared. The ANCOVA results suggest that there was a significant difference between m ales and females in the treatment group compared to those in the control group: although both males and females benefited from DBS, females (in the treatment group) appear to have benefited more from DBS instruction than males (in the treatment group). Thi s 0 Female 1 Male ( 0 Control group 1 Treatment group)
88 statistically significant, interaction rejects the second null hypothesis (H o = males = females ). Research question three: p roblem s olving c ompetency across r ace This section presents the results of analyses intended to answer research question thre e: Are the observed effects of DBS different for students of different races? An analysis of covariance was used to assess whether there were significant differences between the average total problem solving scores of Black and Hispanic students, after con trolling for problem solving pretest scores. The small number of Asian students (one each in control and treatment groups), White students (all three in control group) and Native American students (one in control group, two in treatment group), warranted t heir exclusion from the analyses. The following assumptions were tested a) independence of observations, b) normal distribution of the dependent variable, c) homogeneity of variance, d) linear relationships between the covariate and dependent variable, and e) homogeneity of regression slopes. All assumptions were met. The results indicate that the problem solving competencies of Black and Hispanic students in the treatment and control groups were not significantly different, F (1, 51) = 1.07, p = .305, eta 2 = .02. The eta 2 = .097. Thus the combination of group, gender and race explains about ten percent of the difference in variance of problem solving competency between the control and treatment group. These results, while accepting the third null hypothesis suggests significantly different effects of DBS on Hispanic males and Black males. The means and standard deviations for Black and Hispanic females and males in both treatment and control groups are presented in Table XI.
89 Table XI Adjusted and Unadjusted Group, Race and Gender Means and Variability for Problem Solving Competency Using Problem Solving Pretest Scores as Covariate Unadjusted Adjusted N M SD M SE Control group 41 3214.07 2056.72 2831.31 285.93 Black 21 2288.86 1118.61 2545.29 354.23 Female 13 2264.31 1087.38 2441.11 426.42 Male 8 2328.75 1243.06 2649.47 560.72 Hispanic 20 4185.55 2376.71 3174.53 467.72 Female 9 3774.44 2813.74 2579.79 719.53 Male 11 4521.90 2030.37 4066.64 451.61 Treatment 33 5139.88 1934.76 5305.72 313.18 Black 19 5055.00 2019.20 5219.07 434.45 Female 7 4999.00 2655.63 5431.57 636.90 Male 12 5087.67 1679.83 5006.58 596.84 Hispanic 14 5255.07 1882.59 5409.69 450.47 Female 8 6371.63 1230.13 6174.24 620.48
90 Table XI continued Male 6 3766.33 1562.66 4262.86 643.11 Table XII Analysis of Covariance for Problem Solving Competency as a Function of Group, Race and gender Using Problem Solving Pretest Scores as Covariate Source df MS F p eta 2 Problem solving pretest 1 47604131.04 21.90 Race 1 1276836.92 .59 .447 .11 Group*Race 1 2335186.45 1.07 .305 .02 Group*Gender*Race 1 11919431.53 5.48 Error 51 2202501.27 The significant effects and interactions between group, gender and race are depicted in Figure XIII XVI below. Figure XIII shows that females benefited from DBS, as th eir mean problem solving scores improved, however female Hispanic students saw a greater improvement in their mean problem solving competency score.
91 Figure XI I Control Groups by Race Figure XIV tment and Control Groups by Race (Females) 0 Control group 1 Treatment group (1 Black 2 Hispanic) (1 Black 2 Hispanic) 0 Control group 1 Treatment group (M ales)
92 Figure XIV, on the other hand shows Black males benefiting more from DBS than their Hispanic counterparts in problem solving competency. Within each race, the effects of DBS on female and male students can also be compared. Figure XV shows that among Figure XV. Comparison of Problem Solving Competency Scores of Black Female and Male Students the Black students females had a slight edge over males although the mean problem solving competency scores of both groups improved after the treatment. Also, among the Hispanic students, Figure XVI below shows that females appear to have benefited more from DBS instruction than their male counterparts, reversing the edge males had prior to the problem solving pretest. The mean problem solving competency scores for Hispanic males in the treatment and control group are similar, implying that in general Hispanic males appear not to have benefited from the DBS instruction. (Black s tudents ) 0 Female 1 Male (0 Control group 1 Treatment group)
93 Figure XVI. Comparison of Problem Solving Competency Scores of Female and Male Hispanic Students Research question four: problem s olving c ompetency across SES This section presents results of the analysis intended to contribute to the resolution of research question four: are the problem solving competency differences (observed between treatment and control groups) consistent with SES? An analysis of covariance was conducted to ascertain whether there were differences in the effect of DBS on the problem solving competencies of students with different SES after controlling for problem solving pretest scores. The following assumptions were met: a) independence of o bservations, b) normal distribution of the dependent variable, c) homogeneity of variance, d) linear relationships between the covariate and dependent variable, and e) homogeneity of regression slopes. To enable easy comparison, SES values were grouped int o three categories with equal widths: low (8 27.99), mid (28 47.99) and high (48 66). The results indicated that the observed differences in problem solving competencies between students with different SES in the treatment and control (Hispa nic students) 0 Female 1 Male ( 0 Control group 1 Treatment group )
94 group were not significantly, F (2, 51) = .86, p = .428, eta 2 = .03. No interactions with SES (e.g. group and gender; group, gender and race) were statistically significant. The means and standard deviations for the treatment and control groups are presented in Table XII I for low, mid and high SES groups. Table XIII Adjusted and Unadjusted SES Group Means and Variability for Problem Solving Competency Using Problem Solving Pretest Scores as Covariate Unadjusted Adjusted N M SD M SE Control Low SES 19 4133.11 2477.92 3485.69 379.48 Mid SES 16 2392.63 1122.14 2193.09 471.35 High SES 6 2494.33 1403.01 2809.76 666.68 Treatment Low SES 18 5204.11 1848.27 5054.98 377.14 Mid SES 12 4723.92 2108.43 4824.11 422.17 High SES 3 6418.33 1696.65 6282.18 851.81
95 Table XIV Analysis of Covariance for Problem Solving Competency as a Function of SES Group Using Problem Solving Pretest Scores as Covariate Source df MS F p eta 2 Problem solving pretest 1 47604131.04 21.90 SES 2 4326850.82 .08 .147 .07 Group*SES Group 2 1875317.77 .86 .428 .03 Error 51 2174182.62 Differences in the problem solving competencies of the SES groups in the treatment and control groups were not significant. The slopes of graphs in Figure XVIII however show Figure XVII. Graph of Problem Solving Posttest Scores for Treatment and Control SES Groups 1 Low SES 2 Mid SES 3 High SES (0 Control group 1 Treatment group)
96 that high and medium SES groups appear to have benefited more than low SES groups from DBS, with higher SES students scoring the highest on average. Summary In the above sections analyses of covariance were cond ucted to compare the mean problem solving competency scores of a) control and treatment groups, b) female and male students, c) Black and Hispanic students (within and between the two groups with aggregation across gender), and d) low SES, mid SES and high SES students. These analyses were intended to answer research questions one to four respectively: 1) Does DBS have an y effect on the problem solving competencie s of students in a high school traditional c hemistry class ? 2) D oes the effect of DBS on proble m solving competency vary depending on gender ? 3) D oes the effect of DBS on problem solving competency vary depending on race ? 4) D oes the effect of DBS on problem solving competency vary depending on SES? The above ANCOVA results suggest that a) DBS imp roves the problem solving competency of students in a high school traditional chemistry class, thereby disproving null hypothesis one, which states that there is no effect of DBS on problem solving competency (H o = control = experimental ), b) the effects of DBS on problem solving competency significantly varies depending on gender, rejecting null hypothesis two, which states that there is no difference in the effect of DBS depending on gender (H o = female = male ), c) the effects of DBS on problem solvin g competency does not significantly vary depending on race, accepting null hypothesis three, which states that there is no difference in the effect of DBS depending on race (H o = Black = Hispanic ), d)
97 there is a statistically significant interaction betw een race and gender when the control and treatment groups were compared, suggesting that DBS instruction improves problem solving competency of Black females, Black males and Hispanic females but not Hispanic males, and e) the effects of DBS on problem sol ving competency does not significantly vary depending on SES, accepting null hypothesis four, which states that there is no difference in the effect of DBS depending on SES (H o = LowSES = midSES = High SES ). DBS and Chemistry Achievement This section presents analyses needed to answer research questions five to eight respectively: 5) Do es DBS have any effect on c hemistry achievement of students in a high school traditional c hemistry class ? 6) Does the effect of DBS on chemistry achievement vary dependi ng on gend er? 7) Does the effect of DBS on chemistry achievement vary depending on race? 8) Does the effect of DBS on chemistry achievement vary depending on SES? Research quest ion five: DBS and chemistry achievement. The intent of research question five was to ascertain whether students who were given DBS instruction did better on a test of chemistry concepts than students in a control group. The results of the analysis are shared below. These results show that there was no significant difference in the chemistry achievement of students in the control and treatment groups. An ANCOVA was performed, with CCI pretest as covariate to investigate if DBS instruction produced any significantly different CCI scores for the treatment group
98 compared with the contro l group. The following assumptions were met: a) independence of observations, b) normal distribution of the dependent variable, c) homogeneity of variance, d) linear relationships between the covariate and dependent variable, and e) homogeneity of regressi on slopes. All assumptions were met. The results indicate that there were no significant differences in knowledge gained in chemical change and chemical energy concepts between students in the treatment and control groups, F (1, 51) = 0.45, p = .51, eta 2 = .04. The means and standard deviations for the two groups are presented in Table XV. Thus DBS instruction appears not to have any effect on the chemistry achievement of high school students in a traditional chemistry. Table XV Adjusted and Unadjusted Group Means and Variability for Chemistry Concepts Inventory (CCI) Using CCI Pretest Scores as Covariate Unadjusted Adjusted N M SD M SE Control group 41 13.27 2.40 13.00 0.35 Treatment group 33 13.67 1.63 13.58 0.39
99 Table XVI Analysis of Covariance for CCI as a Function of Group Using CCI Pretest Scores as Covariate Source df MS F p eta 2 CCI pretest 1 7.63 2.33 .133 .44 Group 1 1.47 .45 .506 .01 Error 51 4.71 Research question six: DBS and chem istry achievement across gender. The goal of research question six was to investigate whether any effects of DBS depended on gender. The results of the analysis are shared below. These results show that there was no significant difference in the chemistry achievement between female and male students in the control and treatment groups. An ANCOVA was then performed, with CCI pretest as covariate to investigate if DBS produced any significantly different CCI scores for female than male students. Thi s analysis was intended to help resolve research question six. The following assumptions were tested and met: a) independence of observations, b) normal distribution of the dependent variable, c) homogeneity of variance, d) linear relationships between the covariate and dependent variable, and e) homogeneity of regression slopes. All assumptions were met. The results indicate that there were no significant differences in CCI scores when females and males from the treatment group were compared with those
100 in the control group, F (1, 51) = 0.91, p = .35, eta 2 = .02. The means and standard deviations for the two groups are presented in Table XVII. Table XVII Adjusted and Unadjusted Gender Means and Variability for Chemistry Concepts Inventory (CCI) Using CCI Pretest Scores as Covariate Unadjusted Adjusted N M SD M SE Control group Female 22 12.73 2.64 12.78 0.52 Male 19 13.89 1.97 13.26 0.46 Treatment group Female 15 13.93 1.94 13.52 0.54 Male 18 13.44 1.34 13.65 0.54
101 Table XVIII Analysis of Covariance for CCI as a Function of Gender Using CCI Pretest Scores as Covariate Source df MS F p eta 2 CCI pretest 1 7.63 2.33 .133 .044 Group 1 1.47 .45 .506 .009 Group*Gender 1 2.98 .91 .345 .017 Error 51 3.28 The differences between the CCI scores for female and male students in the treatment and control groups were not statistically significant. Figure XVIII, however shows that there was just a small gain in the CCI scores of both female and male students in the treatment group compared with those of the control group. However, females in the treatment group scored higher than their male counterparts.
102 Figure XVIII. Graph of CCI Posttest Scores for Female and Male Students in Treatment and Control Groups Research question seven: DBS and chemistry achievement across race. The intent of research question seven was to ascertain whether any effect of DBS instruction on chemistry achievement depended on race. The results of the analysis are shared below. These results show that there was no significant difference in the chemistry achievement of Black and Hispanic students in the treatment group when compared with those in the control group. An ANCOVA was then performed, with CCI pretest as covariate to investigate if DBS instruction produced any significantly different CCI scores f or Black than Hispanic students. The following assumptions were met: a) independence of observations, b) normal distribution of the dependent variable, c) homogeneity of variance, d) linear relationships between the covariate and dependent variable, and e) homogeneity of regression slopes. All assumptions were met. The results indicate that there was no ( 0 Control group 1 Treatment group) ( 0 Female 1 Male)
103 significant differences in CCI scores when Black and Hispanic students from the treatment group are compared with those in the control group, F (1, 51) = 3. 09, p = .085, eta 2 = .06. The means and standard deviations for the two groups are presented in Table XIX. Table XIX Adjusted and Unadjusted Race Means and Variability for Chemistry Concepts Inventory (CCI) Using CCI Pretest Scores as Covaria te Unadjusted Adjusted N M SD M SE Control group 41 13.27 2.40 13.00 .350 Black 21 12.05 1.99 12.19 .430 Female 13 11.46 1.85 11.80 .533 Male 8 13.00 1.93 12.59 .675 Hispanic 20 14.55 2.14 13.97 .574 Female 9 14.56 2.60 13.77 .886 Male 11 14.5 5 1.81 14.27 .566 Treatment 33 13.67 1.63 13.58 .390 Black 19 13.42 1.47 13.57 .540
104 Table XIX continued Female 7 13.86 1.35 13.83 .780 Male 12 13.17 1.53 13.32 .740 Hispanic 14 14.00 1.84 13.59 .557 Female 8 14.00 2.45 13.22 .761 Male 6 14.00 .63 14.14 .789 Table XX Analysis of Covariance for CCI as a Function of Race Using CCI Pretest Scores as Covariate Source df MS F p eta 2 CCI pretest 1 7.63 2.30 Race 1 5.22 1.59 .447 .11 Group*Race 1 10.14 3.09 .305 .02 Group*Gender*Race 1 .28 .09 .771 .002 Error 51 3.28
105 When females and males in the two races in the treatment group were compared to those in the control group no significant differences are found (i.e. the interaction, group*gender*race was not significant), F (1, 51) = .086, p = .771, eta 2 = .002. Resea rch question eight : DBS and chemistry achievement across SES The intent of research question eight was to ascertain whether any effect of DBS instruction on chemistry achievement depended on SES. The results of the analysis are shared below. These results show that there was no significant difference in the chemistry achievement of students from low, mid and high SES subgroups when the treatment group was compared with the control group. An ANCOVA was performed, with CCI pretest as covariate to investigate if DBS produced any significantly different CCI scores for students in different SES groups. This was intended to assist in answering research question eight. The following assumptions were met: a) independence of observations, b) normal distribution of t he dependent variable, c) homogeneity of variance, d) linear relationships between the covariate and dependent variable, and e) homogeneity of regression slopes. All assumptions were met. The results indicate that there are no significant differences in CC I scores when students from the treatment group are compared with those in the control group, by their SES groups F (2, 51) = .67, p = .52, eta 2 = .03. The means and standard deviations for the two groups are presented in Table XXI.
106 Table XXI Adjusted and Unadjusted SES group Means and Variability for Chemistry Concepts Inventory (CCI) Using CCI Pretest Scores as Covariate Unadjusted Adjusted N M SD M SE Control group 41 13.27 2.40 13.00 .350 Low SES 19 13.95 2.55 13.32 .470 Mid SES 16 12.69 2.21 12.78 .577 High SES 6 12.67 2.16 12.87 .819 Treatment group 33 13.67 1.63 13.58 .385 Low SES 18 13.78 1.40 14.03 .467 Mid SES 12 13.75 1.87 13.89 .544 High SES 3 12.67 2.31 12.57 1.047
107 Table XXII Analysis of Covariance for CCI as a Function of SES Group Using CCI Pretest Scores as Covariate Source df MS F p eta 2 CCI pretest 1 7.63 2.33 .133 .04 SES 1 2.49 .76 .472 .03 Group*SES 2 2.19 .67 .518 .03 Error 51 3.2 8 Summary In the above sections analyses of covariance were conducted to compare the mean chemistry concepts inventory scores of a) control and treatment groups, b) female and male students, c) Black and Hispanic students, and d) low SES, mid SES and high SES students. These analyses were intended to answer research questions five to eight respectively: 5) Does DBS have an y effect on the chemistry achievement of students in a high school traditional c hemistry class ? 6) D oes the effect of DBS on chemistry achievement vary depending on gender ? 7) D oes the effect of DBS on chemistry achievement vary depending on race ? 8) D oes the effect of DBS on chemistry achievement vary depending on SES? The above ANCOVA results suggest that a) DBS do es not improve the chemistry achievement of students in a high school traditional chemistry class, thereby accepting
108 null hypothesis five, which states that there is no effect of DBS on problem solving competency (H o = control = experimental ), b) the eff ects of DBS on chemistry achievement does not vary depending on gender, accepting null hypothesis six, which states that there is no difference in the effect of DBS on chemistry achievement depending on gender (H o = female = male ), c) the effects of DBS on chemistry achievement does not significantly vary depending on race, accepting null hypothesis seven, which states that there is no difference in the effect of DBS depending on race (H o = Black = Hispanic ), d) the effects of DBS on chemistry achieveme nt does not significantly vary depending on SES, accepting null hypothesis eight, which states that there is no difference in the effect of DBS depending on SES (H o = LowSES = midSES = High SES ). Correlation between Problem Solving Competency and Chemistry Concepts Score Research question nine : problem solving as predictor of chemistry scores To investigate whether the level of chemistry concepts acquired by a student may be predicted by his/her problem solving competency, a regression analysis was performed. This analysis involved all students, without a differentiation of control and treatmen t groups. The following assumptions were tested and met, a) independence of observations, b) linearity, and c) the dependent variable (chemistry concepts score) was approximately normally distributed. Problem solving competency (M = 4320.13, SD = 2331.54), significantly predicted the chemistry concepts score (M = 13.63, SD = 2.33), R 2 = .27. This implies that 27% of the variance in the chemistry concepts score is predicted by problem solving competency. Also an r valu e
109 R 2 ) of 0.53 indicates a high effect size. The beta weight, presented in Table XXIII, indicate that when the problem solving competency increases by one unit, chemistry concepts score increase by 0.001 units. The preceding analysis therefore disproves t he null hypothesis, which states that problem solving competency is not predictive of chemistry concepts score (H o = CCI = Problem solving ). Table XXIII Simple Linear Regression Analysis for Problem Solving Competency Predicting Chemistry Concepts Sco re (N = 82) Variable B SEB Problem Solving Competency 0.001 0.00 .53*** Constant 11.35 .47 Note R 2 = .27; F (1, 80) = 31.03, p *** p Summary This study investigated the effects of DBS on the problem solving competencies of high school students in a traditional chemistry class. It explored differences in the effects of DBS among groups by race, gender and SES. The study also ascertained if probl em solving competency was predictive of chemistry achievement. ANCOVA was used to analyze the data. The findings are as follow: a) DBS significantly improved the problem solving competency of students in the study, b) DBS significantly improves the problem solving competency of both males and females, with a slight urge among females, c) the differences in the effects of DBS in improving problem solving competency among Black and Hispanic students in this study was not statistically
110 significant, however, Bl ack students and Hispanic female students showed significant improvement in problem solving competency after the DBS instruction, d) DBS did not statistically significantly improve the problem solving competency of students of particularly SES group(s), an d e) Problem solving competency is a strong predictor of higher chemistry concepts score among students in both treatment and control groups.
111 CHAPTER V CONCLUSIONS Introduction S cience ( and math ) achievement in the United States has been on the decline over the past couple of decades, both nationally and internationally (Ornstein, 2010; U. S. Department of Education 2004, 2006; NEAP, 2005) The following achievement gaps have been identified as contributing to the decline: a) a chievement gaps between males and females (NSF, 2008; Ornstein, 2010) and b) achievement gaps between White and Asian students on the one hand and Black and Hispanic students on the other (Ornstein, 2010; Clewell & Ginorio, 1996; Creswell & Houston 1980) Instructio nal methods such /critical thinking, problem solving ability, science process skills and consequently science achievement (Mehalik, Doppelt and Schunn 2008; Fortus, Dershimer, Krajcik, Marx and Mam lok Naaman, 2004, 2005; OECD 2003; Chang, 2001a, 2001b). The purpose of this study was to investigate the effects of DBS on problem solving competency and science achievement, thereby contributing to the search for ways to close the achievement gap betwee n males and females, students from different races and the rich and poor. Specifically, the questions addressed were: 1) Does DBS have an y effect on the problem solving competencie s of students in a high school traditional c hemistry class ? 2) Does the effect of DBS on problem solving competency depend on gender? 3) Does the effect of DBS on problem solving competency depend on race ? 4) Does the effect of DBS on problem solving competency depend on SES?
112 5) Do es DBS have any effect on the c hemistry achievement of students in a high school traditional c hemistry class ? 6) Does the effect of DBS on chemistry achievement vary depending on gender ? 7) Does the effect of DBS on chemistry achievement vary depending on race ? 8) Does the effect of D BS on chemistry achievement vary depending on SES? 9) Is the problem solvi ng competency of students in a traditional c hemi stry class predictive of their c hemistry achievement? The results of the study suggest that DBS contributes to the problem solving c ompetency of students, whe n treatment and control groups we re compared. The observed effects of DBS on problem solving competency were also significantly different f or students of different gender: females benefiting significantly more than males. The effe ct of DBS on the problem solving competency of Black and Hispanic students was not significantly different. However, Hispanic males in the treatment group did not score significant higher than those in the control group, suggesting that DBS instruction may not be useful in improving problem solving skills of students in this group. One reason that may account for this similarity between Hispanic males in the control and treatment groups is the observation that during the deign project a couple of groups tha t had more Hispanic males consistently sought to solve problems that were not only overly simplistic but also did not meet all the constraints laid out for the project. Most female students on the other hand were more focused and more resolved to meet the constraints of the project. Even though some did not produce a finished elaborate design, their determination was an indication of their acceptance of the challenge and consequently may have passed through the DBS instruction, acquiring intended skills.
113 T he observed effects of DBS on problem solving competency scores were also found not to depend on SES. Also, DBS had no statistically significant effect on the chemistry achievement scores of students in the study. The results however suggest a strong corre lation between problem solving competency and chemistry achievement. These findings will be discussed below followed by study limitations and implications for research and practice. Findings and Interpretation of R esults Effects of DBS on Problem Solving Competency Research questions one to four were intended on assessing whether DBS had any effects on problem solving competency and if there was, how the effect differed depending on gender, race and SES. The results suggest a significant difference in pro blem solving competency between the treatment group and the control group. Due to the difficulty in measuring problem solving as a whole, the PISA problem solving protocol focuses on three areas of problem solving, namely, decision making, system analysis and design, and troubleshooting. These three components were also present in the DBS instruction particularly during the design project. First students needed to decide on exactly what problem they wanted to solve. This stage was not that challenging sinc e the decision was driven primarily by interest. Decision making was however present in the other stages as well. The next stage of system design involved identifying the various components of the system. Students, in applying the design process were requi red to produce drawings showing different angles of their system, including a plan (top view) and an elevation (side view). During this stage they had to decide on the types of materials to use to produce the best desired
114 results. Students also had to dete rmine the arrangement of system components that would produce the best results. Students had to make all these decisions cognizant of the chemistry concepts or reaction(s), which form the fundamental aspect of the system. This stage was quite challenging f or the students, as they were clearly not used to this level of critical thinking. Students needed a lot of guidance and direction or else they were inclined to give up. In fact they preferred being told what to do at this stage. During the third stage whe n they had completed their design and the system was either not working at all or not working as anticipated, students had to troubleshoot by investigating possible causes. Even if the system initially worked students had to determine conditions for optimu m results. During this and the previous stages students had to think critically in order to move on with their projects. The level of difficulty of each of these stages may have produced different learning environments for the students that may reflect in their performance on the problem solving assessment. The results of the problem solving competency pretest put the treatment and control groups at practically the same level in all three measured categories of problem solving. The use of ANCOVA as analysi s further leveled the playing field by way of controlling for these pretest scores. The results showed that the treatment group performed better on the posttest than the control group on all three problem solving categories (Tables IV, VI and VIII). The a bove results support findings by Kolodner, Camp, Crismond, Fasse, Gray, Holbrool, Puntambekar, and Ryan, ( 2003 ) and Silk, Schunn and Strand, ( 2007) Their findings suggest that e ngaging students in design based learning or problem based learning within a science classroom has the potential of helping students develop problem
115 solving skills and scientific inquiry skills The difference between the above referenced statements and th e findings of the current study is that whereas the current study provides empirical evidence relating to a chemistry classroom the other is not based on any evidence. Effects of DBS on Problem Solving Competency across Gender Differences in science achievement between males and females have been adequately researched and established. Males appear to perform better than females in science particularly at the high school level ( Ingels & Dalton, 2008; Bacharach, Baumeister & Furr 2003; Jones, Mullis, Raizen, Weiss, &Weston, 1992) The search for ways to close this achievement gap is ongoing. In this regard the results of this study may contribute an understanding of instructional methods that support female excellence in science. The data from the current study suggests that there is no statistically significant difference the problem solving scores when males and females in the treatment are compared with those in the control groups (Table X). However, using the adjusted means o f problem solving score in Table X for treatment and control groups, females (mean gain = 2564.19) appear to have benefited more from DBS instruction than males (mean gain = 1314.26). This relationship is displayed in Figure XI. Although this interaction b etween group and gender is not statistically significant, it is worth replicating this study with a larger group, since a larger sample size may improve the reliability and power of analysis. Thus when females in the treatment group are compared with those in the control group, it becomes evident that females may stand to benefit more than males.
116 The items on the problem solving protocol were generally not gender biased and as a result they add to the credibility of any conclusions drawn from the above res ults. It is however, worth noting that if females would break through the glass ceiling in STEM related careers they must be continually supported in working past societal stereotypes about female and male roles: that males build while females nurture. E ffects of DBS on Problem Solving Competency across Race Conversations involving achievement gaps cannot be complete without the trailing White and Asian Students. This study thus als o sought to investigate the effects DBS may have on problem solving competency depending on race. Research question three sought to ascertain the effects of DBS instruction on the problem solving competency of different races so as to determine if DBS has a potential to help close the achievement gap associated with race. Due to circumstances beyond researcher control, the number of Black and Hispanic students in the study sample was larger than other races. Also, the numbers of Asian, Native American and W hite students in the study sample were too small to be included in the analysis. If they had been included deductions on the effects of DBS on problem solving competency for these groups would not be meaningful. The results (Table XII) show that there is no significant difference in the problem solving scores of Black and Hispanic students. Due to the fact that these results are close to being significant, the interactions are worth discussing to bring to the fore the potential differences between subgroup s in terms of benefits of DBS. Figure XII depicts the similarity in the gains in problem solving score for Black and Hispanic females. Figure XIII shows the problem solving competency scores of Black males in the treatment group
117 increasing while those of t heir Hispanic counterparts remaining almost unchanged. The Hispanic males therefore appeared to have benefited from DBS: whereas the gains in problem solving scores for Black females and males were close (Figure XII), those of Hispanic females and males sh owed that the females scored higher (Figure XV). Historically, in the school where the study was conducted there has been a disproportionately high number of Black and Hispanic students in traditional science courses. The racial composition of the study s ample vis vis that is the school as a whole attests to this observation: 2.4% Asian, 48.8% Black, 41.5% Hispanic, 3.6% Native American and 3.6% White. The current distribution of students of different races in this school includes: 4.3% Asian, 41.7% Blac k, 23.3% Hispanic, 0.9% Native American and 29.3% White. Most of the Black and Hispanic students in the classes studied give a couple of reasons for avoiding higher level courses. These include their a) lack of motivation and b) perception that science is difficult. Such students hitherto wish to get by with a grade of D, even in traditional courses. If DBS can improve the problem solving competency of these students it can go a long way in improving their self confidence. Expanding this study to improve th e power and reliability of the analysis is therefore important. Expansion may include involving students from more school sites. Effects of DBS on Problem Solving Competency across SES The results of the PISA 2003 problem solving competency assessment sug gest a strong correlation between problem solving competency and SES (OECD, 2003). It is common knowledge that a high proportion of Black, Hispanic and Native American students fall within low SES in the United States. The implication of the PISA 2003 prob lem solving assessment results is that students from these races are more likely to
118 have low problem solving competencies. The goal of research question four was thus to establish the correlation between SES and problem solving and to ascertain if DBS has a potential of improving the problem solving competency of low SES students. In their study of the effect s of DBS on science achievement among middle s chool students by gender, socio economic status (SES) and race ethnicity, Mehalik, Doppelt and Schuun (200 8) report ed that low achieving African American students benefited the most from DBS The results of the current study show that there is no significant difference between the problem solving skills of low SES, mid SES and high SES when students in the c ontrol and treatment groups are compared. It may however be noted that while the interaction between group and SES group was not significant (Table XIV), all three SES groups in the treatment groups had higher problem solving scores than their peers in the control group (Figure XVI). This suggests that all SES groups appear to have benefited from DBS in their problem solving competency. The insignificant effect of DBS on problem solving skills across SES group remained so even after reducing the SES groups to two (lowSES and midSES), negating any fear that that SES groupings earlier used may be unrepresentative of the larger population.. Effect of DBS on Chemistry Achievement across Gender, Race and SES The results of preliminary studies by Fortus, Dershime r, Krajcik, Marx and Mamlok Naaman, (2005) and Puntambekar & Kolodner, 2005 imply that DBS and other inquiry based pedagogies have the potential of helping students develop science knowledge The results of the current study, however suggest that DBS does not have a statistically significant effect of chemistry achievement of the students in this study,
119 neither across gender, race nor SES. This implies that teachers and administrators are likely not to shun DBS or they may have a neutral attitude to using i t in their classrooms/schools. However, if DBS improves problem solving competency, as suggested from the answer to research question one and problem solving competency correlated to chemistry achievement, then there will be a long term benefit of DBS not only in improving problem solving competency but also indirectly improving chemistry achievement. Problem Solving Competency as Predictor of Chemistry Achievement Data from the PISA 2003 revealed a high correlation of 0.8, between problem solving competenc y and science achievement. The current study suggests that DBS has a significant effect on problem solving competency. If problem solving is a predictor of chemistry achievement then most of the students whose problem solving competency scores improved or students who scored high on the problem solving protocol would do well on the CCI. However, the results of the current study indicated no difference in the chemistry achievement between the treatment and control group, although there was a significant diff erence in problem solving score between the two groups. When all students in the study (treatment and control groups together) were pooled for the analysis there was a strong correlation between problem solving scores and CCI scores. In other words problem solving scores appeared to be a predictor of chemistry achievement. Hence the observation that even though problem solving competency on the treatment group improved but their chemistry achievement was not different from students in control group implies that the benefits of DBS towards chemistry achievement were not immediate.
120 During the design challenge tasks students were expected to use some of the concepts and knowledge tested by the Chemistry Concepts Inventory (CCI). Thus the design challenge tasks had direct bearing (knowledge wise and concepts wise) with what was assesse d by the CCI. The PISA problem solving items however had no direct bearing on the design challenge. Despite these facts students in treatment group improved on problem solving items over the control group. This was not the case for Chemistry Concepts Inven tory (CCI). A number of reasons could be advanced to account for the similar CCI scores for the treatment and control groups: a) Students in the treatment group may have been so focused on solving their specific problem that they lost attention of chemistr y concepts that were tested, b) students in control group are more academically motivated and performed better, c) the DBS instruction did not foster the acquisition of broad base chemistry knowledge: perhaps problem solving competency alone is not suffici ent to enable students to score higher on the CCI. Problem Solving Theories and Current Study Three problem solving theories were used as lenses to understand the results of the current study, namely, constructivist, expert novice and cognitive theories. Through the identification of need based problems by students for resolution, to the construction of prototypes, various student actions may be interpreted by these theories. This section discusses if and how these theories were at play during the design challenge projects as students attempted solving specific problems. In attempting to solve their chosen problem, students tapped into their prior knowledge (misconceptions or otherwise). For instance students who were enrolled in the r Reserve Officer Training Corps) program were more inclined to
121 identify MRE (meals ready to eat) as a source of chemical energy. MREs are food packages provided to military personnel. The package comes with a sachet of powdered magnesium, which upon mixin g with water generated enough heat to warm up food. However, these students had no understanding of the chemical reaction responsible for the heat generated. Thus by the end of their projects such students exhibit ed better conceptua l understanding of their domain. Also, at the beginning of the project students demonstrated little to no declarative conceptual understanding but by the end of the project could explain the origins of energy change. Throughout the project however, students had a hard time with d eciding the procedure they needed to apply in solving the problem. More opportunities for students to attempt similar challenges may provide more insight to students thereby showing their growth in expert problem solving procedural skills. The five compone nts of the DBS framework were intended to assist in the solve. The framework also challenged students to elicit both declarative and automated skills in order to resolve prob lems. In other words beyond the problem being resolved in the current study, students would recognize the need to look inward to elicit knowledge and skills required to solve any problem on hand. Students in the current study were successfully guided to th is realization since the teacher/researcher refrained from giving direct answers to student questions. At times some students were frustrated that they did not get direct answers. In the end however, they appreciated that they had the answer to most of the ir questions and needed to process deeper for themselves.
122 Constructivist, novice expert and cognitive theories, as reviewed above explain the observed significant effect of DBS instruction in improving the problem solving competencies of students, when the control and treatment groups were compared. This relationship may be due to the fact the DBS framework and the five step DBS learning cycle are consistent with these theories. By tapping into their prior knowledge students constructed new knowledge for th emselves. Also, as they were challenged by the design problem they developed the declarative conceptual understanding and procedural knowledge needed for problem resolution. Finally, the desire by students to eliminate their cognitive dissonance was enhanc ed by their inspiration to resolve a problem in order to meet their own need(s). Summary The findings of the study may apply to high school students in a traditional chemistry course, in an urban district similar to that in this study. The findings are as follow: a) DBS significantly affects the problem solving competency of students, b) DBS improves the problem competency of both males and females, with a statistically insignificant urge among females, c) DBS does not produce a statistically significant di fference in problem solving competency between Black and Hispanic students both whom appear to benefit from DBS except Hispanic males, d) DBS had no significant effect on problem solving competency depending on socioeconomic status, and e) Problem solving competency is a strong predictor of higher chemistry concepts score among both treatment and control groups.
123 Generalizations Due to the limitations of sampling and various defining sample characteristics, the generalizability of the findings of this study is restricted to: a) students of traditional chemistry who are taking chemistry for the first time, b) students who are generally not highly motivated to study chemistry, and c) schools in an urban school district with similar student characteristics as the one used in this study. This is because t esting one school makes generalization difficult since the individual school tested may generate better or worse results for students using that particular educational instruction The students in one school may be from a completely different soci oeconomic background or culture and therefore cannot be a representative sample of the population. Delimitations of the Study This study limited itself to high school students in a traditional chemistry class. These students were sophomores who were taking chemistry for the first time and participated in the school district s pre course assessment. The pre course assessment results and CCI pretest scores were used to determine the treatment and control groups. For purposes of homogeneity of student chemistry learning experiences, students who had taken a semester one class in chemistry but who could not complete semester two during the year preced ing this study were not included. Limitations of the Study There are five limitations to the study. First, the present study is not framed in a true experimental research design with random assignment of subjects to treatment and control groups. Therefore the generalizability of the results to the total high school student population of the nation is limited. Like any other age group, high school students
124 are a very heterogeneous population. The fact remains that certain segments of the high school popula tion, by virtue of their micro culture, abilities, etc., may not be included. However, since the research design utilizes student level data, the results can provide valuable preliminary information about the effects of DBS on student problem solving compe tency among different student groups. Secondly, the study does not necessarily establish cause and effect study relationships. This is because in such social science research there are a number of other variables that are either not present in the study c ontext or are not apparent in the study. Third, anytime an instrument is used the results are subject to the known reliability and validity of that instrument. Although some information about the instrument in regard to reliability and validity is known, the instrument may have limitations in measuring what they purport to measure. Only subsequent research with other audiences and with other instruments will help further our understanding of the concepts being measured in the study. The fourth limitation minimize this peer observers were requested and visited two class periods as well as reviewed student work gr aded by teacher/researcher. Finally, the proportion of Asian and White students in the study sample was much lower than what pertains in the school as these students tend to participate in higher level chemistry classes. Hence the few Asian and White stu dents in the study were not included in the analyses that involved race.
125 Implications for Research Due to the exploratory nature of this study, the primary suggestion for future research is to build on the present study by replicating it in the future, with some modifications. These modifications include: a) the use of random sampling techniques that select a number of schools within one or more cities to increase the generalizability, b) to include various academic levels such Advance Placement and Int ernational Baccalaureate, c) include sufficient students from all races, d) include other science subjects such as physics and biology, and e) use DBS curriculum that involve more than one unit over a longer period than used in this study. Secondly, studi es are clearly needed to develop a more broad spectrum problem solving competency protocol. The PISA 2003 problem solving protocol measures only three aspects of problem solving: decision making, system analysis and design, and Trouble shooting. A more com prehensive protocol must perceive p roblem solving primarily as an internal and sequential process that includes cognitive, affective, and psychomotor behaviors. It must be stated that problem solving is a complex concept for measurement and includes ill defined and well defined problems. The later appear to be easier to measure than the former. Finally, if the strong correlation between problem solving and chemi stry achievement is further established, research is needed to identify effective school wide best practices and culture around the use of real world problem solving to spur science achievement. After all, problem solving does not involve a set of skill li mited to successful academic life but rather the very existence of the human race.
126 Implications for Practice http://www.nextgenscience.org/next generation science standards ) is about to be released, after going through the final phase of public review. One of the cornerstones of the new standards is the integration of science with engineering design. In retrospec t, a large scale empirical investigation of the kind in this study is long overdue to identify mainstream trends as well as the deviations from the norm: how do different high school student groups respond to engineering design (or design based science)? T he current study thus provides some insights as to how teachers should view the influences of DBS on their students. The strong correlation between problem solving and chemistry (and hopefully science) achievement suggests that a greater emphasis needs to be placed on problem incorporate problem solving instruction into science classrooms since for the study subjects and conditions, it has been shown to have a significant eff ect on problem solving competency. The challenge is the lack of a comprehensive design based science curriculum. Also, science teachers will need intensive training in DBS to make them effective in integrating science with design. The implications of the c ontinuously widening achievement gaps between different student groups make the findings of this study compelling. According to the McKinsey consulting firm (2009) the gap in science and math achievement between 1983 and 1998 cost the U.S. a Gross Domest ic Product (GDP) of approximately $2 trillion higher. T he achievement gap between Black and Hispanic students on the one hand and w hite and Asian students by 1998 cost the U.S. about $400 to $500 billion. The
127 results of this study suggest that the use of D BS instruction could improve the problem solving competencies of Black students (in particular) as well as Hispanic students. DBS instruction could also improve female achievement since females respond well to it. Collaboration is one of the twenty first century skills that will enable students to be successful at the workplace. DBS instruction also serves to improve collaboration among students as it involves students in group projects. Collaboration provides students the opportunity to learn from each ot her and develop communication skills. Finally, staying current and well informed about research exploring the effects of design based science instruction on problem solving competency and academic achievement is an important exercise in science teacher pro fessional development. Furthermore, science teachers should seek opportunities to share the implications of such research with school administrators, faculty members, and parents of children enrolled in their schools. Summary This study investigated the ef fects of DBS on the problem solving competencies of high school students in a traditional chemistry class. It explored differences in the effects of DBS among groups by race, gender and SES. The study also ascertained if problem solving competency was pred ictive of chemistry achievement. The findings are as follow: a) DBS significantly affects the problem solving competency of students, b) DBS improves the problem competency of both males and females, with a statistically insignificant urge among females, c ) DBS does not produce a statistically significant difference in problem solving competency between Black and Hispanic students both whom appear to benefit from DBS except Hispanic males, d) DBS had no significant
128 effect on problem solving competency depen ding on socioeconomic status, and e) Problem solving competency is a strong predictor of higher chemistry concepts score among both treatment and control groups. Due the limitations of the study the following are recommendations for future research: a) the use of random sampling techniques that select a number of schools within one or more cities to increase the generalizability, b) to include various academic levels such Advance Placement and International Baccalaureate, c) include sufficient students from all races, d) include other science subjects such as physics and biology, and e) use DBS curriculum that involve more than one unit over a longer period than used in this study.
129 References AAAS (American Association for the Advancement of Science) (1967) Science A Process Approach (Washington, DC: AAAS). AAAS (1990). Science for all Americans. New York: Oxford University Press. AAAS (American Association for the Advancement of Science) ( 1993) Benchmarks for Science Literacy (New York: Oxford University Press). Journal of Chemical Education On Line: Library of Conceptual Questions. (2001). American Chemical Society Division of Chemical Education Retrieved January 29, 2012 from http://jchemed.chem.wisc.edu/JCEWWW/Features/CQandChP/CQs/CQIntro.html A merican C ollege T est Program (2006). Developing the stem education pipeline. Iowa Cit y, IA: ACT. American College Testing Program. (2007). ACT High School Profile Report: HS Graduating Class 2007 HS Graduating Class National Report, Iowa City: American College Testing Program. The effects of proble m based active learning in learning. Eurasia Journal of Mathematics, Science & Technology Education, 2007, 3(1), 71 81. American Association of State Colleges and Universities. (200 5). Strengthening the science and mathematics pipeline for a better America. Policy Matters, 2(11). Anderson, C., & Smith, E. (1981). Patterns in the use of elementary school science program materials: An observational study. National Association for Research in Science Teaching Abstracts. Columbus, OH: ERIC. Apedoe, X.S., Reynolds, B., Ellefson, M.R. and Schunn, C. D. (2008). Bringing engineering design into high school science classrooms: the heating/cooling unit. Journal of Science Education and Te chnology, 17 454 465. Atkin, J. M. & Black, P. (2003). Inside science education: A history of curricular and policy change. New York: Teachers College Press. Ault, C. R. (1994) Research on problem solving: earth science. In D. L. Gabel (ed.) Handbook of Research on Science Teaching and Learning (New York: Macmillan), 269 283. Astin, H. S., & Sax, L. J. (1996). Developing scientific talent in undergraduate women. In C. Davis, A. B. Ginorio, C. S. Hollenshead, B. B. Lazarus, P. M. Rayman, &
130 Associates. (Eds.), The equity equation: Fostering advancement of women in the sciences, mathematics and engineering (pp. 96 121). San Francisco, CA: Jossey Bass. Bacharach, V. R., Baumeister, A. A. & Furr, R. M. (2003). Racial and Gender Science Achievement Gaps in Secondary Education, The Journal of Genetic Psychology: Research and Theory on Human Development, 164(1), 115 126. Baker, E. D., Hope, L., & Karandjeff, K. (2009). Contextualized teaching and learning: A faculty primer. Sacramento, CA: The Research and Pl anning Group for California Community Colleges, Center for Student Success. Retrieved February 10, 2012 from http://www.careerladdersproject.org/docs/CTL.pdf Barratt, W. (2006). The Barratt simplified measure of social status (BSMSS) measuring SES. Unpublished paper, Indiana State University, Terre Haute, Indiana. Barrows, H. S., & Kelson, A. (1993). Problem based learning in secondary education and the Problem based Learning Institute (Monograph). Springfield: Southern Illinois University School of Medicine. Baruch, R., & Nagy, J. N. (1977). Females and males in the potential scientist pool: A study of the early college years. Washington, DC: National Science Foundation. Creswell, J. L ., & Houston, G. M. (1980). Sex related differences in mathematics achievement in Black, Chicano, and Anglo adolescents. Houston, TX: University of Houston. (ERIC Document Reproduction Service No. ED198079). Bartel, A.P., Lichtenberg, F.R., & Vaughan, R.J (1992). Technological change, trade, and the need for educated employees: Implications for economic policy (NCEE Brief). Washington, DC: National Center on Education and Employment. Berryman, S. E. (1983). Who will do science?: Minority and female attai nment of science and mathematics degrees. New York: Rockefeller Foundation. Baynes, K. (1994). Designerly play. Loughborough: Loughborough University of Technology. Baxter, G. P. & Glaser, R. (1997). An approach to analyz ing the cognitive complexity of s cience performance assessments (Technical Report 452), National Center for Research on Evaluation, Standards and Student Testing (CRESST), Los Angeles, CA. Benbow, C. P. & Arjmand, O. (1990). Predictors of high academic achievement in mathematics and scie nce by mathematically talented students: A longitudinal study. Journal of Educational Psychology 82(3), 430 441.
131 Bergin, D, & Cooks, H. (2002). High school students of color talk about accusations of Urban review 34(2), 113 134. Bernst ein, B. (1975). Class, codes and control: Towards a theory of educational transmissions. Boston: Routledge & Kegan Paul. Blumenfeld, P.C., Soloway, E., Marx, R.W., Krajcik, J.S., Guzdial, M., & Palincsar, A.S. (1991). Motivating project based learning. Ed ucational Psychologist, 26 (3 & 4), 369 398. Bodner, G. (1986). Constructivism: A theory of knowledge. Journal of Chemical Education, 63(10), 873 877. Boud, D., & Feletti, G. (1997). The Challenge of Problem based Learning (2nd ed.). London: Kogan Page. Bouillion, L.M., & Gomez, L.M. (2001). Connecting school and community with science learning: Real world problems and school community partnerships as contextual scaffolds. Journal of Research in Science Teaching, 38, 878 898. Bradshaw G F (1992) The airplane and the logic of invention. In: Gidere RN (E d) Cognitive models of science. University of Minnesota Press, Minneapolis Bransford, J. D., & Stein, B. S. (1984). The IDEAL Problem Solver: A Guide to Improving Thinking. New York, NY: W. H. Freeman & Co. Bransford, J. D., Brown, A. OL. & Cockling, R. R. (Eds.) (1999). How People Learn: Brain, Mind, Experience, and School. National Academy Press: Washington, DC. P. (2010). Toward equity through participation in modeling instruction in introductory university physics. PRST: PER 010106 1. Brophy SP, Bransford JD (2001) Design methods for instructional modules in bioengineering. In: Proceedings of the 2001 American Society of Engineering Education. http://vanth.org/ Publications.html Retrieved March 23, 2012. Brotman, J. S. & Moore, F. M. (2008). Girls and Science: A Review of Four Themes in the Science Education Literature. Journal of Research in Science Teaching 45(9), 971 1002. Brown, B. (2004). Discursive identity: Assimilation into the culture of science and its implications for minority students. Journal of research in science teaching, 41(8), 810 834.
132 CTGV (1992). The Jasper series as an example of anchor ed instruction: Theory, program description, and assessment data. Educational Psychologist, 27, 291 315. Carmichael, K, (2007). Plugging the leaky pipeline: How academic deans support the persistence of underrepresented minority students in science and ma thematics Ann Arbor, MI: Pro Quest. Catsambis, S. (1995). Gender, race, ethnicity, and science education in the middle grades. Journal of Research in Science Teaching. 32, 243 257. Champagne, A. B., & Klopfer, L. F. (1977). A sixty year perspective on three issues in science education: I whose ideas are dominant? II representation of women III reflective thinking and problem solving. Science Education, 61, 43 1 452. Champagne, A. B. and Klopfer, L. F. (1981). Structuring process skills and the solution of verbal problems involving science concepts. Science Education 65, 493 511. Chang, C. Y. (2001a) Comparing the impacts of a problem based computer assisted instruction and the direct interactive teaching method on student science achievement. Journal of Science Education and Technology 10, 147 153. Chang, C. Y. (2001b) A problem solving based computer assisted tutorial for the earth sciences. Journal of Computer Assisted Learning 17, 263 274. Chang, C. Y., & Barufaldi J. (1999) The use of a problem solving based instructional International Journal of Science Education 21 373 388. Chang, C. Y. & Mao, S. L. (1999) Comparison of Taiwan sci with inquiry group versus traditional instruction. The Journal of Educational Research 92, 340 346. solving ability in earth science. Int. J. Sci. Educ., 24(5), 441 451. Chi, M.H., Glaser, R., & Farr, M.J. (1988). The Nature of Expertise. Hillsdale, NJ: Lawrence Erlbaum. Cirino, P. T., Chin, C. E., Sevcik, R. A., Wolf, M., Lovett M. & Morris R. D. (2002). Measuring socioeconomic status: Reliability and preliminary validity for different a pproaches Assessment 9(2), 145 155 from the perspective of diversity. In C. Davis, A. B. Ginorio, C. S. Hollenshead, B. B. Lazarus, P. M. Rayman, & Associates. (Eds.), The equity equation:
133 Fostering advancement of women in the sciences, mathematics and engineering (pp. 163 231). San Francisco, CA: Jossey Bass. College Board. (2007). National Summary Report, 2007 Retrieved April 7, 2012 from http://www.collegeboard.com/student/testing/ap/exgrd_sum/2007.html Creswell, J. L., & Houston, G. M., (1980). Sex related differences in mathematics achievement in Black, Chicano, and Anglo adolescents. Houston, TX: University of Houston. (ERIC Document Reproduction Service No. ED198079). Davis, C., Ginorio, A. B., Hollenshead, C. S., Lazarus, B. B., Rayman, P. M., & Associates. (Eds.). (1996). The equity equation: Fostering advancement of women inthe sciences, mathematics and engineering. San Francisco, CA: Jossey Bass. Davis, M., Hawley, P., McMullan, B., & Spilka, G. (1997). Design as a catalyst for learning Alexandria, VA: Association for Supervis ion and Curriculum Development. Department of Education, National Center for Education Statis tics. (2004). Highlights from the Third International Mathematics and Science Study (TIMSS) 2003, NCES2005 005, Washington, DC, pp. 1 25. Department of Education, Office of Education Research and Improvement. (2006). The 2005, NCES 2006 466, Washington, DC. Dewey, J. (1916). Democracy and education: An introduction to the philosophy of education. New York: Macmillan. Dewey, J. (1938a). Experience and education. New York, NY: Collier. Dewey, J. (1938b). Logic: The Theor y of Enquiry New York, NY: Henry Holt and Company, Inc.). Dewey, J.(1938c). Experience and Education. New York, NY: Collier. Dewey, J. (1943). Valuation Judgments and Immediate Quality. Journal of Philosophy, 40, 309 317. Donovan, M. S., & Cross, C. T (2002). Minority students in special and gifted education National Research Council. Committee on Minority Representation in Special Education. Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press. Doppelt, Y ., Mehalik, M. M., Schunn, C. D., Silk, E. and Krysinski D. (2008). Engagement and Achievements: A Case Study of Design Based Learning in a Science Context. Journal of Technology Education. 19(2)
134 Drew, D. E. (1996). Aptitude revisited: Rethinking math a nd science education for Baltimore, MD: The Johns Hopkins University Press. Delaware Valley Industrial Resource Center and National Council for Advanced Manufacturing. 2006. The STEM Talent Development Forum: A Dialogue for Action. DVIRC: Philadelphia. Dimitrov, D. M., & Rumrill Jr, P. D. (2003). Pretest postt est designs and measurement of change. Work (Reading, Mass.) 20 (2), 159. Eubanks I D & Eubanks L P (1993) ACS test item bank for high school chemistry. American Chemical Society Division of Chemical Education Examinations Institute, Milwaukee Fadigan, K.A.,& Hammrich, P.L. (2004).A longitudinal study of the educational and career trajectories of female participants of an urban informal science education program. Journal of Research in Science Teaching, 41, 835 860. Fang, A. (2004). Scientific literacy: A systemic functional linguist ic perspective. Science education, 89(2), 335 347. Festinger, L. (1962). Cognitive dissonance. Scientific American, 20,7 93 102. Fioriello, P. (2010). Understanding the Basics of STEM Education. Retrieved January 30, 2011 from: http://drpfconsults.com/understanding the basics of stem education/ Fleischman, H.L., Hopstock, P.J., Pelczar, M.P., and Shelley, B.E. (2010). Highlights From PISA 2009: Performance of U.S. 15 Year Old Students in Reading, Mathematics, and Science Literacy in an International Context (NCES 2011 004). U.S. Department of Education, National Center for Education Statistics. Washington, DC: U.S. Government Printing Office. Fortus, D., Dershimer, R.C., Krajcik, J., Marx, R. W., & Mamlok Naaman, R. (2004). Design based science and student learning. Journal of Research in Science Teaching 41(10), 1081 1110. Fortus, D., Krajcik, J., Dershimer, R.C., Marx, R. W., & Mamlok Naaman, R. (2005). Design based s cience and real world problem solving. International J ournal of Sci ence Educ ation, 27(7) 855 879. Fusco, D. (2001). Creating relevant science through urban planning and gardening. Journal of Research in Science Teaching, 38, 860 877. Gagn, E.D., Yekovich, C.W., & Yekovich, F.R. (1993). The Cognitive psychology of school learning. New York: Harper Collins.
135 Gndara, P. (2005). Fragile futures: Risk and vulnerability among Latino high achievers Princeton, NJ: Policy Evaluation and Research Center, Educational Testing Service. Retrieved from: http://www.ets.org/Media/Research/pdf/PICFRAGFUT.pdf Geban, O., Askar, P. and Ozkan, I. (1992) Effects of computer simulations and problem s olving approaches on high school students. Journal of Educational Research 86 5 10. Gilbert, J., & Calvert, S. (2003). Challenging accepted wisdom: Looking at the gender and science education question through a different lens. International Journal of Science Education, 25, 861 878. Gonzales, P., Guzmn, J. C., Partelow, L., Pahlke, E., Jocelyn, L., Kastberg, D., et al. (2004). Highlights from the Trends in International Mathematics and Science Study (TIMSS) 2003 (NCES 2005 005). U.S. Department of Education, National Center for Education Statistics. Washington, DC: U.S. Government Printing Office. Gonzales, P., Williams, T., Jocelyn, L., Roey, S. Kastberg, D., & Brenwald, S. (2008). Highlights from the Trends in International Mathematics and Scienc e Study (TIMSS) 2007 U.S. Department of Education, National Center for Education Statistics. Washington, DC: U.S. Government Printing Office. Glatthorn, A. A., & Baron, J. (1985). The good thinker. In A. L. Costa (Ed.) Developing Minds: A Resource Book f or Teaching Thinking (pp. 49 53). Alexandria, VA : Association for Supervision and Curriculum Development. Grant, M. M. (2002). Getting a grip on project based learning: theory, cases and recommendations. Meridian: A Middle School Computer Technologies Journal 5 (1) Gross, S. (1988). Participation and performance of women and minorities in mathematics: Findings by gender and racial/ethnic group Rockville, MD: Montgomery County Public Schools, Carver Educational Services Center. Gross, S. (1989, Mar ch). Early mathematics performance and achievement: The beginning of a downward spiral for blacks and Hispanics. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA. Hammersley, M. (1993). On the teach er as researcher. Educational Action Research 1(3), 425 445. Hanson, S. L. (1996). Lost talent: Women in the sciences. Philadelphia, PA: Temple University Press.
136 Hanushek, E. A. & Rivkin, S. G. (2003). How to improve the supply of high quality teachers. Brookings Papers on Education Policy. Retrieved on February 2, 2010 at http://edpro.stanford.edu/Hanushek/admin/pages/files/uploads/Te acher%20qualit y.Brookings.pdf Hazari, Z. S., Tai, R. H. & Sadler, P. M. (2007). Gender differences in introductory university physics performance: The influence of high school physics preparation and affect. Science Education 91(6) 847 876. Helpman, E (2004). The Mystery of Economic Growth Cambridge MA: Harvard University Press. Holland, J. (1981). Social class and changes in orientation to meaning. Sociology 15, 1 18. Hmelo Silver, C. E. (2004). Problem based learning: What and how do students learn? Educational Psychology Review 16(3), 235 266. Hanushek, E. A., & Rivkin, S. G. (2009). Harming the best: How schools affect the black white achievement gap. Journal of Policy Analysis and Management 28 366 393. Huck, S. W. & McLean, R. A. (1975). Using a repeated measures ANOVA to analyze data from a pretest posttest design : A potentially confusing task, Psychological Bulletin 82 511 518. Ingels, S. J., & Dalton, B. W. (20 08). Trends a mong High School Seniors, 1972 2004 (NCES 2008 320). Washington, DC: National Center for Education Statistics, Institute for Education Sciences, U.S. Department of Education. ITEA. (2002). Standards for technological literacy. Reston, VA: Int ernational Technology Education Association. Jennings, E. (1998). Models for pretest p osttest data: Repeated measures ANOVA revisited, Journal of Educational Statistics 13 273 280. Jones, L. R.,Mullis, I. V. N., Raizen, S. A., Weiss, I. R., &Weston, E. A. (1992). The graders. Washington, DC: Office of Education Research and Improvement, U.S. Department of Education. Jones, B. F., Rasmussen, C. M., & Moffitt, M. C. (1997). Real l ife problem solving : A collaborative approach to interdisciplinary learning. Washington, DC: American Psychological Association.
137 Johnson, P. E., Ahlgren, A., Blout, J. P., & Petit, N. J. (1981). Scientific reasoning: Garden paths and blind alleys. Research in Science Education: New Questions, New Directions. 87 114). Jovanovic, J., & Steinbach King, S. (1998). Boys and girls in the performance based American Educational Research Journal, 35, 477 496. Kaufman, P. (1991). The relationship between postsecondary and high school course taking patterns: The preparation of 1980 high school sophomores who entered postsecondary institutions by 1984. Washington, DC: National Center for Education Statistics, U.S. Department of Education. Kimbell, R., Stables, K., & Green, R. (1996). Understanding practice in design and technology Buckingham: Open University Press. Knapp,M S., Shields,P M., & TurnbullB J. (1995). Academic challenge in high poverty classrooms. Phi Delta Kappan, 76, 770 776. Kolodner, J.L. (1993). Case based reasoning San Mateo, CA: Morgan Kaufman Publishers. Kolodner, L. J. (2002). Facilitating the learning of design practices: Less ons learned from inquiry into science education. Journal of Industrial Teacher Education, 39 (3). Kolodner, J. L., Camp, P. J., Crismond, D., Fasse, B., Gray, J., Holbrool, J., Puntambekar, S., & Ryan, M. (2003). Problem based learning meets case based rea soning in the middle school science classroom: putting learning by design into practice. J Learn Sci 12(4), 495 547. Kost, L. E., Pollock, S. J. & Finkelstein, N. D. (2009). Unpacking gender differences in hysics. PERC Proceedings. Krajcik, J. S., Blumenfeld, P. C., Marx, R. W., Bass, K. M., Fredricks, J., & Soloway, E. (1998). Inquiry in project based science classrooms: Initial attempts by middle school students. The Journal of the Learning Sciences, 7 (3& 4), 313 350. Krajcik, J., Czerniak, C., & Berger, C. (2002). Teaching science in elementary and middle school classrooms: A project based approach (2nd ed.). Boston: McGraw Hill. Krathwohl, D. R. (2004). Methods of educational and social science research : An integrated approach. Long Grave, IL: Waveland Press Inc.
138 Krause, S., Birk, J., Bauer, R., Jenkins, B. & Pavelich, M. J. (2004). Development, testing, and application of a chemistry concept inventory. 34 th ASEE/IEEE Frontiers in Education Conference, Savannah, GA. LaVeist, T. A. (2005). Disentangling race and socioeconomic status: A key to understanding health inequalities. Journal of Urban Health: Bulletin of the New York Academy of Medicine 82(2), 26 34. Lee, H. S., & Songer, N.B. (2003). Making authentic science accessible to students. International Journal of Science Education, 25, 923 948. Leech, N. L., Barrett, K. C. & Morgan, G. A. (2008). SPSS for intermediate statistics: Use and interpretation, 3 rd edn. New York, NY: Taylor & Francis Group. Lehman, J. D., George, M., Buchanan, P. and Rush, M. (2006). Preparing teachers to use problem centered, inquiry based science: Lessons from a four year professional development project. Interdisciplinary Journal of Problem based Learning 1(1), 76 99. Lipman, M. (1991). Thinking in education. New York: Cambridge University Press. Ma mlok, R. Dershimer, C. Fortus, D., Krajcik, J & Marx, R (2001). Learning science by designing artifacts (LSDA): A case study of the development of a design based science curriculum. Paper presented at the Annual Meeting of the National Association for Research in Science Teaching. Retrieved March 30, 2012 from http://www.eric.ed.gov/PDFS/ED458127.pdf Maple, S. A., & Stage, F. K. (1991). Influences on the choice of math/science major by gender and ethnic ity. American educational Research Journal, 28(1) 37 60. Maxwell, S., Delaney H.D. & Manheimer, J. (1985). ANOVA of residuals : Correcting an il lusion by using model comparisons and graphs, Journal of Educational Statistics 95 136 147. Mayer, R. E. ( 1975). Information processing variables in learning to solve problems. Review of Educational Research, 45, 525 541. Mayer, R. E. & Wittrock, M. C. (2006) olving. In P. A. Alexander and P. H Winne (Eds.), Handbook of Educational Psychology (2nd ed.). Mahwah, NJ: Lawrence Erlbau Associates. Mazzeo, C., Rab, S. Y., & Alssid, J. L. (2003). Building bridges to college and careers: Contextualized basic skills programs at community colleges Brooklyn, NY: Workfo rce Strategy Center.
139 McKinsey & Company (2009). The economic impact of the achievement gap in American schools. Washington, DC; McKinsey & Company. Mehalik, M. M., Doppelt, Y. and Schunn, C. D. (2008). Middle school science through design based learning versus scripted inquiry: better overall science concept learning and equity gap reduction. J Eng Educ 97(1), 1 15. Miller K. (1996). Differences between expert and novice writers Unpublished manuscript. Miller, P. H., Blessing, J. S., & Schwartz, S. (2006). Gender differences in high school International Journal of Science Education 28, 363 381. Mooney, M., & Laubach, T. A. (2002). Adventure engineering: a design centered, inquiry based approach to middl e grade science and mathematics education. Journal of Engineering Education, 91(3), 309 318. Mulford D S (1996) Unive rsity, IN Mullis, I. V. S., Dossey, J. A., Owen, E. H., & Phillips, G. W. (1993). NAEP 1992 math report card for the nation and the states. Washington, DC: Government Printing Office. National Assessment of Educational Progress. (2005 ). ort Card: Science Retrieved August 8, 2008, from http://nationsreportcard.gov/science_2005 National Education Longitudinal Study. (2004). Washington, DC: National Center for Educational Statistics. Retrieved March 2 7, 2012 from http://nces.ed.gov/surveys/nels88/ National Res earch Council. (1996) National Science Education Standards. Washington, DC: National Academy Press. National Research Council. ( 1996 ) The national science standards. Retrieved April 5, 2005 from the World Wide Web: http://www.nap.edu/readingroom/books/nses foundation. NSF, Arlington, VA. NSF, Division of Science Resou rces Statistics, Science and Engineering Degrees: 1966 2006 (detailed statistical tables in NSF 08 321, NSF, Arlington, VA, 2008); available at www.nsf.gov/statistics/nsf08321/
140 National Science Foun dation. (2008). Science and Engineering Indicators 2008 Retrieved April 7, 2012 from http://www.nsf.gov/statistics/seind08/c0/c0i.htm Newell A, Simon H. (1972). Human Problem Solving. Englewood Cliffs, NJ: Prentice Hall. Next Generation Science Standards Draft II. (2013). Retrieved April 3, 2013, from http://www.nextgenscience.org Nickerson, R.S. (1994). The teaching of thinking and problem solving. In R.J. Sternberg (Ed.), Thinking and problem solving. San Diego, CA: Academic Press. Novak, A. & Krajcik, J. S. (2005). "Using Learning Technologies to Support In quiry in Middle School Science. In Flick, L. & Lederman, N. (Eds) Scientific Inquiry and Nature of Science: Implicat ions for Teaching, Learning, and Teacher Education. Netherlands: Kluwer Publishers. Oakes, J. (1990). Opportunities, achievement and choice: Women and minority students in science and mathematics. Review of Research in Education, 16, 153 222. OECD. (2003 ). The PISA 2003 Assessment Framework. Mathematics, Reading, Science and Problem Solving Knowledge and Skills Paris: OECD. OECD. (2004). Problem Solving for First Measures of Cross Curricular Competencies from PISA 2003 Paris: OECD OEC D. (2005). PISA 2003 Technical Report Paris: OECD techniques) Ankara. Ormrod, J.E. (1987). Human learning. Upper Saddle River, NJ: Prentice Hall. Osborn, A. F. (196 3). Applied Imagination. New York, NY: Scribners. Partnership for 21 st Century Skills. (2009). 21 st century skills map. Retrieved February 20, 2012 at www.21stcenturyskills .org Parnes, S. J. (1967). Creative Behavior Guidebook. New York, NY: Scribners. Peng, S. S., Wright, D., & Hill, S. T. (1995). Understanding racial ethnic differences in secondary school science and mathematics (NCES Research and Development Report 95 710). Washington, DC: U.S. Government Printing Office. Petersdorf, R. G. (1991). N ot a choice, an obligation. Washington, DC: Association of American Medical Colleges.
141 Peterson, K. D., & Yaacobi, D. (2006). Self concept and perceptions of role behavior of high school science students and teachers: New assessment instruments. Journal of Research in Science Teaching, 16(5), 433 438. Piaget, J. (1977). The development of thought: Equilibrium of cognitive structures. Pizzini, E. (1987). Project STEPS: Science Textbook Extensions through Problem Solving. Washington, DC: National Science Fou ndation. Pizzini, E., Abell, S. & Shepardson, D. (1988). Rethinking thmking in the Science classroom. The Science Teacher, 55, 22 25. Pizzini, E.L., Shepardson, D.P., & Abell, S.K. (1989). A rationale for and the development of a problem solving model of instruction in Science Education. Science Education, 73(5), 523 534. Plucker, J. A., Burroughs, N., & Song, R. (2010). Mind the (other) gap! The growing excellence gap in K 12 education Center for Evaluation and Education Policy. Bloomington, IN: Indian a University School of Education. Plya, G. (1945). How to Solve It. Princeton, NJ: Princeton University Press. Prepare and Inspire: K 12 Science, Technology, Engineering, and Ma th (STEM) States. Retrieved February 2, 2012 from http://www.whitehouse.gov/ostp/pcast. Project Lead the Way. (2011). Overview of Project Lead the Way. Retrieved January 31, 2 012 from http://www.pltw.org/ Pollock, S. J., Finkelstein, N.D. & Kost, L. (2007). Reducing the Gender Gap in the Physics Classroom PR ST Phys. Ed. Rsrch. 3 010107. Presseisen B. Z. (1985). Thinking skills: Meanings and models. In A. L. Costa (Ed.) Developing Minds: A Resource Book for Teaching Thinking, (pp. 34 48). Alexandria, VA: Association for Supervision and Curriculum Development. Puntambekar, S. & Kolodner, J. (2005). Toward implementing distributed scaffolding: Helping students learn science from d esign Journal of Research in Science Teaching, 42( 2 ), 185 217
142 Reardon, S. (2008). Differential growth in the black white achievement gap during elementary school among high and low scoring students (Working Paper 2008 7). Stanford, CA: Institute for Research on Education Policy & Practice, Stanford University. Retrieved from: http://www.stanford.edu/group/irepp/cgi bin/joomla/docman/differential growth in the black white achievement gap reardon/download.html Reeff, J. P., Zabal, A. & Blech. C. (2 006). The Assessment of ProblemSolving Competencies. A Draft Version of a General Framework. Bonn: Deutsches Institut fr rwa chsenenbildung. (Retrieved March 3, 2012 from http ://www.die bonn.de/esprid/dokumente/doc 2006/reeff06_01.pdf ). Rivet, E. A. & Krajcik, S. J. (2004). Achieving standards in urban systematic reform: An example of a sixth grade project based science curriculum. Journal of Research in Science Teaching, 41 (7), 669 692. prior knowledge and experiences to foster understanding of middle school science. Journal of Research in Science Teaching, 45(1) 79 100. Roberts, P. ( 1995). The place of design in technology education. In D. Layton (Ed.) Innovations in science and technology education 5, UNESCO. Rorty, R. (1991). Objectivity, relativism, and truth: Philosophical papers. 1 Roychoudhury, A., Tippins, D.J., & Nichols, S .E. (1995). Gender inclusive science teaching: A feminist constructivist approach. Journal of Research in Science Teaching, 32, 897 924. Savery, J. R. (2006). Overview of problem based learning: Definitions and distinctions. Interdisciplinary Journal of P roblem based Learning 1(1) 9 20. Savery, J.R. & Duffy, T. M. (1995). Problem Based learning: An instructional model and its constructivist framework. Educational Technology 35(5) 31 38. Scantlebury, K., & Baker, D. (2007). Gender issues in science ed ucation research: Remembering where the difference lies. In S. Abell & N. Lederman (Eds.), Handbook of research on science education (pp. 257 286). Mahwah, NJ: Lawrence Erlbaum. Scott, L. A., Rock, D. A., Pollack, J. M., Ingels, S. J., & Quinn, P. (1995). Two years later: Cognitive gains and school transitions of NELS: 88 eighth graders (National Center for Education Statistics Report NCES 95 436). Washington, DC: U.S. Government Printing Office.
143 Seiler, G; Tobin, K; & Sokolic, J. (2001) Design, technology, and science: Sites for learning, resistance, and social reproduction in urban schools. Journal of research in science teaching 38 (7), 746 767. Shepardson, D. P., & Pizzini, E. L. (1994). Gender, achievement, and perception towards science activities. School Science and Mathematics 94(4), 18 25. Simon, D. P., & Simon, H. A. (1978). Individual differences in solving physics problems. In R. S. Siegler (Ed.) Hillsdale, NJ: Lawrence Erlbaum Associat es. Singer, J., Marx, R.W., Krajcik, J.S., & Clay Chambers, J. (2000). Constructing extended inquiry projects: Curriculum materials for science education reform. Educational Psychologist, 35, 165 178. Silk, E. M., Schunn, C. D., & Strand, C. M. (2007). The impact of an engineering design curriculum on science reasoning in an urban setting. In: Proceedings of the National Association for Research in Science Teaching New Orleans, LA, USA. 15 17. Sternberg, R. J. (1985). Critical th inking: Its nature, measurement, and improvement. In F. R. Link (Ed.) Essays on the Intellect. Alexandria, VA: Association for Supervision and Curriculum Development. ` Stevens, J. (1996). Applied multivariate statistics for the social sciences ( 3rd ed. ). New Jersey: La wrence Erlbaum Stewart, J. (1982). Two aspects of meaningful problem solving in science. Science Education, 66, 731 741. Sullivan, F.R. (2006). The ideal science student and problem solving. Dissertation Abstracts International: Section A : Humanities and Social Sciences. 66(9 A), 3214. Sullivan, F. R., & Lin, X. (2006). The ideal science student and problem solving. 7 th International Conference on Learning Sciences, 737 743. Retrieved April 19, 2010 from World Wide Web: http://portal.acm.org Taber K S (2003) Mediating mental models of metals: acknowledging the priority of Science Education 87 732 758 The Center for Education and Work, 2008 Increasing STEM retention for underrepresented students. Retrieved July 23, 2012, from www.cew.wisc.edu
144 Thomas, J. W., Mergendoller, J. R., & Michaelson, A. (1999). Project based learning: A handbook for middle and high school teachers. Novato, CA: The Buck Institute for Education. Thomas, J. W. (2000). A review of research on project based learning. San Rafael, CA: Autodesk Foundation. Torp, L., & Sage, S. (2002). Problems as possibilities: Problem based learning for K 16 education (2nd ed n .). Alexandria, VA: Association for Supervision and Curriculum Development. Ulrich KT, Eppinger SD (2004) Product design and development, 3 rd edn. Boston, MA: McGraw Hill/I rwin. U.S. De partment of Labor. (1991). What work requires of schools: A SCANS. Springfield, Missouri: University School of Medicine. U.S. Bureau of Labor Statistics. (2009); available at http://ftp. bls.gov/pub/special.requests/lf/aat11.txt U. S. Government Accountability Office. (2005). Higher Education: Federal Science, Technology, Engineering, and Mathematics Programs and Related Trends. Washington, DC: U. S. Government Accountability Office. va n Langen, A., & Dekkers, H. (2005). Cross national differences in participating in tertiary science, technology, engineering and mathematics education. Comp Educ., 41(3), 329 350. Visser, Y. L. (April, 2002). Effects of problem based and lecture based ins tructional Strategies on problem solving performance and learner attitudes in a high school genetic class. Paper presented at the 2002 Annual Meeting of the American Educational research Association, New Orleans. Volet, S., McGill, T., & Pears, H. (1995). Implementing process based instruction in regular university instruction: Conceptual, methodological and practical issues. European Journal of Psychology of Education, 10, 385 400. Von Glasersfeld, E. (1989). Cognition, construction of k nowledge, and teaching. Wavering, J. J. (1980). What are the basics of science education? What is important to know, how to use knowledge or how to obtain answers? Science and Mathematics, 80, 633 637. Winne, P. & Mark, P. (1977). Reconceptualizing research on teaching. Journal of Educational Psychology, 69, 668 678.
145 Wisely, W. C. (2009). Effectiveness of contextual approaches to developmental math in California community colleges (Unpublished doctoral dissertation). University of the Pacific, Stoc kton, CA. Woods, D. R. 1994. Problem based learning: How to gain the most from PBL. Waterdown, Ontario: Donald R. Woods. Wrigley, H. S. (1998). Knowledge in action: The promise of project based learning. Focus on Basics 2 (D), 13 17. Zoller, U. (1987). The fostering of question asking capability: A meaningful aspect of problem solving in chemistry. Journal of Chemical Education, 64, 5 10 5 12.
146 APPENDIX A
148 APPENDIX B PISA 2003 SAMPLE PROBLEMS AND SCORING RUBRIC Energy Needs
150 Cinema Outing
153 Transit System
155 Library System
157 Design by Numbers
159 Course Design
167 Appendix C Features of the Three Types of Problem Solving skills
168 Appendix D PISA 2003 Problem Solving Scale
169 APPENDIX E ASSESSING ACADEMIC GAINS FROM THE HEATING/COOLING UNIT Chemical Concepts Inventory Sample Items: This inventory consists of 22 multiple choice questions. Carefully consider each question and indicate the one best answer for each. Several of the question s are paired. In these cases, the first question asks about a chemical or physical effect. The second question then asks for the reason for the observed effect. 1. Which of the following must be the same before and after a chemical reaction? a. The sum of the masses of all substances involved. b. The number of molecules of all substances involved. c. The number of atoms of each type involved. d. Both (a) and (c) must be the same. e. (e) Each of the answers (a), (b), and (c) must be the same. 2. Assume a beaker of pure water has been boiling for 30 minutes. What is in the bubbles in the boiling water? a. Air. b. Oxygen gas and hydrogen gas. c. Oxygen. d. Water vapor. e. Heat. 3. A glass of cold milk sometimes forms a coat of water on the outside of the glass (Ofte n referred to as 'sweat'). How does most of the water get there? a. Water evaporates from the milk and condenses on the outside of the glass. b. The glass acts like a semi permeable membrane and allows the water to pass, but not the milk. c. Water vapor condense s from the air. d. The coldness causes oxygen and hydrogen from the air combine on the glass forming water. 4. What is the mass of the solution when 1 pound of salt is dissolved in 20 pounds of water? a. 19 Pounds. b. 20 Pounds. c. Between 20 and 21 pounds. d. 21 p ounds. e. More than 21 pounds.
170 5. The diagram represents a mixture of S atoms and O 2 molecules in a closed container. Which diagram shows the results after the mixture reacts as completely as possible according to the equation: 2S + 3O 2 2SO 3 6. The circle on the left shows a magnified view of a very small portion of liquid water in a closed container. What would the magnified view show after the water evaporates? 7. True or False? When a match burns, some matter is destroyed.
171 a. True b. False 8. What is the reason for your answer to question 7? a. This chemical reaction destroys matter. b. Matter is consumed by the flame. c. The mass of ash is less than the match it came from. d. The atoms are not destroyed, they are only rearranged. e. The match weigh s less after burning. 9. Heat is given off when hydrogen burns in air according to the equation 2H 2 + O 2 2H 2 O Which of the following is responsible for the heat? a. Breaking hydrogen bonds gives off energy. b. Breaking oxygen bonds gives off energy. c. Forming hydrogen oxygen bonds gives off energy. d. Both (a) and (b) are responsible. e. (a), (b), and (c) are responsible. 10. Two ice cubes are floating in water: After the ice melts, will the water level be: a. higher? b. lower? c. the same? 11. What is the re ason for your answer to question 10? a. The weight of water displaced is equal to the weight of the ice. b. Water is more dense in its solid form (ice). c. Water molecules displace more volume than ice molecules. d. The water from the ice melting changes the water level.
172 e. When ice melts, its molecules expand. 12. A 1.0 gram sample of solid iodine is placed in a tube and the tube is sealed after all of the air is removed. The tube and the solid iodine together weigh 27.0 grams. The tube is then heated until all of the iodine evaporates and the tube is filled with iodine gas. Will the weight after heating be: a. less than 26.0 grams. b. 26.0 grams. c. 27.0 grams. d. 28.0 grams. e. more than 28.0 grams. 13. What is the reason for your answer to question 12? a. A gas weighs less than a solid. b. Mass is conserved. c. Iodine gas is less dense than solid iodine. d. Gasses rise. e. Iodine gas is lighter than air. 14. What is the approximate number of carbon atoms it would take placed next to each other to make a line that would cross this dot: a. 4 b. 200 c. 30,000,000 d. 6.02 x 10 23 15. Figure 1 represents a 1.0 L solution of sugar dissolved in water. The dots in the magnification circle represent the sugar molecules. In order to simplify the diagram, the water molecules have n ot been shown.
173 Which response represents the view after 1.0 L of water was added (Figure 2). 16. 100 mL of water at 25C and 100 mL of alcohol at 25C are both heated at the same rate under identical conditions. After 3 minutes the temperature of the alcohol is 50C. Two minutes later the temperature of the water is 50C. Which liquid received mor e heat as it warmed to 50C? a. The water. b. The alcohol. c. Both received the same amount of heat. d. It is impossible to tell from the information given. 17. What is the reason for your answer to question 16? a. Water has a higher boiling point then the alcohol. b. Water takes longer to change its temperature than the alcohol. c. Both increased their temperatures 25C. d. Alcohol has a lower density and vapor pressure. e. Alcohol has a higher specific heat so it heats faster. 18. Iron combines with oxygen and water from the air to form rust. If an iron nail were allowed to rust completely, one should find that the rust weighs: a. less than the nail it came from. b. the same as the nail it came from. c. more than the nail it came from. d. It is impossible to predict.
174 19. What is the reason for your answer to question 18? a. Rusting makes the nail lighter. b. Rust contains iron and oxygen. c. The nail flakes away. d. The iron from the nail is destroyed. e. The flaky rust weighs less than iron. 20. Salt is added to water and the mixture is stirred until no more salt dissolves. The salt that does not dissolve is allowed to settle out. What happens to the concentration of salt in solution if water evaporates until the volume of the solution is half the original volume? (Assume temperature remains constant.) The concentration a. increases. b. decreases. c. stays the same. 21. What is the reason for your answer to question 20? a. There is the same amount of salt in less water. b. More solid salt forms. c. Sa lt does not evaporate and is left in solution. d. There is less water. 22. Following is a list of properties of a sample of solid sulfur: i. Brittle, crystalline solid. ii. Melting point of 113 o C. iii. Density of 2.1 g/cm 3 iv. Combines with oxygen to form sulfur dioxi de Which, if any, of these properties would be the same for one single atom of sulfur obtained from the sample? a. i and ii only. b. iii and iv only.
175 c. iv only. d. All of these properties would be the same. e. None of these properties would be the same.
176 APPENDIX F CHEMISTRY CONCEPTS AND BIG IDEAS IN HEATING AND COOLING UNIT Subsystem & Big Idea Key concepts Reaction I Energy released or absorbed during chemical transformations is dependent on the shape and structure of the particles involved in the transformation. that have mass and occupy space. The composition of particles determines their physical and chemical properties. an increase or decrease in tempera ture. temperature of the system. Endothermic reactions are measured by a decrease in the temperature of the system. with each other. eractions are the attraction between particles. Interactions between particles may result in transformations. particles. temperature o f the reaction involving the rearrangement of these ions will be lower. to the nucleus and the attraction of the valence electrons of one nucleus to another nucleus. Reaction II Energy released or absorbed during chemical transformations is dependent on the mass and temperature change in the system. results in more particle interactions, and consequently increases the energy of the system. the system. e not directly proportional to increases/ decreases in temperature. energy. Container Energy transfers from particles with high kinetic energy to particles with lower kinetic energy through collisions. composition that determines how they interact with the environment. when two objects are in contact. the transfer of kinetic energy through conduction. to transfer energy between adjoining atoms. quickly are called conductors. insulators.
177 APPENDIX G STUDENT ACTIVITY FOR CONTROL AND TREATMENT GROUPS Topic Experimental Group Activity Control Group Activity Pretests Students provide biographical data (race gender, SES) Students take pretests (CCI) and PISA Problem Solving Students provide biographical data (race gender, SES) Students take pretests (CCI) and PISA Problem Solving Introduction Engineering Design Students assigned to watch Class discussion to relate the two videos in terms of the design process. Students completed worksheet enabling them to t hink about how the engineering design process was applied by End of unit design challenge project presented, with options clearly stated on handout Students watched the video and had t o write an essay about the role of science in ending WWII. No end of unit design challenge project *Unit Activities Conservation of Mass Students were provided with materials and required to design an experiment to determine if mass is conserved at the end of a chemical reaction. Types of Chemical Reactions Students learned about seven types of chemical reactions: Combustion, Synthesis, Single displacement, Double displacement, Decomposition, Neutraliza tion and Redox. (Lecture, worksheets, homework, experiments test) Students performed experiments involving endothermic and exothermic reactions. Students designed a prototype gas powered rocket propelled by igniting a mixture of hydrogen and oxygen gases i n a plastic tube (rocket). Students required to design an experiment to determine the best solvent for paints (from double displacement reactions). Students practiced how to write different types of chemical Conservation of Mass Students were presented with directions as to how to carry out the same experiment. Types of Chemical Reactions Students learned about seven types of chemical reactions: Combustion, Synthesis, Single displacement, Double displacement, Decomposition, Neutralization and Redox. (Lecture, worksheets, homework, experiments, test) Students performed experiments involving endothermic and exothermic reactions. No gas po wered experiments Students provided instructions on how to determine the best of three solvents for paint. Students practiced how to write different types of chemical equations. Students prepare wet cells and investigate factors that determine optimum volt age
178 equations. Students prep are wet cells and investigate factors that determine optimum voltage Mini Project design challenge: students required to convert their wet cell into a dry (portable) cell to charge their cell phones any time. Students design experiments to investigate factors that affect speed of chemical reactions. Students perform electroplating experiment and research the applications of electroplating and sacrificial anode Design Cycle activity Stoichiometry Students practice balancing of chemical equations Students practice solving stoichiometric problems. No mini project Students provided instructions on how to investigate the effects of temperature, concentration, surface area and specific substances (catalysts) on reaction rate. Students perform electroplating experiment (No applications research) Stu dents practice balancing of chemical equations Students practice solving stoichiometric problems. Students practice balancing of chemical equations Students practice solving stoichiometric problems. Unit Design Challenge Project Students design solutions to problems: options include a) using heat energy change from chemical reactions to solve problems at a winter camp, summer camp, dating, sports, etc. b) design an age appropriate toy that uses chemistry concepts learned in this unit De s ign Cycle activity Designs to be presented to classmates as a board of a company looking for ideas to invest in. No unit design Project Posttests Complete CCI PISA Problem Solving Competency test Design challenge open ended survey (Appendix H) Complete test APPENDIX G CONTINUED *Unit on atomic structure and chemical bonding was completed before above unit.
179 APPENDIX H SITE PRINCIPAL APPROVAL
180 APPENDIX I DISTRICT APPROVAL LETTER September 13, 2012 Your research project Effects of design based chemistry instruction on the science problem solving skills and science achievement among different groups of high school students has been reviewed and approved by the Denver Public Schools IRB for implementation based upon the following conditions: 1. The voluntary nature of the study is mad e clear to all potential participants including pupils, teachers, and administrators. participate 2. The researchers agree to maintain the anonymity of the re search participants as outlined in your proposal. 3. All rules in the district's research procedures are followed including maintaining the anonymity of the district, the schools, and the study participants. 4. If your request involves the release of data you agree to limit the use of said data to the terms specified in your application. The data will not be released to any third party and you agree not to copy, reproduce, disseminate transmit, license, sublicense, assign, lease, or release the data to any oth er party. All data should be maintained in a secure fashion with access being restricted to the persons identified in the research application to prevent unauthorized use of the data. Following the use of the data for the prescribed reasons the data shou ld be destroyed. 5. This letter does not reflect a commitment on behalf of Denver Public Schools towards the requestor. At any point the approval status involving the release of data or access to students/staff for research may be withdrawn. A violation o f any of the conditions within this letter and/or deceptive practices by the researcher will lead to immediate termination of all research privelages. Furthermore, the release of future data and/or research privelages may be indefinitely terminated. 6. A re port of the findings is made available to the Department of Accountability, Research & Evaluation at the conclusion of the study. 7. This letter is returned by mail or via FAX (720 423 3646) prior to initiating your study with the requestor acknowledging agr eement with the terms described above by signature. Please contact Accountability, Research & Evaluation at 720 423 3736 if you have any questions. DENVER PUBLIC SCHOOLS 900 Grant Street, Rm 610, Denver, CO 80203 Department of Accountability, Research & Evaluation
181 Please return this letter with the following statement verified by signature: I, __ Cobina Adu Lartson _, agree to abide by the conditions described in this document and will carry out my research practices in accordance with those conditions. I assume complete responsibility for the described study and will work according to best practices w hen working with Denver Public Schools data and/or conducting scientific inquiry within the Denver Public Schools district. __________________________ Signature of Requestor
182 APPENDIX J PARENT CONSENT LETTER
183 APPENDIX K STUDENT CONSENT LETTER