Citation
Predicting undergraduate mathematics success

Material Information

Title:
Predicting undergraduate mathematics success an analysis of UCDHSC placement methods and the accuplacer exam
Creator:
Olson, Gary A
Publication Date:
Language:
English
Physical Description:
xii, 63 leaves : ; 28 cm

Subjects

Subjects / Keywords:
Accuplacer (Computer program) ( lcsh )
Prediction of scholastic success -- Colorado -- Denver ( lcsh )
Mathematical readiness ( lcsh )
Mathematics -- Study and teaching (Higher) -- Colorado -- Denver ( lcsh )
Genre:
bibliography ( marcgt )
theses ( marcgt )
non-fiction ( marcgt )

Notes

Bibliography:
Includes bibliographical references (leaves 61-63).
General Note:
Department of Mathematical and Social Sciences
Statement of Responsibility:
by Gary A. Olson.

Record Information

Source Institution:
|University of Colorado Denver
Holding Location:
|Auraria Library
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
62879140 ( OCLC )
ocm62879140
Classification:
LD1193.L622 2005m O57 ( lcc )

Full Text
PREDICTING UNDERGRADUATE MATHEMATICS SUCCESS: AN
ANALYSIS OF UCDHSC PLACEMENT METHODS AND THE
ACCUPLACER EXAM
by
Gary A. Olson
B.A., Carroll College, 2003
A thesis submitted to the
University of Colorado at Denver
in partial fulfillment
of the requirements for the degree of
Master of Science
Applied Mathematics
2005


This thesis for the Master of Science
degree by
Gary A. Olson
has been approved
Midge Cozzens


Olson, Gary A. (M.S., Applied Mathematics)
Predicting Undergraduate Mathematics Success: An Analysis of UCDHSC
Placement Methods and the Accuplacer Exam
Thesis directed by Professor Michael Jacobson
ABSTRACT
Predicting student success has long been a question of interest for admis-
sion counselors throughout the United States. With the changing face of the
nations post-secondary population, a need has also developed for mathematics
departments to address this question.
Existing research examines the validity of several methods designed for pre-
dicting undergraduate success. High school record, standardized test scores,
extra-curricular activities and combinations of all three have historically been
successful predictors. However, limited research has been conducted on predict-
ing success in lower-level mathematics courses.
In recent years standardized test scores have become less valuable for plac-
ing students in lower level mathematics courses; placement exams such as
ACCUPLACER have taken their place. Validity studies for these exams are
crucial for institutions to analyze cut-off scores and ensure students are appro-
priately placed in courses that match their skill-level.
In June 2003, the Colorado Commission of Higher Education (CCHE) ap-
proved three tests for the purpose of entry and secondary-level assessment: the
m


ACT Assessment Test, the College Board Scholastic Aptitude Test, and the
College Board ACCUPLACER. In Spring 2004, the University of Colorado at
Denver and Health Sciences Center (UCDHSC) began using ACCUPLACER
scores as mandatory placement criteria for all mathematics students enrolling
in three courses, Analytical Geometry & Calculus I, College Algebra, and Alge-
bra for Social Sciences and Business. Two of the initial goals of the placement
procedure were to lower withdrawal rates and increase passing (A/B/C) rates
for the three courses.
This thesis begins with a literature search and analysis of past research
devoted to predicting college success. Major studies from 1980 to 2004 are pre-
sented and compared. Research on the prediction of lower-level undergraduate
mathematics success is also discussed as well as two current studies of mathe-
matics placement validity.
The ACCUPLACER Exam is then analyzed along with a comparison of
placement techniques for different post-secondary institutions throughout Col-
orado. UCDHSC course data are then analyzed and predictors of mathematics
success are discussed. Finally, statistical analysis is performed for all three
classes and conclusions are drawn about the impact of ACCUPLACER on both
withdrawal and passing rates.


This abstract accurately represents the content of the candidates thesis. I
recommend its publication.
v


DEDICATION
This thesis is dedicated to my parents and three sisters for all of their love and
support during my academic pursuits.
!
I


ACKNOWLEDGMENT
There are many people that I would like to thank for their help during my thesis
experience. First, I would like to thank my two readers and committee members,
Dr. William Briggs and Dr. Midge Cozzens for all of their help and support.
I would also like to thank my advisor Dr. Michael Jacobson for all of his help
during the last year. I am grateful for the opportunity and support he gave me
to perform research in the area of mathematics education. I would also like to
thank Jennifer Thurston for her help in the extraction of my data set and all of
the statistics professors and students at UCD for their insight. A special thanks
also goes to Leslie Varys for her SPSS insight and statistical genius.


CONTENTS
Figures ............................................................. x
Tables.............................................................. xi
Chapter
1. Introduction...................................................... 1
2. Classical Methods for Predicting Long-Term College Success.... 3
2.1 SAT Verbal Scores............................................... 4
2.2 SAT Math Scores................................................. 5
2.3 SAT Verbal and SAT Math Scores ...................... 6
2.4 High School Record.............................................. 7
2.5 SAT Verbal Score, SAT Math Score, and High School Record ... 9
2.6 Extra-Curricular Activities.................................... 10
2.7 Problems Associated with Predictors of College Success......... 12
3. Placement Techniques for Lower-Level Undergraduate Courses .... 14
3.1 High School Coursework......................................... 15
3.2 SAT/ACT Scores................................................. 15
3.3 Placement Tests................................................ 16
3.4 Placement Exam Validity ......................... 17
4. The ACCUPLACER Exam ............................................. 20
5. Current Placement Techniques for Colorado Colleges .............. 23
6. UCDHSC Placement Techniques for Spring 2004 ..................... 30


6.1 Data Description.................................................. 31
6.2 Correlation Among Variables ...................................... 31
6.3 Analysis of the Results........................................... 34
6.4 Limitations of the Data .......................................... 34
6.5 Analyzing College Algebra Cut-Off Scores.......................... 35
6.6 MA 1110 Fall 2004 Statistics...................................... 38
7. Current ACCUPLACER Results.......................................... 40
7.1 Analysis of MA 1401............................................... 41
7.2 Analysis of MA 1110............................................... 44
7.3 Analysis of MA 1070 .............................................. 46
8. Discussion.......................................................... 49
9. Conclusions......................................................... 51
9.1 Future Analysis................................................... 52
Appendix
A. Descriptive Statistics Tables....................................... 54
References............................................................. 61


FIGURES
Figure
6.1 Aecuplacer Versus Final Grade Spring 2004 ....................... 35
6.2 Distribution of Course Grades for Students Scoring Between 72-85 . 36
6.3 Aecuplacer Versus Final Grade Fall 2004 ......................... 39
x


TABLES
Table
2.1 Predicting Cumulative GPA From SAT Verbal Scores.............. 4
2.2 Predicting Cumulative GPA from SAT Math Scores................ 5
2.3 Predicting Cumulative GPA From SAT Verbal & Math Scores ... 7
2.4 Predicting Cumulative GPA from High School Record ................. 8
2.5 Predicting GPA From SAT Verbal, Math, & High School Record . 10
6.1 Numerical Equivalent of Letter Grades......................... 32
6.2 Correlations for College Algebra Spring 2004 ..................... 33
6.3 Correlations for College Algebra Fall 2004 ....................... 38
7.1 Independent-Samples T-Test for MA 1401 Withdrawals............ 42
7.2 Independent-Samples T-Test for MA 1401 Mean Student Grades . . 43
7.3 Calculus Comparisons.......................................... 43
7.4 Independent-Samples T-Test for MA 1110 Withdrawals............ 45
7.5 Independent-Samples T-Test for MA 1110 Mean Student Grades . . 45
7.6 College Algebra Comparisons................................... 46
7.7 Independent-Samples T-Test for MA 1070 Withdrawals............ 47
7.8 Independent-Samples T-Test for MA 1070 Mean Student Grades . . 48
7.9 Business Algebra Comparisons ..................................... 48
A.l Descriptive Statistics for MA 1401 Withdrawals................ 55
A.2 Descriptive Statistics for MA 1110 Withdrawals................ 56


A.3 Descriptive Statistics for MA 1070 Withdrawals....................... 57
A.4 Descriptive Statistics for MA 1401 Student Grades.................... 58
A.5 Descriptive Statistics for MA 1110 Student Grades.................... 59
A.6 Descriptive Statistics for MA 1070 Student Grades.................... 60


1. Introduction
Post-secondary mathematics departments currently use a variety of methods
to place mathematics students in courses commensurate with their abilities.
Each institution develops placement techniques based upon curriculum, student
population, methodology, and placement test accessibility. Present mathematics
placement methods include the use of ACT and SAT standardized test scores,
high school record, a combination of standardized test scores and high school
record, and placement exams like ACCUPLACER.
Since 2002, the University of Colorado at Denver and Health Sciences Center
(UCDHSC) has required all freshmen without AP credits or completed college
level courses in mathematics to take the math ACCUPLACER. Beginning with
spring semester 2004, the UCDHSC mathematics department began using math
ACCUPLACER scores for placement decisions in three different mathematics
classes: MA 1070-Algebra for Social Sciences and Business, MA 1110-College
Algebra, and MA 1401-Analytical Geometry and Calculus I.
After three semesters of using ACCUPLACER scores, it is now relevant
to examine the validity and success of the current methods. This paper first
explores the classical methods used by admission officers to predict academic
success throughout college. Placement techniques for lower-level undergraduate
classes are then discussed and placement validity is examined for two cases in the
literature. Next the ACCUPLACER exam itself is analyzed along with current
placement procedures at four-year institutions throughout Colorado. Finally,
1


UCDHSC mathematics placement techniques are discussed and correlations are
developed to examine predictors of success in UCDHSC College Algebra courses.
Statistical analysis is then used to analyze withdrawal rates, passing rates, and
mean student grades for ACCUPLACER required classes.
Initial results show that ACCUPLACER is a statistically significant predic-
tor of both final exam score and final course grade for MA 1110 students. Also,
withdrawal rates for all three courses has statistically decreased with the use
of ACCUPLACER. Mean student grades for MA 1401 students after ACCU-
PLACER have increased, but analysis shows no statistical difference for mean
student grades of MA 1110 and MA 1070. Finally, passing rates have increased
for MA 1401, but no statistical change is observed for MA 1110 and MA 1070.
2


2. Classical Methods for Predicting Long-Term College Success
Predicting student success has long been a question of interest for admission
decisions. Counselors aim to admit students who will be successful in their
chosen program and eventually attain their degree. Research suggests that the
best predictor of college graduation is first year college grade point average [27].
In addition, first year grades are the single most revealing indicator of successful
adjustment to the intellectual demands of a particular colleges course of study
[10]. To aid counselors who seek predictions of first year GPA from just an
application packet, researchers have analyzed the best methods for estimating
first year college GPA.
Since 1980, several major studies have analyzed predictors of college success.
Throughout most of the studies, five main methods are analyzed: SAT Verbal
score, SAT Math Score, a combination of SAT Verbal and SAT Math score,
High School Record, and a combination of SAT Verbal score, SAT Math score,
and High School Record. In addition to these five cognitive predictors, non-
cognitive factors such as extra-curricular activities, admission essays, and college
interviews have also been researched.
In a 2001 College Board research report by Burton and Ramist, the major
collegiate success studies since 1980 were grouped and analyzed. The results will
be summarized according to the five main cognitive predictors of college suc-
cess. Correlations for each individual study were used to determine a weighted
correlation for each of the five main predictors. The results are detailed in the
3


sections that follow.
2.1 SAT Verbal Scores
Table 2.1 summarizes the results of nine major studies conducted between
1980 and 1998. Each study examined the correlation between SAT Verbal score
and undergraduate cumulative GPA [4],
Predictor Paper Year # of Students Correlation
Young & Barrett 1992 91 .17
Shoemaker 1986 296 .21
Shoemaker 1986 238 .23
SAT Crews 1993 336 .37
Verbal Elliot & Strenta 1988 927 .38
Moffat t 1993 28 .42
Ra 1989 170 .42
Young 1991 1,564 .46
Total Students 4,155 .40
Table 2.1: Predicting Cumulative GPA From SAT Verbal Scores
A weighted average (the average of the reported correlations weighted by
the number of students included in each study) gives an overall correlation of
r = .40 between SAT Verbal scores and first year undergraduate GPA. Notice
that this weighted average is highly dependent upon Youngs 1991 study and the
1988 study by Elliott and Strenta. These were large studies which contributed
4


significantly to the overall weighted average. Also of interest is the particularly
low correlation of r = 17 associated with the Young and Barrett study of 1992.
This low correlation could be attributed to the fact that only 91 students were
analyzed, of which several were outliers [28].
2.2 SAT Math Scores
Table 2.2 summarizes the results of nine studies conducted between 1986
and 1993. In each study, SAT Math scores were analyzed as predictors of un-
dergraduate GPA.
Predictor Paper Year # of Students Correlation
Ra 1989 170 .28
Crews 1993 336 .31
Elliot & Strenta 1988 927 .34
SAT Moffat t 1993 28 .35
Math Shoemaker 1986 238 .35
Young & Barrett 1992 91 .41
Shoemaker 1986 296 .43
Young 1991 1,564 .46
Moffatt, 1993 505 .49
Total Students 4,155 .41
Table 2.2: Predicting Cumulative GPA from SAT Math Scores
5


A weighted average shows that for 4,155 students there is an overall corre-
lation of r = .41 between SAT Math scores and GPA. Comparing these results
with that of SAT Verbal scores, one can conclude that either measure will result
in predictions with similar accuracy.
In recent years the ACT exam has also been extensively used, particularly
in the Western United States. Preliminary analysis shows that ACT Verbal and
Math scores produce similar results and correlations as the SAT [4].
2.3 SAT Verbal and SAT Math Scores
Table 2.3 summarizes results of ten major studies which analyze the com-
bination of SAT Verbal and SAT Math scores as a predictor of college GPA.
In each study, multiple regression analysis was used to determine the combina-
tion of S AT Verbal and Math scores which would produce the best predictor of
student performance at each respective institution.
Notice that the weighted average produces a correlation of r = .36 between
the SAT Verbal and Math combination and college GPA. This correlation is
lower than either SAT Verbal or SAT Math alone. This occurs because the
correlations are based on different samples of students and institutions. In par-
ticular, the 1992 study by Baron and Frank examined 3,816 students and only
found a predictor correlation of r = .20. This is perhaps due to the combination
of Verbal and Math score that they chose to use [2]. If the samples were compa-
rable, the correlation for the combination of SAT Verbal and Math scores would
be higher [4] and prove to be a slightly better predictor for college success than
either SAT Verbal or SAT Math alone.
6


Predictor Paper Year # of Students Correlation
Baron & Frank 1992 3,816 .20
Nettles & Thoeny 1986 4,094 .31
SAT Moffatt 1993 28 .34
Verbal Wolf & Johnson 1995 201 .34
and Ra 1989 170 .39
SAT Tracey & Sedlacek 1985 1,339 .40
Math Willingham 1985 3,442 .41
Elliot & Strenta 1988 927 .43
Ragosta & Braun 1991 2,473 .52
Moffatt 1993 505 .56
Total Students 16,995 .36
Table 2.3: Predicting Cumulative GPA From SAT Verbal & Math Scores
2.4 High School Record
Table 2.4 summarizes results from twelve major studies which compare high
school record to college GPA. Each study used either cumulative high school
GPA, high school class rank, or a combination of the two in order to define high
school record.
From the table it can be seen that high school record has been the most
studied predictor of college success. These twelve studies involved 25,175 partic-
7


Predictor Paper Year # of Students Correlation
Baron & Frank 1992 3,816 .30
Young & Barrett 1992 91 .31
Young 1991 1,564 .35
Wolf & Johnson 1995 201 .40
High Elliot & Strenta 1988 927 .41
School Nettles & Thoeny 1986 4,094 .41
Record Shoemaker 1986 238 .41
Ra 1989 170 .44
Willingham 1985 3,442 .45
Leonard & Jiang 1995 10,000 .46
Shoemaker 1986 296 .48
Crews 1993 336 .59
Total Students 25,175 .42
Table 2.4: Predicting Cumulative GPA from High School Record
8


ipants from different ethnicities, backgrounds, geographical location, and college
institution.
Overall, the weighted average signifies a correlation of r = .42 between
high school record and undergraduate success. Notice this correlation is slightly
higher than either SAT verbal or SAT math scores. Past studies have consis-
tently shown this to be the case.
With the rise of grade inflation, the validity of using high school record
alone has become a concern [23]. High school GPA is highly dependent upon
which region, state, school district, and even school a student came from. As the
meaning of GPAs vary between schools and more students are being categorized
in the A and B range, admissions counselors need to combine GPA with other
criteria in order to make accurate predictions.
2.5 SAT Verbal Score, SAT Math Score, and High School Record
As a result of grade inflation, most post-secondary schools have instituted
a combination of high school record, SAT Verbal, and SAT Math scores for ad-
mission decisions. Table 2.5 summarizes results from five major studies in which
predictors were developed from a combination of all three measures. Multiple
regression was used to determine the best combination of SAT Verbal, SAT
Math, and high school record at each institution.
The weighted average shows a correlation of r -- .52 between college GPA
and the combination of SAT Verbal, SAT Math, and high school record. Also
of interest is that every major study had a correlation higher than any of the
weighted averages for the other predictors. The results from these studies illus-
trate that the combination of SAT results and high school record provide the
9


best prediction of college success.
Predictor Paper Year # of Students Correlation
SAT Verbal, Leonard & Jiang 1995 10,000 .49
Math, and Willingham 1985 3,442 .53
High Ra 1989 170 .58
School Young 1991 1,564 .58
Record Ragosta & Braun 1991 2,473 .62
Total Students 17,649 .52
Table 2.5: Predicting GPA From SAT Verbal, Math, & High School Record
2.6 Extra-Curricular Activities
Another factor that aids the admissions process, is the participation in extra-
curricular activities. High school students are encouraged to join and partici-
pate in several extra-curricular activities to strengthen their college applications.
However, research has consistently shown that participation in these activities
is not a stand-alone predictor of college success.
In Willinghams study from 1985, twenty-five thousand applications were
received for nine different colleges. All of the applications were examined and
admissions counselors made decisions on which students would be successful at
their institution. Several factors went into determining admission, including the
use of high school GPA and extra-curricular activities [27],
10


After admission was granted, 4,814 of tlie students were followed through
four years of college and their accomplishments were recorded. Although extra-
curricular activities played a large role in determining admission, only high
school GPA was determined to be a predictor of college success [27]. Likewise,
in a 1994 study by Wade and Walker, honors students were tracked for two years
of college. Admissions at the small southern university were dependent upon
GPA, class rank, ACT scores, extra-curricular activities, and selection of honors
courses. Again, high school GPA proved to be the only consistent predictor of
success [24],
Finally, the Standard Research Service of the ACT program conducted a
study of 10,758 college freshmen in which multiple criteria for determining col-
lege success were examined. In an attempt to predict English success, each
student was evaluated based upon ACT scores, GPA, and yearbook and high
school newspaper staff participation [14]. Both the ACT English score and the
students high school GPA were determined to be valid predictors of English
success. However, extra-curricular activities were shown to have no real positive
correlation to success in English courses [7].
Although grade point average is the most consistent predictor for success,
Rice and Darke contend that extra-curricular activities are an indirect predictor
of success [20]. In their 2000 study, participants were randomly selected from
Leadership or Academic Scholarship students at a public university. While the
researchers agreed that GPA is an accepted predictor of success, they also con-
tend that, the strongest addition to the traditional predictors of college success
was a measure of persistent and successful extra-curricular accomplishment in
11


high school.
2.7 Problems Associated with Predictors of College Success
Recent concern has been raised that predicting college success based upon
standardized test scores and high school record alone may only provide accurate
predictions for majority groups. Research indicates that traditional methods of
prediction have decreased validity when applied to minority and non-traditional
students and other factors need to be considered when examining their applica-
tions.
Ramist et al. [19], conducted a study on the predictive validity of SAT
scores for Blacks in 11 different mainly White colleges. The researchers found a
correlation of r = .09 between SAT and Black students grades while there was a
correlation of r = .176 for SAT and White students grades. This study is one of
many that have led critics to argue that the assessment provided by standardized
tests is culturally and educationally inappropriate for students from racial and
ethnic minority groups [4].
A difference has also been found for the predictive validity of the SAT be-
tween males and females. Burton and Ramist [4], point out that the SAT his-
torically under-predicts the first year college performance of females and slightly
over-predicts the performance of males. Many factors have been considered as
possible explanations. One contribution to this difference is that males enroll
in more advanced placement math and science classes in high school, leading
to slightly higher SAT scores [12]. Other studies have shown that non-cognitive
predictors have more influence on predictor validity for females than their male
counterparts. Areas such as self-esteem, motivation, and student support were
12


deemed more important additions to the use of standardized tests for female
students and could potentially explain the SATs historical under-prediction of
female success [11].
13


3. Placement Techniques for Lower-Level Undergraduate Courses
In todays post-secondary environment, many students enter without the
proper background necessary for successful completion of mathematics courses
required for their field of study. According to the Colorado Commission of Higher
Education (CCHE), 26.6% of Colorado public high school graduates (7,507 stu-
dents) entering Colorado public higher education in 2002-2003 were assigned
to remediation. Of the students assigned to remediation, 85.3% were assigned
remediation in mathematics [6]. Therefore, proper placement and recommenda-
tion techniques play a crucial role in ensuring that students are placed in the
appropriate course commensurate with their current skill level. Currently about
90% of post-secondary institutions utilize some form of placement and devel-
opmental instruction [22], College administrations seek placement procedures
that are simple, economical and defensible; while faculty seek a procedure that
matches appropriate student skill with course objectives.
Although much research has been conducted on predicting college success,
fewer studies analyze predictors of lower-level undergraduate mathematics suc-
cess. Traditional predictors of math success have been the number of high school
mathematics classes and scores on standardized tests (SAT/ACT exam). In re-
cent years, the use of placement exams such as ACCUPLACER have also been
used to determine appropriate student placement.
The following sections examine the literature on methods for predicting un-
dergraduate mathematics success and developing predictive validity for place-
14


inent techniques.
3.1 High School Coursework
The effect of high school courses on lower-level undergraduate mathematics
success was studied by Roth et. al [21] in 2001. The study found that taking
more higher level math courses in high school is an accurate predictor of scoring
well on aptitude tests commonly required for admission into four-year baccalau-
reate institutions. Even the students with average or poor grades in these high
level courses were better prepared for college level work and less likely to need
non-credit or remedial math. The exposure to higher level coursework helped
students recognize concepts covered on the placement exams and achieve pass-
ing scores. Students who took four years of high school math were also found to
be better prepared for immediate entrance into post-secondary education. The
fourth year of mathematics was deemed as a bridge which links the material
from high school to college [15].
3.2 SAT/ACT Scores
Standardized test scores are often used as one option for determining which
students require remedial mathematics. For instance, a student with an ACT
Math score of 19 or above, or an SAT Math score of 460 or above would fulfill
remediation requirements set by CCHE. If post-secondary institutions use these
standardized test scores for specific course placement, they are usually accompa-
nied by an alternative cut-off score on a placement examination. This is due to
research suggesting that standardized test scores are good predictors of mathe-
matics exam scores, but are not overall predictors of success in a mathematics
15


class. Intangible and non-cognitive factors such as desire, motivation, and peer-
study have been shown to have a large effect on overall performance in the class
and offset much of the correlation due to test-taking skills. These tests are still
considered predictors of college success, but are becoming outdated measures to
predict mathematics success; as a result standardized tests are often replaced
by new placement techniques [25].
3.3 Placement Tests
Many community colleges and four-year institutions are beginning to imple-
ment placement tests as either the sole means of placement, or as an important
part of the placement decision [12]. Several placement tests are available and
common tests used within the U.S. include:
1. Assessment of Skills for Successful Entry and Transfer (ASSET)
2. New Jersey College Basic Skills Placement Test (NJCBSPT)
3. Mathematical Association of America Placement Test Program (MAA)
4. Descriptive Tests of Mathematical Skills (DTMS)
5. ACCUPLACER Test
6. ACT COMPASS Tests
The majority of these placement exams are actually sets of tests which
include one or more components in mathematics [3]. These tests can be admin-
istered in a variety of ways. In an effort to best prepare students for college,
many high schools are now administering placement exams to their graduating
16


students the last month of school. Other placement exams are administered to
students during required orientation periods before the first semester of classes.
Placement exams can also be administered prior to specific courses and deemed
as prerequisite requirements for successful enrollment.
3.4 Placement Exam Validity
According to the American Mathematical Association of Two-Year Colleges
(AMATYC), regardless of how the placement test is administered, the main
goal of testing is to determine the highest level of mathematics appropriate to
students educational goals, at which they have the prerequisite knowledge to
be successful [18]. AMATYC goes on to state: placement tests should provide
a measure of students abilities not only to show mastery of algorithmic skills,
but also to think critically and solve problems.
This leads to one of the main drawbacks of placement testing; the perceived
notion that capable students may be denied access to higher-level mathematics
instruction at which they could have been successful. Independent studies by
Armstrong [1], Jenkins [9], and Isbell [8] have all determined that a large portion
of students actually succeed in mathematics classes even though their test scores
labeled them as unprepared. This notion has driven research on placement
validity and the development of placement techniques that avoid unfairly holding
back potentially successful students.
Two recent studies have addressed the issue of developing better predictive
placement techniques. Judith Marwick [12] recently completed a study at a Mid-
western University which compared three alternative mathematics placement
methods with the rigid placement by exam score alone. All students who took
17


the ACCUPLACER exam during the summer of 2001 were randomly assigned to
one of four placement methods: ACCUPLACER exam score alone, high school
preparation alone, a balance of the two measures, or student choice constrained
by the two measures [12]. The study found that overall the students performed
equally well in their mathematics course regardless of which method was used for
placement. When comparing placement test scores with high school record, it
was determined that placement of the students would have been approximately
the same regardless of which technique was used. Final recommendations show
that placement by ACCUPLACER score alone denied many students access to
the appropriate course in which they could have been successful. Marwick con-
cluded that placement techniques should involve a combination of high school
record and placement test score with students placed in the higher-level course
recommendation.
Another ongoing study of placement technique validity has been conducted
at Indiana University Purdue University Indianapolis (IUPUI). At IUPUI a
computerized mathematics placement exam is used which consists of 40 objective
items ranging from arithmetic operations to introductory calculus. A students
raw score is determined by the number of items answered correctly. Based on
the score students are given recommendations for placement in one of three
different remedial courses, or allowed to take the corresponding college course
of their choice dependent on their academic major.
As students were allowed to choose whether to follow the placement rec-
ommendation, they naturally formed two separate groups for comparison, com-
pliant and noil-compliant. Predictive validity of the mathematics placement
18


exam was then studied by analyzing the mathematics results of the two differ-
ent groups.
For the compliant students who followed the placement test course rec-
ommendation, the correlation between placement score and mathematics exam
score was .23. For students who did not follow placement recommendations (i.e.
chose to take a more difficult course), the correlation between placement score
and mathematics exam scores was .25. Also of interest is the correlation between
placement test score and final course grade. For compliant students the average
correlation coefficient was r = .19 and for non-compliant students r .17.
As a result of this study, IUPUI concluded that its current placement tech-
niques needed revision. Students who were not following placement recom-
mendations were succeeding in higher level coursework and there was no sta-
tistical difference in the correlation between compliance and performance and
non-compliance and performance. Following the study, the mathematics depart-
ment switched to a Computerized Adaptive Testing (CAT) procedure much like
the ACCUPLACER Exam. Preliminary results show an improvement in cor-
relation between placement score and mathematics success; however, the non-
compliant students still performed as well as the compliant students indicating
that placement test scores still prevented capable students from enrolling in
suitable coursework.
19


4. The ACCUPLACER Exam
With the help of committees of college faculty, The College Board developed
the ACCUPLACER Computerized Placement Tests. These are adaptive tests
that choose the difficulty of questions based on responses to previous items.
There are ten different tests available which measure reading, writing, English,
and mathematics ability. Within the mathematics category, three different tests
are available to measure varying skill levels: the Arithmetic Test, the Elementary
Algebra Test, and the College-Level Mathematics Test. Each of these tests is
multiple choice with no enforced time limit.
The College-Level Mathematics Test assesses proficiency from intermediate
algebra through pre-calculus. Twenty questions are asked and the topics covered
include algebraic operations, solutions of equations and inequalities, coordinate
geometry, applications and other algebra topics, and functions and trigonometry.
The Elementary Algebra Test is twelve questions long and covers topics includ-
ing operations with integers and rational numbers, operations with algebraic
expressions, and solving equations, inequalities, and word problems. Finally,
the Arithmetic Test consists of sixteen questions covering operations with whole
numbers and fractions, operations with decimals and percents, and applications
and problem solving.
ACCUPLACER utilizes Item Response Theory (1RT) to create a more ef-
ficient, and individualized assessment of student mathematical skills. fRT grew
out of the need to individualize assessment and adapt questions based on the
20


skill level the student exhibits. The IRT technique that ACCUPLACER uses
first creates a latent ability, 8, that can be assessed given items of known dif-
ficulty, b. IRT also includes parameters to account for item discrimination, a,
and guessing, c [26]. These are combined into a 3-parameter model:
PW = C+
P(8) probability of respondent with ability 6 to answer a question
of difficulty b correctly.
a = item discrimination. Proportional to the slope: 0.425a(l c).
b = item difficulty
c = guessing parameter
8 = respondents ability
This logistic model is referred to as an Item Characteristic Curve. Ability
discrimination is theoretically maximized when P(8) is close to the inflection
point for a bank of items. If there is no guessing, this point would be around
50%. In order to ensure maximum discrimination for each item, the set of
questions should differ for those of varying ability levels. In the Computerized
Adaptive Test format, answering a question correctly will raise the difficulty of
the next question and an incorrect response lowers the difficulty of the question
that follows. This process keeps P(8) near the curves inflection point until 8
can be estimated within an acceptable error range [26]. This procedure requires
21


fewer questions than non-adaptive tests and can reduce both time and facilities
needed to administer the exams.
There are also drawbacks of the ACCUPLACER, particularly in the Elemen-
tary Algebra Exam. The adaptive nature of the test results in large penalties for
incorrect answers chosen early in the exam. If a student answers initial questions
incorrectly, future questions are worth fewer points. This can lead to difficulty
building up to questions of higher point value. Also, the difference between an
ACCUPLACER score of 72-85 and above 85 is due to one additional correct
answer. Therefore, one correct guess can effect a students chances of achieving
a cut-off score.
22


i
i
!
i
j
i
5. Current Placement Techniques for Colorado Colleges
Colorado law (CRS 23-1-113) requires all incoming freshmen and transfer-
students to be assessed for their basic skills in reading, writing, and mathe-
matics. In order for a student to show proficiency in mathematics they must
meet one of four different requirements set by CCHE. A student who achieves a
score of 19 or higher on the ACT assessment mathematics exam, a score of 460
or higher on the SAT Math section, or a score of 85 or higher on the ACCU-
PLACER elementary algebra exam does not require remediation. In addition,
if a student has successfully completed a college-level mathematics course or a
remedial mathematics course, no additional assessment is necessary.
Following this May 2000 legislation, most schools throughout Colorado have
implemented placement procedures for lower level mathematics courses based
loosely upon remediation assessment guidelines set out by CCHE. The following
gives a brief summary of placement techniques used at various colleges and
universities throughout Colorado.
Adams State College
Students entering Adams State College with an ACT mathematics score of less
than 19 (470 SAT mathematics score) are required to take the ACCUPLACER
Elementary Algebra Test. Based on the results of that exam students are advised
as follows:
1. Score of 85 or above:
Student may enroll in Finite Mathematics or College Algebra.
23


2. Score of 55-84:
Student may enroll in Intermediate Algebra.
3. Score of 40-54:
Student may enroll in Basic Algebra Skills Class.
4. Score of 39 or lower:
Student may enroll in Arithmetic Skills Course.
5. ACT Score of 26 or higher:
Student may enroll in Calculus I as their first college math course. Oth-
erwise the student must first take College Algebra before enrolling in Cal-
culus.
Colorado Christian University
Students entering Colorado Christian University with appropriate ACT or SAT
scores may enroll in the course of their choice. Students who have not taken
either the ACT or SAT exam must take the math placement exam COMPASS.
Based on the results of this exam the student will be advised to take remedial
math or proceed with the appropriate college level course.
Colorado School of Mines (CSM)
No exams are required for students enrolled at CSM. The entry-level mathemat-
ics class for all students at CSM is Calculus I.
Colorado State University
Students who have received a score of three or above on the Advanced Placement
24


Calculus Exam (either AB or BC), or have transfer credit in a math course at
the level of college algebra or above are not required to take any placement
examinations. All other students must take the Mathematics Placement Exam
(MPE). This exam consists of 50 multiple choice-multiple response questions
and covers topics in College Algebra, Numerical and Analytical Trigonometry,
and Logarithms and Exponential Functions. In addition, students may take the
Entry Level Mathematics (ELM) Exam. This exam covers intermediate algebra
topics and helps guide student choices for the entry-level mathematics courses
offered at Colorado State University.
Colorado State University-Pueblo
All students are placed according to their ACT or SAT math score. If students
would like to challenge the placement, they may take the ACCUPLACER Exam
and attempt to receive a qualifying score.
Fort Lewis College
All students are required to take the ACCUPLACER placement exam. Each
student begins with the elementary algebra exam. If a student correctly an-
swers a certain proportion of the questions on this exam, they will then take
the college-level mathematics exam. If a student does not successfully answer
enough questions on the elementary algebra exam, the student is then directed
to take the arithmetic exam. Based on the results of these tests, the student is
advised on which course to enroll.
25


Metropolitan State College of Denver
Students with an ACT Math score of 19 or above or an SAT Math score of 460
or above are exempt from placement exams. All other students are required to
take the ACCUPLACER Exam and must take the course corresponding to their
score. The placements are as follows:
1. Score of 0-56 on the Arithmetic Exam:
Students may take Fundamentals of Mathematics.
2. Score of 57-120 on the Arithmetic Exam:
Students may take Pre-Algebra.
3. Score of 45-60 on the Elementary Algebra Exam:
Students may take Introductory Algebra (a Community College of Denver
(CCD) course).
4. Score of 61-84 on the Elementary Algebra Exam:
Students may take Survey of Algebra (a CCD course).
5. Score of 85-99 on the Elementary Algebra Exam:
Student may take any of the following: Integrated Mathematics I, Math-
ematical Modes of Thought, Introduction to Statistics, College Algebra
with peer study, or Finite Math with peer study.
6. Score of 100-120 on the Elementary Algebra Exam:
Student may take either College Algebra or Finite Math.
7. Score of 65-79 on the College Level Math Exam:
Student may take Pre-Calculus.
26


8. Score of 80-120 on the College Level Math Exam:
Student may take Calculus I.
University of Colorado at Colorado Springs (UCCS)
Two math placement exams are offered at UCCS. The first is the Algebra Di-
agnostic which measures intermediate algebra skills. Students scoring above 20
on this exam are also invited to take the Calculus Readiness Test. The scores
are used for placement as follows:
1. Score of 0-8 on the Algebra Diagnostic:
Student is required to take Fundamentals of College Algebra.
2. Score of 9-16 on the Algebra Diagnostic:
Student may take College Algebra
3. Score of 17-19 on the Algebra Diagnostic:
Student may take Elementary Functions of Calculus, Topics in Linear
Algebra, or Calculus for Business and Economics.
4. Score of 20-32 on the Algebra Diagnostic:
Student may take Elementary Functions of Calculus, Topics in Linear
Algebra, or Calculus for Business and Economics. In addition, the student
may take the Calculus Readiness Exam.
5. Score of 0-9 on the Calculus Readiness Exam:
Student may take Elementary Functions of Calculus as preparation for
Calculus I.
27


6. Score of 10-12 on the Calculus Readiness Exam:
Student may take Calculus I along with the corresponding pre-calculus
review offered in the math learning center.
7. Score of 13-25 on the Calculus Readiness Exam:
Student may take Calculus I.
University of Northern Colorado (UNC)
Current placement methods at UNC are based upon orientation advising. At
scheduled orientations, graduate students hold 5-10 minute advising sessions
with incoming students. The students are asked a series of questions regarding
their ACT/SAT math score, the mathematics courses which they took in high
school, the length of time since their last mathematics course, and their intended
major. Based on the information provided, the graduate students make course
recommendations, however, students may register for the class of their choice.
Western State College of Colorado (WSCC)
WSCC uses a combination of ACT scores, SAT scores, and a placement exam
to assess incoming students. Students are placed in the courses according to the
following criteria:
1. Basic Algebra Review:
Students may take this class regardless of their performance on the three
placement techniques.
2. Mathematics for the Liberal Arts, Mathematics for the Managerial Sci-
ences, Algebraic Functions, or Theory of Arithmetic and Geometry I:
28


In order to enroll in any of the above courses a student must attain a score
of 19 or above on the ACT Math exam, score a 460 or above on the SAT
Math exam, pass the ACCUPLACER elementary algebra test with a score
of 85 or above, or pass the Basic Algebra Review class.
3. Transcendental Functions:
A student must pass the ACCUPLACER college-level mathematics test
with a score of 75 or above.
4. Probability and Statistics:
A student must pass the ACCUPLACER college-level mathematics test
with a score of 85 or above.
5. Calculus I:
A student must pass the ACCUPLACER college-level mathematics test
with a score of 95 or above.
The above sections detail placement procedures for several Colorado Col-
leges. It is worth noting that some colleges do not have placement requirements
for all courses. This leads to an alternate way many students choose to fulfill
their remediation requirement; enroll in a lower-level college course that does not
mandate placement procedures. At UCDHSC there are currently no placement
procedures for MA 1010-Mathematics for the Liberal Arts Student, enabling any
student to enroll. A student who is in need of meeting remediation requirements
during the first 30 credit hours, may take MA 1010 and fulfill the requirement.
This option enables the student to gain college credit while fulfilling remedia-
tion.
29


6. UCDHSC Placement Techniques for Spring 2004
In Spring 2004, UCDHSC began mathematics placement procedures for all
sections of MA 1110-College Algebra and MA 1070-Algebra for Social Sciences
and Business. Mandatory placements were made based upon the students per-
formance on the ACCUPLACER Elementary Algebra Exam. For the Spring
semester a passing score of 72 was required in order to continue enrollment in
each class.
All students were given two opportunities to pass the ACCUPLACER Exam,
with a one-day grace period required between sittings. Students who failed the
exam twice could petition the Associate Chair of the Mathematics Department
for a third opportunity at attaining a passing score. Elementary Algebra review
sessions were available in the Math Education Resource Center (MERC) and
utilized by many students. If a student did not obtain a passing score on the
exam by the second week of class they were administratively dropped from the
course and advised to take proper remediation to prepare them for the course.
Detailed data were recorded for all Spring 2004 College Algebra students.
A database was created with the following information for each individual stu-
dent: ACCUPLACER score, ACT score, SAT score, College Algebra instructor,
common final exam score, and final course grade. Statistical analysis on these
data had two goals:
1. Determine if statistically significant correlations exist between ACCU-
PLACER score, ACT score, SAT score, score on the common final exam,
30


and final course grade.
2. Analyze the effect of raising the cutoff score in MA 1110 from 72 to 85.
6.1 Data Description
A total of 151 students initially enrolled in College Algebra for Spring
Semester 2004. Of the 151 enrolled, twelve students never took the ACCU-
PLACER and were administratively dropped. In addition ten students were
unable to attain a score of 72 or above and were also administratively dropped.
A total of 19 students achieved a passing score on the ACCUPLACER Exam,
but dropped the class. Thus, the study had a sample size of 110 students who
passed the ACCUPLACER exam and also finished the course.
By limitations of the data collection, ACCUPLACER scores of four stu-
dents who took the course were unattainable. Also, as the majority of students
in Colorado take the ACT instead of the SAT exam, the data includes ACT
information for 84 students who finished the class and SAT information for only
32. Lastly, final exam scores were only available for 101 students; this indicates
that nine students either did not take the common final exam or their score was
unattainable in the data collection.
6.2 Correlation Among Variables
For purposes of attaining correlations, all letter grades were converted to
the UCDHSC numerical equivalent shown in Table 6.1:
31


> II o A-=3.7
B+-3.3 B=3.0 B-=2.7
C+=2.3 C=2.0 0 1 II I1
D+=1.3 D=1.0 o II i Q
o o II eL
Table 6.1: Numerical Equivalent of Letter Grades
SPSS Software was used to calculate Pearson bivariate correlations among
the five variables and analyze the statistical significance of each. The correlation
matrix in Table 6.2 displays the results of the analysis.
From the correlation analysis, it is evident that the highest correlation oc-
curred between ACT Math score and SAT Math score. This is not a surprising
statistic for the 32 students who took both exams; research has consistently
shown the two standardized tests to be highly correlated. As expected grade on
the final exam and final grade in the class were also highly correlated.
One interesting aspect of this correlation analysis is that ACT Math score
was determined to be a statistically significant predictor of ACCUPLACER score
at the 0.01 level with a correlation of r = .473. However, the correlation between
SAT Math score and ACCUPLACER score was not statistically significant with
r = .293. This could be related to the smaller sample size for SAT scores which
potentially caused error in the analysis.
Also, both ACT math (at the 0.01 level) and SAT math (at the 0.05 level)
were considered statistically significant predictors of final exam score, but not
32


ACT MATH SAT MATH ACCUP Grade Final
ACT MATH Pearson Corr. 1 .742** .473** .191 .328**
Sig. (2-tailed) .000 .000 .082 .004
N 106 32 99 84 77
SAT MATH Pearson Corr. 1 .293 .159 .404*
Sig. (2-tailed) .063 .392 .030
N 42 41 31 29
ACCUP Pearson Corr. 1 .390** .418**
Sig. (2-tailed) .000 .000
N 135 106 98
Grade Pearson Corr. 1 .691**
Sig. (2-tailed) .000
N 110 101
Final Exam Pearson Corr. 1
Sig. (2-tailed)
N 101
** Indicates the Correlation is significant at the 0.01 level (2-tailed)
* Indicates the Correlation is significant at the 0.05 level (2-tailed)
Table 6.2: Correlations for College Algebra Spring 2004
33


final grade in the class. Finally, there is a correlation of r = .418 and r = .390
between ACCUPLACER and final exam and final grade respectively. Both
correlations are significant at the 0.01 level.
6.3 Analysis of the Results
Several conclusions can be drawn from the correlation analysis. The re-
sults support current research which attests that standardized test scores alone
should not be used to predict student success in mathematics courses. Both
standardized test scores were statistically significant predictors of final exam
score, indicating a relationship between test-taking abilities. However, as the
research suggests, standardized tests scores are not a significant predictor of
final grade.
Also of interest, ACCUPLACER is a statistically significant predictor of
both final exam and final grade. Therefore, course data validate the selection
of ACCUPLACER as a placement technique for College Algebra students over
the use of standardized tests.
6.4 Limitations of the Data
Aspects of the data set limit the strength of some results. Correlations
involving SAT Math data should be interpreted with caution. Only 32 of the
110 students who took this course had SAT Math scores available, therefore
the data could be biased. In addition, correlations involving ACT Math scores
may also reflect bias. Although ACT information was available for 84 of the
110 students, the missing information may not be due to random effects. Often
students who do not report ACT or SAT information are low-achieving students
who did not plan on attending college while in high school. This could result in
34


biased results where the excluded cases are heavily weighted with low-achieving
students.
6.5 Analyzing College Algebra Cut-Off Scores
The next question of interest is whether raising the required ACCUPLACER
score to 85 would significantly effect students in College Algebra. Figure 6.1
provides a graphical representation of MA 1110 Spring 2004 course data. The
scatter-plot displays ACCUPLACER score as the independent variable and final
grade as the dependent variable.
4.00
3.00
a>
TJ
as 2.00
U.
O
1.00
0.00
o O 00 o oo o o o oo o
o 00 o ooo
o o o o o
cooooo o oooooo o ooo
o o ooo
00
o ooo ocooa >0000 00 00 o ooo o
o o o o
oo
O O OO 003 o o o
I -------1------I------------1----------1 I I
60.00 70.00 80.00 90.00 100.00 110.00 120.00
ACCUPLACER
Figure 6.1: Accuplacer Versus Final Grade Spring 2004
From the graph, first note that one student took the ACCUPLACER, scored
a 64, and was still able to finish the class. Following up on this student, the
35


i
i
I
I
1
I
t
f
I
I
t
I
i
I
j
I
1
i
Numerical Grade
Figure 6.2: Distribution of Course Grades for Students Scoring Between 72-85
mathematics department associate chair confirmed that approximately one stu-
dent slips through the cracks every other semester and is allowed to take the
class without a passing ACCUPLACER score.
A total of 39 students qualified for College Algebra with an ACCUPLACER
score between 72-85. Figure 6.2 shows the distribution of grades for these stu-
dents.
While eleven students in this range received a D or F and did not pass the
course, 28 students received a passing grade of C or above. These results can be
interpreted in two ways. By examination, 72% of students scoring in the 72-85
36
I
j
|


range passed the course. This seems to indicate that the cut-off score should not
be raised because capable students would be prevented from taking coursework
in which they could be successful. However, many of these students achieved
a score in that range on their first ACCUPLACER attempt. With attendance
at an Elementary Algebra review session and pointed study, these scores could
conceivably be raised to 85 or above. Also, out of the twelve students who
received a failing grade of F in the course, nine fell in the 72-85 range.
In order to make accurate conclusions, students should also be tracked on
whether they achieved their passing ACCUPLACER score on the first, second,
or third attempt. If the majority of the students who received scores in the 72-85
range achieved their score on the first attempt, then the evidence would support
raising the cut-off score to 85. After teaching Elementary Algebra review sessions
and working in the MERC lab, it became apparent that students scoring above
a 72 on the first attempt would most likely raise their score to the appropriate
level on the next attempt. Recall that improving ones score from a 72-85 to
above an 85 requires the student to answer one more question correctly on the
twelve question ACCUPLACER exam.
Another factor influencing the cut-off score for College Algebra is the state
requirement on remediation. As mentioned earlier, the Colorado Commission
of Higher Education requires an elementary algebra score of 85 or above as one
criteria for exemption from remediation. Since College Algebra is a non-remedial
course, UCDHSC does not want to set its placement score lower than CCHEs
remediation requirement.
37


6.6 MA 1110 Fall 2004 Statistics
Beginning in Fall 2004, the required ACCUPLACER score for College Al-
gebra was raised to 85 or above. Limited data from this class providing each
students ACCUPLACER score and final course grade was available. The cor-
relation matrix in Table 6.3 provides predictor information for ACCUPLACER
versus final grade.
ACCUPLACER Final Grade
ACCUPLACER Pearson Corr. 1 .181*
Sig. (2-tailed) .017
N 200 172
Final Grade Pearson Correlation 1
Sig. (2-tailed)
N 173
* Indicates the Correlation is significant at the 0.05 level (2-tailed)
Table 6.3: Correlations for College Algebra Fall 2004
It is interesting to note that ACCUPLACER is a significant predictor of
College Algebra course grades at the 0.05 level. However, the correlation coeffi-
cient of r = .181 is lower than any of the ACCUPLACER correlation coefficients
for courses with a cut-off score of 72.
Figure 6.3 illustrates a scatter-plot for the data with ACCUPLACER score
as the independent variable and final course grade as the dependent variable.
38


Grade
The graphical display illustrates no apparent patterns for the data collected with
a cut-off score of 85 or above.
4.00-
3.00-
2.00-
1.00-
o.oo-
o o ooo o oo oo o oo o oo
o ooo o
o o o o o o oo oo
oooooooooo ooooooooo o oo o

oo oooo o o
oooooo oooooooooooooooo o o oo
o o
d>o ooooo oo o o o
oo o oo ooooooo
r
120.00
I
80.00
i n
90.00 100.00
ACCUPLACER
110.00
Figure 6.3: Accuplacer Versus Final Grade Fall 2004
39


7. Current ACCUPLACER Results
Currently UCDHSC requires passing ACCUPLACER scores for enrollment
in three courses: College Algebra-MA 1110, Algebra for Social Sciences and
Business-MA 1070, and Calculus I-MA 1401. To study the effects of the ACCU-
PLACER exam on these three courses, detailed data were provided for sections
of each class from the Spring 2001 semester to Fall 2005. Each data set included
distribution of individual instructor grades and number of withdrawals from
each class. ACCUPLACER scores, standardized test scores, and administrative
drops were not recorded in the data.
The primary goal of this study was to examine two hypotheses:
1. With the implementation of ACCUPLACER testing, there is a significant
decrease in withdrawal rates for MA 1401, MA 1110, and MA 1070.
2. With the implementation of ACCUPLACER testing,there is a statistically
significant increase in the mean student grades for MA 1401, MA 1110,
and MA 1070.
The secondary goal of the study was to examine the hypothesis:
1. With the implementation of ACCUPLACER testing, there is an increase
in passing rates for MA 1401, MA 1110, and MA 1070.
40


7.1 Analysis of MA 1401
Beginning in Fall 2004, students in Calculus I were required to achieve a
score of 80 or above on the College Level Math portion of the ACCUPLACER
exam. The data set provides detailed information for 44 sections of Calculus
I before the ACCUPLACER requirements and 6 after. This corresponds to a
total of 914 students before ACCUPLACER and 89 students after.
To test the hypothesis that mean withdrawal rates have decreased since the
implementation of the ACCUPLACER, mean withdrawal rates were calculated
for each group in the sample. Withdrawal rate is defined as the number of
students in each section who withdraw from the course after Census Day with
appropriate approval. Mean withdrawal rate for each group is calculated as the
total number of withdrawals for each group(i.e. Before or After) divided by the
number of sections.
An Independent-Samples T-Test with unequal variances was used to de-
termine if there was a significant difference in mean withdrawal rates between
classes before the ACCUPLACER and classes after. The test was set up as
follows:
Hi = Mean withdrawal rates for MA 1401 classes before ACCUPLACER
H2 = Mean withdrawal rates for MA 1401 classes after ACCUPLACER
Ho = Hi H-2 = 0
Ha = Hi -H2>0
41


Table 7.1 displays the results from the test. Withdrawal rates decreased
from a mean of 3.000 student withdrawals per section before ACCUPLACER,
to a mean of 1.1667 after. We conclude that at the 0.05 significance level, we
reject the null hypothesis and find convincing evidence that withdrawal rates
for MA 1401 are lower after the use of ACCUPLACER testing.
t df Sig. (2-tailed) Mean Diff. Std. Error Diff.
2.452 8.441 .038 1.83333 .74757
Table 7.1: Independent-Samples T-Test for MA 1401 Withdrawals
To test the hypothesis that mean student grades have increased since the
implementation of the ACCUPLACER, the following Independent-Samples T-
Test was used.
Pi = Mean of student grades for those who took MA 1401 before ACCUPLACER
11-2 Mean of students grades for those who took MA 1401 after ACCUPLACER
Ho = pi p2 = 0
Ha = hi P'2 < 0
Table 7.2 illustrates the results for the test with unequal variances. The
mean student grades increased from a mean of 2.5647 before ACCUPLACER,
to a mean of 2.8180 after. At the 0.05 significance level, we reject the null
hypothesis and find convincing evidence that mean student grades for MA 1401
are higher after the ACCUPLACER.
42


t df Sig. (2-tailed) Mean Diff. Std. Error Diff.
-2.055 111.396 .042 -.25332 .12329
Table 7.2: Independent-Samples T-Test for MA 1401 Mean Student Grades
To further analyze the effects of ACCUPLACER, average passing rates were
calculated. Passing rates are defined as the percentage of students who received
an A, B, or C in the class relative to all students who finished the class. There-
fore, withdrawals are not considered in the passing rate calculations. Table 7.3
details the results of this analysis.
MA 1401 Before After
Passing Rates 81.27% 88.76%
Table 7.3: Calculus Comparisons
From the comparison in Table 7.3 passing rates increased for MA 1401 with
the use of the ACCUPLACER exam. Variation in individual class sizes pre-
vented further statistical analysis on passing rates for MA 1401.
Limitations of the data
The results for withdrawal rates should be interpreted with caution. Statis-
tical analysis did not provide sufficient proof that the data set is normal. With
only nine degrees of freedom in the data, our results may be slightly biased.
The data for mean student grades is a collection of discrete points (based
on the numerical equivalence of each letter grade) and is not normal. However,
43


because of the large sample sizes for both before and after ACCUPLACER
groups, the Independent-Samples T-Test is an accurate measure.
One further limitation of the data is that the After ACCUPLACER group
does not include any summer course data. This could bias the results.
7.2 Analysis of MA 1110
Beginning in Spring 2004, students in College Algebra were required to
achieve a passing score on the Elementary Algebra portion of the ACCU-
PLACER exam. A score of 72 was considered passing for Spring and Summer
2004, and a score of 85 was considered passing for Fall 2004. The data set
provides detailed information for 35 sections of College Algebra before the AC-
CUPLACER requirements and 15 sections after. This corresponds to a total of
858 students before ACCUPLACER and 329 students after.
We will again test the hypothesis that withdrawal rates have decreased since
the implementation of the ACCUPLACER. The Independent-Samples T-Test
with unequal variances was set up as follows:
Hi = Mean withdrawal rates for MA 1110 classes before ACCUPLACER
H2 = Mean withdrawal rates for MA 1110 classes after ACCUPLACER
H0 = fj>i /x2 = 0
Ha = H\ ~ P2 > 0
Table 7.4 displays the test results. The mean withdrawal rates decreased
from a mean of 5.600 student withdrawals per section before ACCUPLACER, to
a mean of 3.067 after. At the 0.01 significance level we reject the null hypothesis
44


and find convincing evidence that withdrawal rates for MA 1110 are lower after
the use of ACCUPLACER testing.
t df Sig. (2-tailed) Mean Diff. Std. Error Diff.
3.158 30.980 .004 2.5333 .80214
Table 7.4: Independent-Samples T-Test for MA 1110 Withdrawals
The following Independent-Samples T-Test was used to test the hypothesis
that mean student grades have increased for MA 1110 after the implementation
of ACCUPLACER.
Hi = Mean of student grades for those who took MA 1110 before ACCUPLACER
H2 = Mean of students grades for those who took MA 1110 after ACCUPLACER
Ho = Hi H2 = 0
Ha = Hi - < 0
Table 7.5 shows the results for the test with unequal variances assumed. The
mean student grades decreased from 2.4153 before ACCUPLACER to 2.3261
after.
t df Sig. (2-tailed) Mean Diff. Std. Error Diff.
1.103 640.130 .270 .08913 .08079
Table 7.5: Independent-Samples T-Test for MA 1110 Mean Student Grades
45


We can not reject the null hypothesis and there appears to be no statistical
difference between mean student grades for the two groups of MA 1110 students.
Passing rates for MA 1110 were also determined. Table 7.6 details the
results.
MA 1110 Before After
Passing Rates 77.51% 79.64%
Table 7.6: College Algebra Comparisons
Prom the comparison in Table 7.6, passing rates slightly increased for MA
1110 after the implementation of the ACCUPLACER exam. However, the
change is not significant. Limitations on the MA 1110 results are similar to
those for MA 1401.
7.3 Analysis of MA 1070
Beginning in Spring 2004, students in Algebra for Social Sciences and Busi-
ness (Business Algebra) were required to achieve a passing score of 72 on the
Elementary Algebra portion of the ACCUPLACER exam. The data set provides
detailed information for 40 sections of Business Algebra before the implemen-
tation of ACCUPLACER testing and 14 sections after. This corresponds to a
total of 1103 students before ACCUPLACER and 382 after.
Analysis of withdrawal rates will again utilize an Independent Samples T-
Test with unequal variances assumed and is as follows:
46


/i] = Mean withdrawal rates for MA 1070 classes before ACCUPLACER
= Mean withdrawal rates for MA 1070 classes after ACCUPLACER
H0 = Hi H2 = 0
Ha = Hi R2 > 0
Table 7.7 illustrates the test results. Mean withdrawal rates decreased from
4.500 before ACCUPLACER to 2.4286 after. We conclude that at the 0.01 sig-
nificance level we reject the null hypothesis and find convincing evidence that
withdrawal rates for MA 1070 are lower after the implementation of ACCU-
PLACER placements.
t df Sig. (2-tailed) Mean Diff. Std. Error Diff.
2.761 34.343 .009 2.07143 .75032
Table 7.7: Independent-Samples T-Test for MA 1070 Withdrawals
The following Independent-Samples T-Test was used to test the hypothesis
that mean student grades have increased since the implementation of ACCU-
PLACER.
Hi = Mean of student grades for those who took MA 1070 before ACCUPLACER
Hi Mean of students grades for those who took MA 1070 after ACCUPLACER
Ho = H i ~ Hi 0
Ha = Hi H2 < 0
47


Table 7.8 displays the results for the test with unequal variances assumed.
The mean student grades decreased from 2.5157 before ACCUPLACER to
2.4945 after.
t df Sig. (2-tailed) Mean Diff. Std. Error Diff.
.277 665.219 .782 .02118 .07635
Table 7.8: Independent-Samples T-Test for MA 1070 Mean Student Grades
We can not reject the null hypothesis and there appears to be no statistical
difference between mean student grades for the two groups of MA 1070 students.
Passing rates for MA 1070 are detailed in Table 7.9.
MA 1070 Before After
Average Grade Passing Rates 2.389 79.51% 2.38 78.27%
Table 7.9: Business Algebra Comparisons
It can be seen from Table 7.9 that passing rates slightly decreased for MA
1070 after the implementation of the ACCUPLACER exam, therefore showing
no statistical difference. Again, limitations on the MA 1070 results are similar
to those for MA 1401 and MA 1110.
48


8. Discussion
Hypothesis 1: Withdrawal Rates Decreased with the use of ACCU-
PLACER Testing
Prom statistical analysis it has been shown that the use of the ACCU-
PLACER placement exam decreased withdrawal rates in all three classes. These
results may be slightly biased because of the nature of the data sets. As dis-
cussed, evidence does not suggest the data is normally distributed and the small
sample size for classes sampled after the ACCUPLACER may bias the results.
In addition, class data were available for only three semesters of MA 1110 and
MA 1070 and only one semester of MA 1401. In order to further verify these
results, future analysis should be conducted after more data are collected for all
three classes.
Hypothesis 2: Mean Student Grades increased with the use of AC-
CUPLACER Testing
There is evidence that the mean student grades for MA 1401 have increased
with the implementation of the ACCUPLACER exam. There are also limita-
tions to this conclusion. The After ACCUPLACER group data was taken from
students in Fall 2004. Future testing should involve students from Fall, Spring,
and Summer to provide a more accurate representation of the sample. This
conclusion is also limited by instructor variability and the hypothesis should be
re-examined when future data become available.
49


Statistical analysis did not support the hypothesis for MA 1110 or MA 1070.
Mean student grades remained approximately the same for both groups. This
result is not surprising. Instructors have indicated that their grade distributions
would not be altered, even with better prepared students.
Hypothesis 3: Passing Rates have increased since the use of ACCU-
PLACER Testing
There is evidence that passing rates have increased in MA 1401 while no
change has been seen in MA 1110 and MA 1070. The results for MA 1110 and
MA 1070 support the research on non-cognitive factors which play a role in
student achievement. Although the students who enrolled in 1070 are theoreti-
cally better prepared for the class (because they passed ACCUPLACER), other
factors such as determination, hard-work, and motivation play a role in student
achievement and final course grade. Future analysis should be conducted on
MA 1070 after the passing ACCUPLACER score has been raised to 85.
50


9. Conclusions
With the changing face of post-secondary education, placement methods
play a crucial role in mathematics department operations. At the University
of Colorado at Denver and Health Sciences Center, placement techniques are
particularly important because of the high number of noil-traditional students.
Just as valid placement techniques are a necessity, evaluation of the placement
process must be on-going. With the completion of three semesters of courses
utilizing mandatory placement procedures, initial assessments must be made.
After analyzing course data for MA 1401, MA 1110, and MA 1070 it is
clear that the use of ACCUPLACER has reduced the number of withdrawals
in these courses. Also, the passing rate for MA 1401 has increased while those
for MA 1110 and MA 1070 have remained relatively constant. Analysis also
justifies the decision to raise the cut-off score in MA 1110 to 85 or above and the
recommendation is to also raise the score in MA 1070 to 85 or above. Statistical
analysis reveals that the mean of student grades increased for MA 1401 and did
not change for 1110 or 1070.
Instructor reaction also provides further analysis of the ACCUPLACER
placement techniques. Interviewing those who taught both before and after
ACCUPLACER reveals positive feedback. Instructors report that students are
better prepared for their classes and the main impact of ACCUPLACER has
been to reduce the amount of time spent on review material. In addition, in-
structors state that, course material runs smoothly with less explanation needed
51


for past troublesome areas such as factoring and algebraic manipulation.
Concern has been raised that capable students may be excluded from taking
a course in which they could be successful. Analysis of Spring 2004 College
Algebra data showed that the majority of students in the 72-85 range had the
ability to pass the course. However, with 2-3 opportunities to pass the exam,
most of these students could feasibly raise their score to the appropriate level.
Overall, initial assessment of ACCUPLACER testing results in a favorable
recommendation to continue placement procedures. This assessment should be
on-going and the study should be extended with the acquisition of a few more
semesters of data.
9.1 Future Analysis
Future analysis for this study would involve working with Metropolitan State
College of Denver (MSCD) to analyze placement techniques. MSCD currently
has one of the most extensive placement procedures in the United States and
has been using placement techniques longer than UCDHSC. Access to their data
could provide useful analysis with less bias due to small sample sizes.
In addition, detailed data should be collected for all three courses requir-
ing ACCUPLACER. For every section, each individuals ACCUPLACER score,
ACT Score, Final Exam Score, and Final Course Grade should be recorded.
This information could provide insight into ACCUPLACER as a predictor of
success in each of the three courses.
Finally, future work would also involve developing a better measure for suc-
cess than final course grade. Possible alternatives are course outcomes assess-
ment surveys and self-questionnaires administered at the end of the semester
52


which include self descriptions of confidence in course material. These alterna-
tive measures could be useful in assessing self-described student success in each
course, rather than instructor determined success.
53


Appendix A. Descriptive Statistics Tables
Tables A.l, A.2, and A.3 give detailed descriptive statistic information for
withdrawals from MA 1401, MA 1110, and MA 1070. Tables A.4, A.5, and
A.6 give detailed descriptive statistic information for student grades in the same
three classes.
54


ACCUPLACER Measure Statistic Std Error
After Mean 1.667 .65405
After 95% Confidence Interval: Lower Bound -.5146
After 95% Confidence INterval: Upper Bound 2.8479
After 5%Trimmed Mean 1.0741
After Median .5000
After Variance 2.567
After Standard Deviation 1.60208
After Interquartile Range 2.5
After Skewness 1.354
After Kurtosis 1.240 1.741
Before Mean 3.00 .36205
Before 95% Confidence Interval: Lower Bound 2.2699
Before 95% Confidence Interval: Upper Bound 3.7301
Before 5% Trimmed Mean 2.8434
Before Median 2.00
Before Variance 5.767
Before Standard Deviation 2.40155
Before Interquartile Range 4
Before Skewness .865 .357
Before Kurtosis .219 .702
Table A.l: Descriptive Statistics for MA 1401 Withdrawals
55


ACCUPLACER Measure Statistic Std Error
After Mean 3.0667 .63596
After 95% Confidence Interval: Lower Bound 1.7027
After 95% Confidence INterval: Upper Bound 4.4307
After 5%Trimmed Mean 2.9630
After Median 2.000
After Variance 6.067
After Standard Deviation 2.46306
After Interquartile Range 4.00
After Skewness .541 .580
After Kurtosis -.770 1.121
Before Mean 5.6000 .48887
Before 95% Confidence Interval: Lower Bound 4.6065
Before 95% Confidence Interval: Upper Bound 6.5935
Before 5% Trimmed Mean 5.6111
Before Median 5.00
Before Variance 8.365
Before Standard Deviation 2.89218
Before Interquartile Range 5.00
Before Skewness .177 .398
Before Kurtosis -.975 .778
Table A.2: Descriptive Statistics for MA 1110 Withdrawals
56


ACCUPLACER Measure Statistic Std Error
After Mean 2.4286 .56173
After 95% Confidence Interval: Lower Bound 1.2150
After 95% Confidence INterval: Upper Bound 3.6421
After 5%Trimmed Mean 2.3651
After Median 1.500
After Variance 4.418
After Standard Deviation 2.10180
After Interquartile Range 4.00
After Skewness .552 .597
After Kurtosis -1.393 1.154
Before Mean 4.500 .49743
Before 95% Confidence Interval: Lower Bound 3.4939
Before 95% Confidence Interval: Upper Bound 5.5061
Before 5% Trimmed Mean 4.3889
Before Median 3.00
Before Variance 9.897
Before Standard Deviation 3.14602
Before Interquartile Range 5.75
Before Skewness .718 .374
Before Kurtosis -.840 .733
Table A.3: Descriptive Statistics for MA 1070 Withdrawals
57


ACCUPLACER Measure Statistic Std Error
After Mean 2.8180 .11619
After 95% Confidence Interval: Lower Bound 2.5871
After 95% Confidence INterval: Upper Bound 3.0489
After 5%Trimmed Mean 2.9089
After Median 3.00
After Variance 1.201
After Standard Deviation 1.09613
After Interquartile Range 1.70
After Skewness -1.118 .255
After Kurtosis .847 .506
Before Mean 2.5647 .04124
Before 95% Confidence Interval: Lower Bound 2.4837
Before 95% Confidence Interval: Upper Bound 2.6456
Before 5% Trimmed Mean 2.6274
Before Median 3.00
Before Variance 1.554
Before Standard Deviation 1.24672
Before Interquartile Range 1.70
Before Skewness -.666 .081
Before Kurtosis -.517 .162
Table A.4: Descriptive Statistics for MA 1401 Student Grades
58


ACCUPLACER Measure Statistic Std Error
After Mean 2.3261 .06709
After 95% Confidence Interval: Lower Bound 2.1942
After 95% Confidence INterval: Upper Bound 2.4581
After 5%Trimmed Mean 2.3624
After Median 2.300
After Variance 1.481
After Standard Deviation 1.21695
After Interquartile Range 1.00
After Skewness -.566 .134
After Kurtosis -.514 .268
Before Mean 2.4153 .04502
Before 95% Confidence Interval: Lower Bound 2.3269
Before 95% Confidence Interval: Upper Bound 2.5036
Before 5% Trimmed Mean 2.4614
Before Median 2.700
Before Variance 1.739
Before Standard Deviation 1.31857
Before Interquartile Range 1.60
Before Skewness -.567 .083
Before Kurtosis -.783 .167
Table A.5: Descriptive Statistics for MA 1110 Student Grades
59


ACCUPLACER Measure Statistic Std Error
After Mean 2.4945 .06574
After 95% Confidence Interval: Lower Bound 2.3652
After 95% Confidence INterval: Upper Bound 2.6238
After 5%Trimmed Mean 2.5494
After Median 3.00
After Variance 1.651
After Standard Deviation 1.28492
After Interquartile Range 1.70
After Skewness -.656 .125
After Kurtosis -.624 .249
Before Mean 2.5157 .03883
Before 95% Confidence Interval: Lower Bound 2.4395
Before 95% Confidence Interval: Upper Bound 2.5919
Before 5% Trimmed Mean 2.5730
Before Median 3.00
Before Variance 1.663
Before Standard Deviation 1.28963
Before Interquartile Range 1.70
Before Skewness -.634 .074
Before Kurtosis -.634 .147
Table A.6: Descriptive Statistics for MA 1070 Student Grades
60


REFERENCES
[1] W.B. Armstrong. Explaining community college outcomes by analyzing
student data and instructor effects. Unpublished Doctoral Dissertation;
University of California Los Angeles.
[2] J. Baron and M.F. Norman. SATs, achievement tests, and high-school class
rank as predictors of college performance. Educational and Psychological
Measurement, 52:1047-1055, 1992.
[3] C. Brown. Placement testing and remedial mathematics for post-secondary
students: Prescription for success? Memorial University of Newfoundland,
Winter 1999.
[4] N. W. Burton and L. Ramist. Predicting success in college SAT studies of
classes graduating since 1980. Technical Report 2001-2, The College Board,
2001.
[5] A. Cripps. LIsing artificial neural nets to predict academic performance.
Technical Report 0-89791-820-7, Department of Computer Science Middle
Tennessee State University, 1996.
[6] M. DeGrow. Cutting back on catching up: Reducing the need for remedi-
ation in Colorado higher education. Technical Report IP-11-2003, Indepen-
dence Institute, 2003.
[7] J. Dvorak. Publications experience as a predictor of college success. Jour-
nalism Quarterly, 66:702-706, 1989.
[8] V.K. Isbell. Student placement in entry level courses of reading, writing,
and mathematics in community colleges. Dissertation Abstracts Interna-
tional, 50-07:A1871, 1988.
[9] B.G. Jenkins. Meeting the diverse needs of two-year college students
through appropriate course placement. ERIC Document Reproduction Ser-
vice, ED 341444, 1991. Redlands Community College, OK.
[10] J.L. Kobrin and G.B. Milewski. Students with discrepant high school GPA
and SAT I scores. College Entrance Examination Board, Research Notes
15, 2002.
61


[11] T. A. Lovegreen. Predicting the academic success of female engineering
students during the first year of college using the SAT and non-cognitive
variables. Masters thesis, Virginia Polytechnic Institute and State Univer-
sity.
[12] J. Marwick. Alternative methods of mathematics placement. The Commu-
nity College Enterprise, Fall:41-50, 2002.
[13] H. R. Mzumara and M. D. Shermis. Predictive validity of placement test
scores for course placement at IUPUI: Summer and Fall 2000. Technical
Report 317-274-2620, Indiana University Purdue University Indianapolis,
2001.
[14] P. D. Neal. Extra-curricular activities as predictors of college success. Cal-
ifornia State University-Bakersheld.
[15] C. T. ONeil. The effect of an NCTM standards-based curriculum on stu-
dent performance in college mathematics courses. Masters thesis, Central
Connecticut State University.
[16] F. L. Oswald, N. Schmidt, B. H. Kim, L. J. Ramsay, and M. A. Gille-
spie. Developing a bio-data measure and situational judgment inventory as
predictors of college student performance.
[17] K. Pearce and R. M. Anderson. The effects of placement examinations
and enforcing prerequisites on student success in entry-level mathematics
courses. Texas Tech University Department of Mathematics and Statistics.
[18] AMATYC Placement and Assessment Committee. Initial placement of
two-year college students into the mathematics curriculum. Position Paper:
http: //www.amatyc.org/PositionPapers/Placement.html.
[19] L. Ramist, C. Lewis, and L. McCamley-Jenkins. Student group differences
in predicting college grades: Sex, language, and ethnic groups. Technical
Report 93-1, College Board Report, 1994.
[20] N.D. Rice and E.M. Darke. Differences between leadership and academic
scholarship recipients retention and cumulative grade point average. Col-
lege Student Affairs, 19:20-28, 2000.
[21] J. Roth, G.G. Grans, R. L. Carter, M. Ariet, and M.B. Resnick. Effect of
high school course-taking and grades on passing a college placement test.
High School Journal, 84:72-87, 2001.
62


[22] R. Sawyer. Decision theory models for validating course placement tests.
Journal of Educational Measurement, 33:271- 290, 1996.
[23] T. M. Spitzer. Predictors of college success: A comparison of traditional
and nontraditional age students. NASP, 38:82-98, 2000.
[24] B.D. Wade and C. Walker. Assessing the effect of various academic pa-
rameters on the academic performance of honors students at a southern
university. Education, 115:63-69, 1994.
[25] P. Waycaster. The best predictors of success in developmental mathematics
courses. Inquiry, 9, 2004.
[26] T. Willett. Accuplacer validation and comparison to MDTP. Cabrillo
College Office of Institutional Research, April 7th, 2000.
[27] W.W. Willingham. Success in college: The role of personal qualities and
academic ability. College Entrance Examination Board and Educational
Testing Service, College Board Publications:New York, 1985.
[28] J.W. Young and C.A. Barrett. Analyzing high school transcripts to improve
prediction of college performance. Journal of College Admission, 137:25-29,
1992.
63