Citation
Acquisition vs. retrieval deficit

Material Information

Title:
Acquisition vs. retrieval deficit the nature of verbal memory impairment in relapsing-remitting multiple sclerosis
Creator:
Mitchell, Sandra M
Place of Publication:
Denver, Colo.
Publisher:
University of Colorado Denver
Publication Date:
Language:
English
Physical Description:
61 leaves : ; 28 cm

Thesis/Dissertation Information

Degree:
Master's ( Master of Arts)
Degree Grantor:
University of Colorado Denver
Degree Divisions:
Department of Psychology, CU Denver
Degree Disciplines:
Psychology
Committee Chair:
Lafosse, Jose M.
Committee Members:
Albeck, Dave S.
Towler, Annette

Subjects

Subjects / Keywords:
Multiple sclerosis ( lcsh )
Memory disorders ( lcsh )
Memory disorders ( fast )
Multiple sclerosis ( fast )
Genre:
bibliography ( marcgt )
theses ( marcgt )
non-fiction ( marcgt )

Notes

Bibliography:
Includes bibliographical references (leaves 53-61).
General Note:
Department of Psychology
Statement of Responsibility:
by Sandra M. Mitchell.

Record Information

Source Institution:
University of Colorado Denver
Holding Location:
Auraria Library
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
54530987 ( OCLC )
ocm54530987
Classification:
LD1190.L645 2003m M57 ( lcc )

Full Text
ACQUISITION VS. RETRIEVAL DEFICIT: THE NATURE OF VERBAL
MEMORY IMPAIRMENT IN RELAPSING-REMITTING
MULTIPLE SCLEROSIS
by
Sandra M. Mitchell
B.A., California State University, 2000
A thesis submitted to
University of Colorado at Denver
in partial fulfillment
of the requirements for the degree of
Master of Arts
Psychology
2003


This thesis for the Master of Arts
degree by
Sandra M. Mitchell
has been approved
by
Dave S. Albeck
Annette Towler
H-28-03
Date


Mitchell, Sandra Marie (M.A., Psychology)
Acquisition vs. Retrieval Deficit: The Nature of Verbal Memory Impairment in
Relapsing-Remitting Multiple Sclerosis
Thesis directed by Associate Professor Jose M. Lafosse
ABSTRACT
Memory impairment is one of the most extensively studied cognitive effects
of multiple sclerosis (MS), yet the specific nature of the deficit remains unclear. The
primary purpose of this study is to determine whether memory deficits observed in
MS can be attributed to acquisition or recall impairment, or some combination of
these. Fifty-three patients with clinically definite multiple sclerosis and 31 matched,
healthy controls (HC) completed the California Verbal Learning Test, a commonly
used neuropsychological measure of learning and memory. The MS patient group
demonstrated poor performance on measures of acquisition, free recall, and
recognition compared to the HC group, with the greatest impairment on recall tasks.
There was no difference between the MS and HC groups on a variety of learning
strategies. When the MS groups was split into better and worse recall groups,
however, it was found that the MS patients with deficient recall utilized less semantic
clustering and demonstrated less consistency than their better recall counterparts.
m


Meta-analysis of the existing literature utilizing list-learning measures revealed
moderate to large effects on acquisition and delayed free recall, with acquisition more
impaired than free recall. While there has been considerable debate in
neuropsychology about which theory best demonstrates the exact nature of this
memory deficit in multiple sclerosis, our results reveal that impairments in both
learning and retrieval contribute to the memory difficulties experienced by patients
with multiple sclerosis.
This abstract accurately represents the content of the candidates thesis. I recommend
its publication.
iv


ACKNOWLEDGEMENTS
I would like to thank Nancy Leech, Ph.D., in the Department of Education for her
assistance on the meta-analysis preparation. I would also like to thank my advisor,
Jose Lafosse, Ph.D. for his limitless patience and encouragement on this project.


CONTENTS
Tables...............................................................ix
CHAPTER
1. INTRODUCTION...........................................................1
Subtypes of Multiple Sclerosis........................................2
Relapsing-Remitting MS.............................................2
Primary-Progressive MS.............................................2
Secondary Progressive MS...........................................3
Progressive-Relapsing MS...........................................3
Benign MS..........................................................3
Literature Review.....................................................4
Purpose of Study......................................................9
2. METHODS...............................................................11
Overview.............................................................11
Participants.........................................................12
Neuropsychological Tests.............................................12
Mini-Mental State Examination.....................................12
Beck Depression Inventory-2.......................................13
California Verbal Learning Test...................................13
vi


California Verbal Learning Test-II....................................13
Procedure................................................................15
Dependent Variables......................................................16
Trials 1-5 Total......................................................16
Long-Delay Free Recall................................................16
Recognition Hits......................................................17
False-Positive Rate...................................................17
Discriminability Index................................................17
Semantic Clustering...................................................17
Serial Clustering (Bi-directional)....................................18
Across-Trial Recall Consistency.......................................18
Memory Testing...........................................................18
3. RESULTS...................................................................20
Study 1:CVLT.............................................................20
Study 2: CVLT-II.........................................................26
Learning Characteristics..............................................29
Study 3: Meta-Analysis...................................................31
Procedure.............................................................31
Analyses..............................................................33
Results...............................................................35
Vll


4. DISCUSSION..................................................39
Strengths and Limitations................................47
Summary..................................................49
APPENDIX
Effect Sizes by Individual Studies..........................50
REFERENCES ....................................................53
vm


TABLES
Table
2.1 Characteristics of the Multiple Sclerosis (MS) and Healthy
Control (HC) Groups....................................................14
3.1 Characteristics of the Multiple Sclerosis (MS) and Healthy
Control (HC) Groups by CVLT Version Completed.........................21
3.2 Memory Performance CVLT Takers Only (A); and CVLT and
CVLT-II Combined, All Results Normed to the CVLT (B)...................23
3.3 Comparison of Learning and Recall based on CVLT Standard Scores
(MS patients only).....................................................25
3.4 Memory Performance CVLT-II Takers Only (A); and CVLT and
CVLT-II Combined, All Results Normed to the CVLT-II (B)................27
3.5 Comparison of Learning and Recall based on CVLT-II Standard Scores
(MS patients only).....................................................29
3.6 Learning Characteristics of MS and HC Groups............................30
3.7 Learning Characteristics of Better and Worse Recaller Groups............31
3.8 Meta-Analysis Results...................................................38
IX


CHAPTER 1
INTRODUCTION
Multiple sclerosis (MS) is the most common neurological disorder in young
adults, with symptoms typically first appearing between the ages of 20 and 40. MS
afflicts approximately 350,000 people in the United States and over 2.5 million
worldwide (National Multiple Sclerosis Society, 2001). In Colorado, the prevalence
rates are 1 in 800 residents10 times more common than in Southern sites (Colorado
HealthSite, 2003; Rocky Mountain MS Center, 2002). MS is more prevalent in
individuals living in the extreme northern and southern hemispheres; with prevalence
rates increasing the further from the equator one lives. In fact, migration at particular
ages can affect an individuals susceptibility to developing MS. A child migrating
from a high- to low-risk area (and vice versa) before puberty acquires the risk of the
area to which he or she has moved. The same relocation by a person over the age of
15 retains the risk characteristic of the area from which he or she moved (National
Multiple Sclerosis Society, 2003).
MS is a progressive disease of the central nervous system, which destroys the
protective myelin sheathing that comprises subcortical white matter tracts. While the
cause of MS is still unknown, it is believed that there is an autoimmune component to
the disease where the body attacks its own myelin. Symptoms manifest differently
1


across individuals depending on location of the lesions. Common symptoms include
difficulties with vision, speech, balance, and coordination; bladder, bowel, and sexual
dysfunction; cognitive changes, mood swings, pain, weakness, numbness, fatigue,
sensitivity to heat and impaired mobility (Multiple Sclerosis Association of America,
2002). The course ofMS is unpredictable. The disease minimally affects some
people while others have rapid progress to total disability, with most people falling
between these two extremes. Although every individual will experience a different
combination of MS symptoms, there are a number of distinct patterns relating to the
course of the disease.
Subtypes of Multiple Sclerosis
Relapsing-Remitting MS
In this form of MS, there are unpredictable relapses during which new symptoms
appear or existing symptoms become more severe. This can last for days, weeks or
months followed by partial or total remission. The disease can then remain inactive
for months or years. The relapsing-remitting form of MS is the most common.
Primary-Progressive MS
This form of MS is characterized by a lack of distinct attacks, but with slow onset and
steadily worsening symptoms with no discernible remission. There is an
2


accumulation of deficits and disability that may level off at some point or continue
over months and years. This form of MS is relatively rare.
Secondary-Progressive MS
For some individuals who initially have relapsing-remitting MS, there is the
development of progressive disability later in the course of the disease often with
superimposed relapses. Prior to modem pharmacological treatments, about half of all
patients with relapsing-remitting MS developed a secondary-progressive course.
Currently, it is unknown how long these medications will delay the progressive
symptoms.
Progressive-Relapsing MS
Like the primary-progressive subtype, these patients experience a steady worsening
of symptoms but also have acute exacerbations as seen in the relapsing-remitting
course without the benefit of recovery periods. This form of MS is also relatively
rare.
Benign MS
Some patients with relapsing-remitting MS experience only a few mild symptoms in
the early stages of the disease and sustain a complete recovery after an exacerbation.
There is no progressive deterioration in this form of MS and there is no permanent
3


disability. Benign MS can only be identified when there is minimal disability 10-15
years after onset.
Literature Review
Memory impairment is one of the most extensively studied cognitive effects
of multiple sclerosis, yet the specific nature of the deficit remains unclear.
Investigators have traditionally viewed such memory impairment as a retrieval deficit
(Beatty, Goodkin, Monson, Beatty, & Hertsgaard, 1988; Rao, Hammeke, McQuillen,
Khatri, & Lloyd, 1984; Rao, Leo, & St Aubin-Faubert, 1989). Studies employing
verbal list-learning tasks provided evidence for the retrieval hypothesis by showing
that while free recall was significantly poorer in MS patients than in healthy controls,
recognition remained intact. This suggested that encoding occurred at some level but
patients were unable to retrieve the information from memory without extra retrieval
cues. Some researchers, however, hypothesized that MS patients also acquired fewer
words during initial learning trials and reexamined memory impairment as an
acquisition deficit (DeLuca, Barbieri-Berger, & Johnson, 1994; DeLuca, Gaudino,
Diamond, Christodoulou, & Engel, 1998; Demaree, Gaudino, DeLuca, & Ricker,
2000; van den Burg, van Zomeren, Minderhoud, Prange, & Meijer, 1987). The
acquisition hypothesis proposes that the reason MS patients recall less is because they
learn less initially. By extension, previous studies of memory in MS may be
4


inherently confounded because they do not control for between-group differences in
initial learning (DeLuca et al., 1994).
In an attempt to control for initial learning differences, recent studies have
used a modified version of the Selective Reminding Test (SRT; Bushke & Fuld,
1974). On the modified SRT, each participant is allowed up to 15 trials to reach a
pre-established criterion to learn the entire list of 10 words instead of the standard
fixed number of 10 trials. Once a subject recalls the entire list on two consecutive
trials, the words are considered learned and the learning phase of the test is
discontinued. Studies using this method have demonstrated that MS patients require
more trials to learn the word list than healthy controls (DeLuca et al., 1994; DeLuca
et al., 1998; Demaree et al., 2000; Gaudino, Chiaravalloti, DeLuca, & Diamond,
2001), but once they acquire the information, they perform as well as controls on both
free recall and recognition tasks (DeLuca et al., 1994; DeLuca et al., 1998). One of
the most consistently reported learning measures in these studies, trials to criterion,
refers to the number of trials a subject needs to successfully recall the entire list two
consecutive times. The trials to criterion variable is unique and has no equivalent on
other verbal list-learning tests, making it difficult to compare modified SRT studies
with studies using traditional list-learning tests. A major problem with using tests
that dont follow standard administration procedures is that there are no normative
data for comparison; so while differences between groups can be identified,
interpretations relative to the general population are more difficult to ascertain.
5


A recent meta-analysis (Thornton & Raz, 1997) concluded that the empirical
support for a retrieval-based account of memory impairment in MS is not strong.
While this meta-analysis provided evidence that recall and recognition are both
impaired in MS, with free recall significantly worse than recognition, it provided no
direct empirical support for the idea that acquisition is impaired. In contrast, a
subsequent meta-analysis concluded that retrieval deficits are the primary memory
impairment in MS (Zakzanis, Leach, & Kaplan, 1999). The meta-analysis by
Zakzanis, however, also did not directly examine whether acquisition is impaired. A
thorough and direct analysis of acquisition, recall and recognition is necessary in
order to support either the retrieval or the acquisition hypothesis. A third meta-
analysis (Wishart & Sharpe, 1997) examined a wide range of neuropsychological
domains including memory and found support for impairment in all three domains:
verbal learning, delayed recall and recognition tasks. While learning showed the
largest effect size, the results of this meta-analysis are difficult to interpret because
the effect sizes for learning and recall were limited to verbal measures but the
recognition effect size was based on both verbal and nonverbal measures combined.
All three meta-analyses included studies that assessed verbal learning with list
learning, paragraph learning, and paired-associates tasks. While all of these tests are
designed to assess memory function, they do not assess acquisition, recall, and
recognition equally well. Moreover, they are fundamentally different and are likely
tapping different mechanisms involved in memory processes. We identified 27
6


studies that used a traditional list-learning task to evaluate memory performance (see
Appendix). Of these, only 12 reported data for acquisition, recall and recognition
performance.
Several possible methodological issues may account for the unclear nature of
memory impairment in MS. As mentioned earlier, it is difficult to compare studies
that have used list-learning tests in which words are presented a fixed number of
times with studies that have used list-learning tests in which words are presented a
variable number of times. It is also difficult to interpret study results that do not
report data for learning, recall and recognition. Some studies that used recognition
tasks report scores corrected for false positives (Diamond, DeLuca, Johnson, &
Kelley, 1997; Rao et al., 1984; Thornton, Raz, & Tucker, 2002) while others do not
(Caine, Bamford, Schiffer, Shoulson, & Levy, 1986; Coolidge, Middleton, & Griego,
1996; Maurelli et al., 1992; Wallace & Holmes, 1993). Failing to correct for false
positives could lead to an artificially high estimate of recognition performance.
Depressive symptoms are frequently observed in patients with multiple
sclerosis. Depression is known to have a negative effect on memory performance.
Clinically depressed patients are often excluded from patient groups (Beatty, Paul,
Wilbanks et al., 1995; Maurelli et al., 1992). However, in Gaudino (2001), five of 64
MS patients had clinically significant levels of depression yet remained in the study
group. Without controlling for depression or excluding these subjects, it is unclear
what effect the depression may have had on the overall MS group scores. Many
7


studies reported elevated depression scores among MS patients but typically at mild
to moderate levels (Krupp, Sliwinski, Masur, Friedberg, & Coyle, 1994; Minden,
Moes, Orav, Kaplan, & Reich, 1990), and not correlated with memory performance
(Amato et al., 1995; Fisk & Archibald, 2001; Landro, Sletvold, & Celius, 2000; Rao
et al., 1989). Other studies failed to examine depression at all (Caine et al., 1986;
Coolidge et al., 1996; Diamond et al., 1997; Maurelli et al., 1992; van den Burg et al.,
1987; Wallace & Holmes, 1993) so it is unknown whether clinically depressed
individuals were included in the patient group.
Another problem with many of the published studies is the tendency to study
mixed groups of MS patients (relapsing-remitting combined with progressives) and
then make global statements about memory performance. In one study (DeLuca et
al., 1994) where relapsing-remitting (RR), primary progressive (PP) and secondary
progressive (SP) patients were examined together, the MS group demonstrated much
more variability than controls in learning and memory abilities, possibly as a function
of using a mixed group of MS patients. In another study where subgroups were
analyzed separately (Gaudino et al., 2001), RR patients did not demonstrate impaired
learning relative to healthy controls. Yet, the authors concluded broadly that
acquisition is impaired in MS, making no subtype distinction. Moreover, when
examining the combined MS group the authors pointed to the contrast between a
large effect size on acquisition and a smaller effect size on recall as support for their
conclusion that acquisition is the primary impairment. However, the most impaired
8


SP patients (6 of the 25) that were included in the computation of the learning effect
size were excluded from the recall effect size. In other words, 24% of the SP patients
(whose recall was, indeed, found to be lower than that of participants who reached the
learning criterion) were not included in the recall analysis, further casting doubt on
the conclusion that acquisition represents the primary impairment.
Purpose of Study
The primary purpose of this study is to determine whether memory deficits observed
in MS can be attributed to acquisition or recall impairment, or some combination of
these. Since the acquisition hypothesis arose from the use of verbal list-learning tasks
(DeLuca et al., 1994; DeLuca et al., 1998; DeLuca, Johnson, Beldowicz, & Natelson,
1995; Demaree et al., 2000; Diamond et al., 1997; Gaudino et al., 2001), this study
will examine verbal memory function as measured by a standard administration of the
California Verbal Learning Test (Delis, Kramer, Kaplan, & Ober, 2000). Initial
descriptive analyses will examine the overall performance of relapse-remitting MS
patients and healthy controls on learning, recall and recognition. Subsequent analyses
will examine the acquisition versus retrieval hypotheses. To accomplish this, we will
divide the MS group into better learners and worse learners and then evaluate
recall performance. If the acquisition hypothesis is correct, and recall is impaired as a
result of deficient acquisition, then worse learners will have poorer recall than better
learners. If the retrieval hypothesis is true, then grouping by learning should be
9


inconsequential and both groups should demonstrate recall impairment and normal
recognition. Next, to evaluate learning performance we will divide the original MS
group based on better and worse recall. In this case, the acquisition hypothesis would
be supported if the worse recall group also demonstrates worse learning. Conversely,
the retrieval hypothesis would be supported if the group with impaired recall showed
relatively normal learning and recognition. Finally, we will examine the existing
literature using meta-analytic techniques to summarize previous verbal memory
findings.
10


CHAPTER 2
METHODS
Overview
This study was designed to look at memory in a variety of ways. First, we
take a traditional approach by comparing MS patients with matched control subjects.
Because data was collected over a four-year time span, we were able to utilize either
the original version of the CVLT or the revised version, CVLT-II. Since these two
instruments were highly correlated we were able to standardize each participants
scores according to two sets of normative data A detailed description of this process
follows. As a result, we were able to analyze the data looking at the smaller groups
of participants who completed a particular version of the CVLT, as well as a larger
combined group with all participants standardized to one set of normative data
regardless of the version actually completed.
Next, we looked only at the MS patients and divided them based on their
learning and recall performance. We were then able to compare the resulting
cognitively better or worse groups on a variety of memory variables and learning
characteristics.
Finally, we prepared a meta-analysis of all existing literature that used list-
learning measures of memory in MS. This meta-analysis looks at individual study
11


effect sizes on the three variables of interest in our study: acquisition, delayed free
recall, and recognition.
Participants
Fifty-three patients with clinically-definite MS (Poser et al., 1983) were
recruited from the Department of Neurology at the University of Colorado Health
Sciences Center, with assistance of the Colorado Chapter of the National Multiple
Sclerosis Society. Thirty-one healthy controls (HC) recruited from the Denver area
also participated in this study. All participants were required to have at least a high
school education. Demographic and clinical characteristics of the study sample are
shown in Table 2.1.
Neuropsychological Tests
Participants completed a neuropsychological battery of tests as part of a
larger, more comprehensive study. Only those tests relevant to this study are
reviewed here.
Mini-Mental State Examination
The MMSE is an 11-item, verbally administered questionnaire that is focused
exclusively on global cognitive impairment. It is designed to cursorily assess a
patients general orientation, memory, attention, and language.
12


Beck Depression Inventory-2
The BDI-2 is a 21-item self-report questionnaire that is focused on the full spectrum
of depressive symptoms experienced in the preceding two-week period. Each item is
rated on a 4-point scale of increasing severity. Participants completed this test
manually at the end of the neuropsychological battery.
California Verbal Learning Test
The original version of the CVLT consists of two, 16-word lists of common items
from four categories designed to measure acquisition, cued and free recall, and
recognition, as well as a variety of learning strategies over several immediate and
delayed recall trials. List A, which is comprised of four semantic categories, is read
aloud at the rate of 1 word per second and the participant is asked to recall the words
to the examiner after each of five presentations. Interference List B, which is also
comprised of four semantic categories, is presented in Trial 6. The interference trial
is followed by a variety of List A recall trials without further presentation to measure
free and cued recall and recognition discrimination.
California Verbal Learning Test-II
The CVLT-II is structured and administered in the same fashion as the CVLT using
different words and semantic categories. The CVLT was revised using a much
13


Table 2.1
Characteristics of the Multiple Sclerosis (MS) and Healthy Control (HC) Groups
MS HC
Characteristic M SD M SD
Age (yrs.) 41.0 (9.2) 41.0 (7.5)
Education (yrs.) 14.9 (2.2) 15.9 (1.4)
MMSE 29.5 (0.8) 29.8 (0.5)
Gender (M/F) (5/48) (5/26)
BDI-2 9.2 (6.7) 4.3 (5.5)
Disease Duration 5.7 (5.3)
Note. Groups were significantly different on education, MMSE, and BDI-2; n = 53
for the MS group and n = 31 for the HC group. MMSE = Mini-Mental State
Examination, BDI-2 = Beck Depression Inventory-2.
14


larger normative sample designed to more closely reflect the general U.S. population
and uses improved measures of learning strategies.
Procedure
Global cognitive functioning was assessed with the Mini-Mental State
Examination (Folstein, Folstein, & McHugh, 1975) and depressive symptoms were
assessed with the Beck Depression Inventory-2 (Beck & Steer, 1987). Both groups
were equivalent in terms of age and gender distribution. Despite a difference of less
than half a point, the HC group scored significantly higher on the MMSE than the MS
group, t(82) = -2.45, p < .05. There was also a small but significant difference
between both groups on years of education, /(81) = -2.61, p < .05. Despite this
significant difference, education was not significantly correlated with the dependent
variables (range = -0.03 0.25). As expected, the MS group endorsed significantly
more depressive symptoms on the BDI-2 than the HC group, /(82) = -3.46, p < .01.
However BDI-2 scores were not significantly correlated with the dependent variables
(range = -0.20 0.13), and the average BDI-2 scores for both groups fell in the
minimal depression category according to the BDI-2 classification.
The MS patients all had the relapsing-remitting subtype of the disease (Poser
et al., 1983), and they had been diagnosed for an average of 5.7 years. None were in
a state of relapse at the time of study participation. The MS patients had either never
15


taken immunomodulatory medications of any kind or had taken them for less than
one year.
Exclusionary criteria for all participants included neurological disorders other
than MS, traumatic brain injury with loss of consciousness, alcohol or other substance
abuse, coronary artery disease, chronic obstructive pulmonary disease, uncontrolled
hypertension, diabetes mellitus, kidney or liver disease, endocrine disorders, visual or
motor impairment that might interfere with neuropsychological testing, severe
depression, and use of medication that was judged to cause drowsiness or confusion.
Some participants in both groups were taking medications for disorders including
depression, anxiety, insomnia, fatigue, spasticity, urinary frequency, neuropathy,
hypertension, and hypercholesterolemia.
Dependent Variables
Trials 1-5 Total
The total of correct items across trials is considered a global index of verbal learning.
A normal score on this measure is indicative of good auditory attention and verbal
learning skills.
Long-Delay Free Recall
The number of correct items recalled after a 20-minute delay is designed to measure
retention without the intervening List B that may result in retroactive interference.
16


Recognition Hits
This measure indicates the number of words the individual can correctly identify on a
forced-choice design. This is not the best recognition measure, however, because a
person who responds yes to both list and distracter words will obtain the same score
as the individual who only identifies the target words. For this reason, it is important
to also look at the false-positive rate or discriminability measures.
False-Positive Rate
This measure indicates the number of distracter words incorrectly identified as List A
words. There are different subtypes of false-positives that are indicative of
increasingly severe memory impairment.
Discriminability Index
Discriminability is computed from both the recognition hits and the false-positives
and is the best measure of recognition.
Semantic Clustering
The score obtained on semantic clustering indicates how well the individual uses
active learning to reorganize the word list into categories. Low semantic-clustering
scores are correlated with poor performance on recall and are often indicative of a
less effective or random learning strategy.
17


Serial Clustering (Bi-directional)
The score obtained on serial clustering indicates how often the individual recalls the
words in the same temporal order in which they were presented. High serial
clustering scores are often correlated with poor performance on recall. The bi-
directional score is a combination of both forward and backward serial recall. It is
common for individuals that employ this style to first echo back the last words on the
list in backward order, then recall the first words in the forward direction. This
measure is designed to be sensitive to this learning strategy.
Across-Trial Recall Consistency
Recall Consistency assesses the frequency with which an individual recalls the same
words from one learning trial to the next. A low consistency score is indicative of a
disorganized learning style and may be signify poor learning plan formulation.
Inconsistent recall may also occur when an individual repeatedly abandons one
learning strategy for another across trials.
\
Memory Testing
Approximately 55% of our participants took the original version of the
California Verbal Learning Test (CVLT; Delis, Kramer, Kaplan, & Ober, 1987) in
Study 1, and approximately 45% took the second version of the CVLT (CVLT-II;
Delis et al., 2000) in Study 2. A comparison study of the CVLT and the CVLT-II
18


found that the raw scores for a number of variables, including trials 1-5 total and
long-delay free recall (LDFR), were nearly identical and highly correlated with each
other, indicating that both test versions are of equivalent difficulty (Delis et al., 2000).
While discriminability is the best measure of recognition, the measures are computed
differently on the CVLT and CVLT-II; therefore they are not directly comparable
from one version to the next.
19


CHAPTER 3
RESULTS
Study 1:CVLT
Thirty-one of the 53 MS patients and 17 of the 31 HC participants completed
the CVLT. Demographic and clinical characteristics of the Study 1 sample are shown
in Table 3.1. There were no significant differences between groups on age, years of
education or MMSE. The MS group reported significantly more depressive
symptoms on the BDI-2 than the HC group, t{46) = 3.17, p < .01; however, both
groups fell in the minimal depression range. Four primary indices were examined:
trials 1-5 total, LDFR, recognition hits and discriminability. Performance on trials 1-
5 total is expressed as a T-score (M = 50, SD = 10) and performance on the remaining
variables is expressed as a standard score (M = 0, SD = 1), based on the CVLT
normative data stratified by age and gender.
A t-test was used to compare groups on trials 1-5 total. Wilcoxon rank-sum
tests were used for analyses involving the remaining variables because they represent
an ordinal scale rather than a true interval scale; however, mean values rather than
ranks are reported for these measures because they are familiar to most
neuropsychologists. The Wilcoxon statistic is expressed as a z-score when N > 25.
20


Table 3.1
Characteristics of the Multiple Sclerosis (MS) and Healthy Control (HC) Groups by
CVLT Version Completed
CVLT CVLT-II
MS HC MS HC
Characteristic M SD M SD M SD M SD
Age 40.3 (9.0) 43.7 (8.0) 42.1 (9.6) 37.6 (5.4)
Education 15.2 (2.2) 16.2 (1.6) 14.6 (2.1) 15.6 (1.0)
MMSE 29.5 (0.9) 29.9 (0.5) 29.5 (0.7) 29.8 (0.4)
Gender (M/F) (2/29) (2/15) (3/19) (3/11)
BDI-2 9.9 (6.4) 4.0 (5.7) 8.2 (7.1) 4.6 (5.5)
Disease Duration 5.7 (5.2) 5.6 (5.6)
Note: CVLT:n = 31 for the MS group and n = 17 for the HC group. CVLT-II: n =
22 for the MS group and n = 14 for the HC group. MMSE = Mini-Mental State
Examination, BDI-2 = Beck Depression Inventory-2.
21


As shown in Table 3.2(A), on trials 1-5 total, the MS group did not differ from
the HC group in their acquisition performance, f(46) = -1.27, p = n.s. The MS group
performed significantly worse, however, on LDFR, z = -2.37, p < .05, while no
significant differences were observed on recognition hits, z = 0.24, p = n.s., or
discriminability, z = 0.93, p = n.s.
In addition to the participants who completed the CVLT, 22 MS patients and
14 HC participants completed the CVLT-II. The MS patients who completed the
CVLT-II did not significantly differ from the MS patients who completed the CVLT
on age r(51) = -0.70, p = n.s., years of education t(5\) = 0.95, p = n.s., MMSE /(51) =
0.07, p = n.s., BDI-21(51) = 0.88, p = n.s., or disease duration since diagnosis, t(51) =
0.06, p = n.s. Similarly, the HC participants who completed the CVLT-II did not
significantly differ from the HC participants who completed the CVLT on years of
education, t{29) = 1.40, p = n.s., MMSE, /(29) = 0.58, p= n.s., or BDI-2, /(20) = -0.32.
Therefore, we decided to include the participants who took the CVLT-II by
converting their raw scores to CVLT standard scores, thereby increasing the power of
our analyses. When all 84 participants scores were standardized according to the
norms from the original CVLT, the MS group acquired fewer words than the controls
on trials 1-5 total, 7(82) = -2.84, p < .01 (see Table 3.2(B)). The MS group also
recalled fewer words than controls on LDFR, z = -3.66, p < .01. There was no
difference, however, on recognition hits, z = -1.48, p = n.s. Discriminability could
22


Table 3.2
Memory Performance CVLT Takers Only (A); and CVLT and CVLT-II Combined,
All Results Normed to the CVLT (B).
MS HC
Measure M SD M SD P d
(A)
Trials 1-5 total 43.8 (10.7) 47.8 (9.4) ns 0.38
Long-delay free recall -0.8 (1.3) 0.1 (0.9) <.05 0.81
Recognition hits -0.2 (1.4) 0.1 (0J) ns 0.18
Discriminability -0.1 (0.7) 0.1 (0.3) ns 0.31
(B)
Trials 1-5 total 40.9 (10.5) 47.5 (9.6) <.01 0.64
Long-delay free recall -0.9 (1.3) 0.1 (1.0) <.01 0.83
Recognition hits -0.3 (1.3) 0.2 (0.8) ns 0.43
Note: (A) CVLT only: n = 31 for the MS group and n = 17 for the HC group. (B)
CVLT and CVLT-II combined: n = 53 for the MS group and n = 31 for the HC
group; ns = not significant
23


not be compared between groups because the recognition trial that this index is based
upon differs on the two versions of the CVLT.
To test our hypothesis that poor acquisition would result in poor recall, we
divided the combined MS group (n = 53) into better learners and worse learners
based on their performance on trials 1-5 total using CVLT T-scores. Better learners
(n = 29) were defined as those who obtained a T-score of 41 or higher, and worse
learners (n = 24) were defined as those who obtained a T-score of 40 or lower (see
Table 3.3). The MS patients who were worse learners performed significantly worse
on LDFR than those who were better learners, z = -3.96, p < .01. However, there was
no significant difference on recognition hits between these two groups, z= 1.81, p =
n.s.
We were also interested in whether poor recall was a function of poor
acquisition. We, therefore, divided our combined MS group into better recallers
and worse recallers based on their LDFR performance using CVLT standard scores.
Better recallers (n = 22) were defined as those who obtained a standard score of 0 or
higher, and worse recallers (n = 31) were defined as those who obtained a standard
score of-1 or lower (see Table 3.3). The MS patients who were worse recallers
performed significantly worse on trials 1-5 total than those who were better recallers,
t(51) = -5.27, p < .01. However, these two groups did not significantly differ on
recognition hits, z = -1.27, p = n.s.
24


Table 3.3
Comparison of Learning and Recall based on CVLT Standard Scores (MS patients
only)
Better Learners Worse Learners
M SD M SD P d
Long-delay free recall -0.3 (0.9) -1.7 (1.3) <.01 1.30
Recognition hits 0.1 (0.9) -0.7 (1.5) ns 0.63
Better Recallers Worse Recallers
M SD M SD P d
Trials 1-5 total 48.2 (7.7) 35.7 (9.0) <.01 1.47
Recognition Hits 0.1 (1.0) -0.5 (1.4) ns 0.45
Note: Better Learners: n = 29, Worse Learners: n = 24, Better Recallers: n = 22;
Worse Recallers: n = 31; ns = not significant
25


Study 2: CVLT-II
As stated earlier, 22 of the 53 MS patients and 14 of the 31 HC participants
completed the CVLT-II. Demographic and clinical characteristics of the CVLT-II
study sample are shown in Table 3.1. There were no significant differences between
groups on age, years of education, MMSE, or BDI-2. Four primary indices were
examined: trials 1-5 total, LDFR, recognition hits and discriminability. Performance
on trials 1-5 total is expressed as a T-score (M = 50, SD = 10) and performance on the
remaining variables is expressed as a standard score (M = 0, SD =1), based on the
CVLT normative data stratified by age and gender.
A t-test was used to compare groups on trials 1-5 total. Wilcoxon rank-sum
tests were used for analyses involving the remaining variables because they represent
an ordinal scale rather than a true interval scale; however, mean values rather than
ranks are reported for these measures because they are familiar to most
neuropsychologists. The Wilcoxon statistic is expressed as a z-score when N > 25.
As shown in Table 3.4(A), the MS group acquired significantly fewer words
than the HC group on trials 1-5 total, t(38) = -2.50, p < .05, and recalled fewer words
on LDFR, z = -2.63, p < .01. No difference on recognition hits was observed, z =
-1.12, p = n.s. However, the MS group performed significantly worse on
discriminability, z = -3.36, p < .01 when compared to controls. Follow-up analysis
indicated that the difference in discriminability is attributable to the MS group
endorsing significantly more false positives, as reflected by a correspondingly higher
26


Table 3.4
Memory Performance CVLT-II Takers Only (A); and CVLT and CVLT-II
Combined, All Results Normed to the CVLT-II (B)
MS HC
Measure M SD M SD P d
(A)
Trials 1-5 total 49.8 (7.7) 56.6 (8.3) <.05 0.85
Long-delay free recall lO 1 (1.1) 0.5 (0.9) <.01 0.99
Recognition hits -0.5 (0.9) -0.1 (0.5) ns 0.49
Discriminability -0.3 (0.9) 0.7 (0.5) <0.1 1.30
(B)
Trials 1-5 total 52.9 (8.5) 57.4 (8.1) <.05 0.55
Long-delay free recall -0.3 (1.1) 0.5 (0.7) <.01 0.83
Recognition hits -0.5 (1.2) -0.2 (0.5) ns 0.36
Note: (A) CVLT-II only: n = 22 for the MS group and n = 14 for the HC group.
(B) CVLT and CVLT-II combined: n = 53 for the MS group and n = 31 for the HC
group; ns = not significant
27


standard score (M = .30, SD = .92), than the HC group (M = -.50, SD = .39), z -
-2.94, p<.01.
In addition to the participants who completed the CVLT-II, 31 MS patients
and 17 HC participants completed the CVLT, as stated earlier. Given the equivalence
of the groups that took the CVLT and CVLT-II, as described previously in Study 1,
we included the participants who took the CVLT by converting their raw scores to
CVLT-n standard scores, thereby increasing the power of our analyses. When all 84
participants scores were standardized according to the norms from the CVLT-II, the
MS group acquired less, t{82) = -2.42, p < .05, and recalled less, z = -3.49, p < .01
than the HC group, as shown in Table 3.4(B). There was no significant difference in
the number of recognition hits, z = -0.43, p = n.s.
To test our hypothesis that poor acquisition would result in poor recall, we
attempted to divide the combined MS group into better learners and worse
learners according to CVLT-II T-scores as detailed in Study 1. The resulting
groups, however, were unbalanced and the worse learners group (n = 4) was too small
to perform statistical analyses.
To test whether poor recall was a function of poor acquisition, we divided our
combined MS group into better recallers and worse recallers based on their
LDFR performance using CVLT-II standard scores. Better recallers (n = 36) were
defined as those who obtained a standard score of -0.5 or higher, and worse recallers
(n = 17) were defined as those who obtained a standard score of-1 or lower (see
28


Table 3.5). The MS patients who were worse recallers performed significantly worse
on trials 1-5 total, t(51) = -6.17 p < .01, and on recognition hits, z = -2.25, p < .05,
than those who were better recallers.
Table 3.5
Comparison of Learning and Recall based on CVLT-II Standard Scores (MS patients
only)
Better Recallers Worse Recallers
M SD M SD P d
Trials 1-5 total 56.6 (6.4) 44.8 (6.8) <.01 1.814
Recognition Hits -0.2 (0.9) -1.1 (1.5) <.05 0.77
Note: Better Recallers: n = 36, Worse Recallers: n = 17
Learning Characteristics
Since the results indicated that the MS group consistently acquired fewer words than
the HC group on trials 1-5 total, we wondered whether both groups employed
different list-learning strategies. Several studies have looked at semantic clustering
(Jennekens-Schinkel, van der Velde, Sanders, & Lanser, 1990; Laatu, Hamalainen,
Revonsuo, Portin, & Ruutiainen, 1999; Rao et al., 1993; Scarrabelotti & Carroll,
1999; Schmidt et al., 2002), serial clustering (Armstrong et al., 1996; DeLuca et al.,
1994; Fulton et al., 1999; Litvan, Grafman, Vendrell, & Martinez, 1988) and
29


consistency measures (Beatty, Krull et al., 1996; Beatty, Wilbanks et al., 1996;
DeLuca et al., 1998; Rao et al., 1989) as possible explanations for deficient learning,
but with mixed results. Once again, scores for all 84 participants were standardized
according to the norms from the CVLT-II, which was designed using more sensitive
learning strategy formulas. As shown in Table 3.6, when comparing the MS and HC
groups, there were no differences in semantic clustering, z = -1.11, p = n.s. or bi-
directional serial clustering, z -0.46, p = n.s. The MS group demonstrated lower
Across Trial Recall Consistency (ATRC) than the HC group, z = -2.22, p < .05.
Table 3.6
Learning Characteristics of MS and HC Groups
MS HC
Characteristic M SD M SD P d
Semantic Clustering 0.7 1.1 1.0 1.2 ns 0.26
Serial Clustering -0.3 1.2 -0.4 1.0 ns 0.10
ATRC 0.2 0.8 0.5 0.8 <.05 0.41
Note: n = 52 for the MS group and n = 31 for the HC group; ns = not significant;
ATRC = across-trial recall consistency
Despite the fact that we found little differences between the MS and HC
groups on learning strategies, we were interested in whether there were differences
30


within the MS group. Therefore, to examine this issue we split the MS group as
before on the CVLT-IILDFR scores to look for differences on learning. As shown in
Table 3.7, the MS patients who scored higher on LDFR used a semantic clustering
strategy more frequently, z = -3.14, p < .01, and demonstrated more recall consistency
across learning trials, z = -3.05, p < .01. There was no difference between the groups
on serial clustering.
Table 3.7
Learning Characteristics of Better and Worse Recaller Groups
Better Recallers Worse Recallers
Characteristic M SD M SD P d
Semantic Clustering 1.0 1.1 0.0 0.9 <.01 0.98
Serial Clustering -0.4 1.3 0.0 0.9 ns 0.30
ATRC 0.4 0.7 -0.3 0.6 <.01 0.93
Note: n = 36 for the better recallers and n = 16 for the worse recallers; ns = not
significant; ATRC = across-trial recall consistency
Study 3: Meta-Analysis
Procedure
The primary purpose of the meta-analysis was to look specifically at MS
patients memory performance on list-learning tasks such as the CVLT (Delis et al.,
31


1987) and the Rey Auditory Verbal Learning Test (Rey, 1964). We followed the
meta-analysis guidelines outlined in Onwuegbuzie & Leech (2003). Studies selected
for inclusion were restricted to those published in English after 1983 that were
designed to compare MS patients to healthy controls on list learning tests with a
minimum of 10 words. Selected studies provided sufficient data to compute at least
one effect size on a measure of acquisition, delayed recall or recognition. An
extensive literature search of the Medline (1966-March Week 1,2003), ERIC (1966 -
March 2003), and Psyclnfo (1872 March Week 1,2003) databases was performed
searching for keywords multiple sclerosis combined with memory,
neuropsychology, or cognitive. In addition to this electronic search, we reviewed
the reference sections in related meta-analysis and review articles for additional
studies. In the end, 29 studies met our inclusion criteria and are presented below.
Cohens d (Cohen, 1988) was determined to be the most appropriate effect
size formula because it uses a pooled standard deviation that does not assume
homogeneity of variance (Zakzanis, 2001) and weights each effect size by study
sample size. In an effort to create an independent set of effect sizes (Lipsey &
Wilson, 2000), no more than one effect size was used for each memory construct in a
given study. On measures of acquisition, total words across all learning trials was the
preferred variable. When a sum across all learning trials was not reported, the score
on the last learning trial was used. On measures of recall, delayed free recall was
always the variable of choice; cued recall measures were never included. On
32


measures of recognition, total hits corrected for false positives was always the
preferred variable; when such discriminability data were not provided, total
recognition scores (i.e., hits) were used. In studies where the MS group was
artificially divided for analysis (e.g., impaired and preserved groups), pooled means
and standard deviations were used to compute an overall effect size for the construct.
For those studies that reported only t values, conversions to d were made according to
Wolf (1986). Finally, when more than one study was published using the same
patient group, the study with the largest number of participants was selected for
inclusion. This was done to prevent any one sample of patients from overly
influencing the meta-analysis results. Refer to Appendix for a complete list of
individual studies and effect sizes included in the analyses.
Analyses
An effect size was computed for each construct as described above. When means and
standard deviations were not provided, inferential data were used to convert t scores
to Cohens d (Glass, McGaw, & Smith, 1981; Wolf, 1986). Since the standardized
mean difference effect size has been shown to be upwardly biased (Hedges & Olkin,
1985), especially when based on small sample sizes (n < 20), it was necessary to
compute a correction. This unbiased d was used as the effect size in subsequent
analyses.
33


Because effect sizes based on larger samples are considered more reliable than
those based on smaller samples, each of the individual effect sizes was weighted by
its study sample size. This step prevents the effect size of a small study from having
the same influence on the overall mean effect size as a larger study. Hedges & Olkin
(1985) have demonstrated that optimal weights are based on the standard error of the
effect size. The actual weights are computed using the inverse variance weight,
which is the inverse of the squared standard error value. Once each effect size was
appropriately weighted by its error, an overall mean effect size was computed for
each memory construct. The inverse variance weights were also used to derive 95%
confidence intervals. Since we were interested in possible differences between MS
subtypes, effect sizes were analyzed separately for relapsing-remitting and
progressive-type patients, although the majority of studies report mixed patient
samples. Mixed groups and studies that did not report subtype data were analyzed
separately. A fail-safe N (Orwin, 1983) was also computed to demonstrate the
number of additional studies with contradictory findings that would be necessary to
invalidate these results by reducing the mean effect size to a small, typically
meaningless effect size of 0.2.
Analysis of homogeneity assesses whether the variance in the individual effect
sizes that comprise the overall mean effect size are significantly different from that
expected by chance (Cooper & Hedges, 1994; Lipsey & Wilson, 2000; Onwuegbuzie
& Leech, 2003). In a homogeneous distribution, the effect size variance is no greater
34


than what could be expected from sampling error alone. If the homogeneity analysis
results in a significant finding, then the variability is likely due to something other
than that expected purely from sampling error. A heterogeneous distribution suggests
that the effect size may not represent a single underlying effect. The statistical test of
homogeneity is represented as a chi-square distribution with k 1 degrees of
freedom, and is computed as Q. The Q- value is compared with the critical values in
the chi-square tables to check for significant heterogeneity. In the event that the null
hypothesis is rejected and the variance is not considered homogeneous, further
analyses are necessary to identify potential mediating variables.
Results
The mean overall effect sizes for acquisition, delayed free recall and recognition are
shown in Table 3.8. Data were analyzed by disease subtype whenever possible;
however, most studies were comprised of mixed groups. Overall, the MS patients
performed poorly on each of the three memory measures when compared to healthy
controls, regardless of disease subtype. Acquisition scores reflected the greatest
effect, with MS patients performance ranging between approximately three-fourths
to one full standard deviation below that of their healthy counterparts. The patient
groups free recall and recognition also reflected moderate impairment.
The relapsing-remitting group demonstrated a large effect for acquisition with
moderate effects for delayed free recall and recognition. While the variance was
35


homogeneous for recall and recognition hits, the acquisition effect was not (Q =
3.86). Since there were only two studies that contributed data to the acquisition effect
size, this marginally significant heterogeneity is likely attributable to the small sample
of studies. The pattern of acquisition and delayed recall results, while consistent with
the overall meta-analysis, differed from the effects observed in our Study 1 and Study
2. Therefore, we decided to re-analyze the relapsing-remitting data by including our
own results from Study 2. We chose Study 2 data over Study 1 because the
normative sample for the CVLT-II is more representative of the general population.
As shown in Table 9, additional data from our study resulted in a homogeneous mean
effect for acquisition, without changing the overall pattern of results.
Studies with exclusively progressive-type (CP) samples of MS patients
produced the largest effects for each of the three memory domains. The very large
effect of MS on acquisition (d = 0.93) has a corresponding overlap of roughly 46%,
which means that more than half of the patients with MS obtained scores lower than
those obtained by the healthy controls (Zakzanis, 2000). Analysis of homogeneity
confirmed that variance did not exceed chance level. While both the RR and CP
groups demonstrated a similar pattern of effects for each memory domain, the CP
group effects were of a higher magnitude than those for the RR group. Any
interpretation of results for the RR or CP groups should be made with caution,
however, since k = 4 and k = 6, respectively, and not all memory constructs are
equally represented within and between studies.
36


The third group consisted of mixed MS subtypes or subtype data was not
reported. Once again, the memory domain with the largest effect size was acquisition
(d 0.79) and has an overlap of 52% with the healthy control distribution. This
means that the remaining 48% of the MS acquisition distribution reflect scores not
obtained by the HC group. The mixed groups pattern of results fell between that of
the other two groups.
37


Table 3.8. Meta-analysis Results
Memory Variables by Disease Subtype k d 95% Lower limit C.I. Upper limit n6 Q
RelaDsine-Remittine
Acquisition 2 0.79 0.51 1.07 6 3.86*
Delayed Free Recall 3 0.47 0.21 0.72 4 3.27
Recognition 1 0.40 0.09 0.71 1 0.00
RelaDsine-Remittine fReanalvsisl
Acquisition 3 0.75 0.51 0.98 8 4.14
Delayed Free Recall 4 0.55 0.33 0.77 7 5.07
Recognition 2 0.41 0.16 0.67 2 0.01
Progressive Tvoe
Acquisition 4 0.93 0.69 1.16 15 4.94
Delayed Free Recall 5 0.70 0.47 0.93 12 1.93
Recognition 3 0.53 0.24 0.82 5 0.26
Mixed SubtvDes
Acquisition 21 0.79 0.68 0.90 62 24.27
Delayed Free Recall 18 0.68 0.56 0.81 44 21.28
Recognition 11 0.40 0.25 0.55 11 7.29
Note: k = number of studies; d = unbiased effect size; C. I. = confidence interval;
Ns = fail-safe N; Q = test of homogeneity, = p < .05.
38


CHAPTER 4
DISCUSSION
While verbal learning has traditionally been assumed to be normal in MS
patients, our study revealed significant differences on acquisition. Furthermore, these
differences were of moderate effect size. Our study supports recent findings that
suggest that deficient acquisition contributes to the delayed recall impairment often
seen in MS memory research (DeLuca, et al., 1994; DeLuca, et al., 1998; Diamond, et
al., 1997; Minden, et al., 1990). While the MS group had significantly lower scores
and were not learning as well as their healthy counterparts as shown in Table 3.2(B),
technically, they are not impaired relative to the CVLT normative sample. The MS
patients in our sample scored less than one standard deviation below the mean, which
classifies them in the clinically low average range. Studies using the modified
version of the SRT (DeLuca, et al., 1998, Demaree, et al., 2000; Guadino, et al.,
2001) can only report that MS patients do not learn as well as controls, not that they
are impaired normatively, because the modified SRT does not have normative data.
Our data help us understand impaired learning in a larger normative context.
Our study also revealed significant differences on delayed free recall
performance, with the MS group scoring lower than the HC group. Moreover, the
effect size for recall is larger than the effect size for acquisition. However, the MS
39


patients arent impaired on recall either, relative to the CVLT normative sample.
Their average performance was still less than one standard deviation below the mean
and in the low average range. In addition, recognition performance does not
significantly differ between groups, and the average performance of the MS group
should again be classified in the low average range normatively. It is not unusual
for such non-significant findings to be equated with normal performance; however,
the corresponding small-to-moderate effect size indicates that recognition
performance among the MS patients is less than ideal. Moreover, it supports the
hypothesis that learning is adversely affected in MS. In fact, the premise for a
retrieval-based account of memory impairment in MS is based largely on findings of
significant differences between MS patients and healthy controls on recall but not on
recognition (Thornton & Raz, 1997). Upon more comprehensive analysis, we see
that these differences are of notable magnitude if not statistical significance.
In sum, our data argue against a simple retrieval deficit explanation for
memory impairment in MS. Rather, they suggest that MS patients demonstrate
diminished free recall, but that they also exhibit reduced ability to learn verbal
information. Overall, the MS patients performed poorly compared to the HC group
across the three memory domains, with the largest effect observed on free recall with
a moderate-to-large effect on acquisition. The small-to-moderate effects on
recognition were unexpected and provide further support for the acquisition
hypothesis.
40


When the raw scores for both groups were standardized using the normative
data from the CVLT-II as shown in Table 3.4(B), the overall pattern of results in
Study 2 paralleled those established using the CVLT norms in Study 1. Our study
again revealed significant differences of moderate-to-large effect size on acquisition;
and while the MS patients on average scored less than half of a standard deviation
below the HC group, their scores were in the average range compared to the
normative sample. Free recall performance was also significantly different between
the two groups and the effect size was large. The MS patients scored nearly one
standard deviation lower than the controls but were still within the normal range
compared to the normative group. Again, there was no significant difference between
the two groups on recognition although there was a moderate effect.
Recognition on the CYLT-II was assessed differently than on the original
CVLT. The revised test made the recognition task more difficult by including
additional distracter words to minimize ceiling effects. The large effect size
differences on discriminability between the two versions, as shown in Tables 3.2(A)
and 3.4(A), reflect the increased difficulty on the CVLT-II and a proclivity toward
identifying more false positives. When the groups were alternatively normed to the
CVLT and the CVLT-II however, the overall recognition hits and the corresponding
effect sizes were comparable.
Another important psychometric distinction between the CVLT and CVLT-II
are their very different normative samples. The original CVLT was criticized for its
41


normative sample of only 273 subjects which over-represented individuals with
higher levels of education. As a result of this highly educated, small normative
sample, the test had a tendency to underestimate performance and overestimate
pathology in individuals who were not college-educated. To correct for this, the
CVLT-II was normed on a sample of 1,087 subjects and education levels were similar
to that in the general U.S. population. As a result of these changes, a raw score that
would have been classified low average on the CVLT could be considered
average on the CVLT-II. Individuals whose standard scores are normed to the
CVLT appear to perform poorly compared to those same individuals presentations
when normed to the CVLT-II. This effect is likely to be exaggerated if the individual
also has 12 or less years of education. For example, we would clinically interpret
performance on trials 1-5 total in Study 1 (M = 40.9; low average) very differently
than those same scores normed to the CVLT-II in Study 2 (M= 52.9; average), if we
do not consider the normative samples on which these standard scores are based. In
the case of our study sample, the MS patients (M = 14.9 years) and the HC group (M
= 15.9 years) had exceptionally high levels of education, even higher than the
normative sample from the original CVLT. This demonstrates quite vividly the
importance of selecting appropriate reference groups.
The meta-analysis was partitioned by subtype since we were interested in
qualitative differences in verbal memory performance that might be subtype-specific.
Studies that looked exclusively at progressive forms of MS (n = 5) revealed the
42


greatest effects on across all memory domains, with large effect sizes on acquisition
and recall and moderate effects on recognition. In contrast, the studies that looked
exclusively at relapsing-remitting patients (n = 3) found the same pattern but with
much smaller effects. While it appears that verbal memory deficits are more
pronounced in the CP group than in the RR group, the majority of studies report data
using mixed subtype groups (n = 21) with no breakdown by subtype.
In the relapsing-remitting group, the pattern of results found in our meta-
analysis are different that the results in our Studies 1 and 2. In the meta-analysis,
acquisition had the greatest effect size followed by recall compared to the reverse
findings in Studies 1 and 2 where recall showed the largest effect. One possible
explanation for the different outcomes may be because there were only three studies
in the relapsing-remitting analyses. Of these, only one study reported data on all
three memory constructs (Ling & Selby, 1998). The results reported in this study
paralleled the effect size pattern of the meta-analysis; however, this study had
considerably more power given the size of their MS patient group (n = 57), than the
patient groups in the other two studies. The second article by Fisk and Archibald
(2001) reported learning and recall data, but their MS sample was small (n = 20) and
there was very little difference between the two effect sizes. As a result, their
findings had little impact on the results reported by Ling and Selby. The third study
by Pozzilli and colleagues (1991) reported a very large effect size on recall but
contributed no data to the overall learning and recognition effects, therefore it is
43


unknown how their patients performance may have impacted the mean effect size
distributions. For example, if the learning effect size had been small-to-moderate, it
would have dramatically altered the overall effect size pattern. This lack of data
demonstrates the importance of including sufficient descriptive statistics in published
reports regardless of statistical significance as suggested by the APA (American
Psychological Association, 2001).
Age may also have played a role in the effect size differences observed
between our Studies 1 and 2 and the meta-analysis. The patient group in the Ling and
Selby study (M = 49.0) was an average of 10 years older than the patients in Fisks
study (M = 39.5) and 18 years older than Pozzillis patients. The patient group in this
study was also considerably older than their control group (M = 23.0), which may
have impacted the magnitude of the memory effect sizes. In our sample, patients had
a mean age of 41.0 and were matched to the controls on age.
To further examine the influence of a relapsing-remitting disease course on
memory function, we reanalyzed the data by including the results from our Study 1.
The overall pattern of results remained the same as in the initial meta-analysis as
acquisition demonstrated the largest effect, followed by delayed free recall that had a
moderate effect, and recognition had a small-to-moderate effect. While the pattern
did not change the magnitude difference between acquisition and recall was reduced
from 0.32 in the initial analysis to a difference of 0.20 in the reanalysis. As more data
44


become available on purely relapsing-remitting samples, it will be interesting to see if
this pattern of memory deficits persists or if a different pattern emerges.
There were 21 studies that used mixed subtype groups or did not report data
separately. Once again, the pattern of results was the same and fell between those
observed in the RR and CP groups. These between group distribution patterns may
be attributable to subtype differences or simply a product of the small sample of RR
and CP studies, but the tendency to report mixed group results makes this question
impossible to resolve. Future studies can avoid such ambiguity by reporting results
according to disease subtype.
Our meta-analysis took a different approach than those published previously.
First, we limited our analyses to only those studies that utilized list-learning measures
of verbal memory. While other kinds of verbal memory tests (e.g., story memory,
paired associates) provide valuable information, these tests tap a variety of encoding
strategies and levels of processing that influence learning (Atkinson & Shiffrin, 1968;
Craik & Tulving, 1975). We limited the studies in our meta-analysis to a single study
type in an effort to minimize such influences. Second, we systematically examined
each of the variables that support both the acquisition and retrieval hypotheses:
acquisition, free recall, and recognition, so that we would have empirical support to
demonstrate our findings. Finally, data for the meta-analysis was coded according to
subtype to explore for patterns associated with disease course. As discussed earlier,
this resulted in a small number of studies and revealed patterns similar to that seen in
45


the mixed group, but seemed to indicate greater deficits across all memory constructs
in the progressive-type group.
Since all of the analyses up to this point demonstrated distinct deficits in
verbal learning, we performed post-hoc analyses in an attempt to identify differences
in the learning characteristics utilized by the groups. We found no significant
differences between the MS and HC groups on semantic or serial clustering, and a
small, significant difference on consistency across learning trials. Upon closer
examination, the ATRC and semantic clustering variables had small-to-moderate
effect sizes, which hinted at a possible underlying effect.
When we looked only at the MS patients and compared the better and worse
recallers, we found distinct differences on how these individuals were learning. First,
those with lower recall scores failed to use semantic clustering as efficiently as
patients with better recall. Oftentimes, individuals who do not use a semantic
clustering strategy will resort to using serial clustering as a means of memorizing a
word-list. While our MS group did not differ significantly from the control group,
there was a small-to-moderate effect size indicating that the patient group relied on
this strategy more than the controls. The serial clustering alone did not demonstrate a
sufficient size effect to account for the diminished use of semantic clustering. The
worse recaller group also recalled words inconsistently across the learning trials,
which may indicate that they either have difficulty formulating a memorization plan
or they abandon one strategy for another during the course of the test. These findings
46


have implications for clinicians working in rehabilitation settings who may find this
information useful to help formulate learning strategies for patients coping with
memory difficulties.
Strengths and Limitations
There are a few limitations that should be considered when interpreting these
results. First, neuroimaging data were not included. While neuroimaging data were
available for most of our patients in some form, individuals were recruited from a
variety of sources and scans were not performed in a uniform manner. If we could
have included data that correlated white matter lesions with verbal performance, it
would have added an interesting dimension to our study. It would be useful to
observe whether the MS patients who were poor learners had more lesions or if
lesions predominated in a particular brain region. Recent studies have indicated
lesion loads, cortical atrophy, third ventricle enlargement and olfactory dysfunction
can be observed using magnetic resonance imaging (Benedict, et al., 2002; Comi, et
al., 1995; Doty, Li, Mannon & Yousem, 1998; Fulton, et al., 1999; Rao, 1995).
Future research would benefit from both structural and functional neuroimaging data.
It is also important to reiterate that our study examined only relapsing-
remitting patients with mild cognitive impairment and these results should be
generalized accordingly. Our results do, however, tell us something important about
the cognitive effects of white matter involvement and further demonstrate that these
47


patients exhibit cognitive dysfunction. These data may be providing us with a
glimpse into the emergence of memory impairment that can then be studied
longitudinally to determine how memory deteriorates over time. The meta-analysis
also speaks to the importance of taking disease subtype into account in future studies.
Finally, the verbal memory of these patients was only evaluated in terms of
their list-learning performance. Other types of verbal memory tasks such as story
memory may produce different patterns of results by tapping different executive
functions and, by extension, different white matter tracts. Analyses that compare and
contrast memory performance on different types of verbal and non-verbal tasks is
another area for future research.
Recent research has shown that immunomodulatory medications can
significantly reduce cognitive decline in MS patients within two years of treatment
(Fischer, et al., 2000; Pliskin, et al., 1996). While this is a potential confound in
studies that do not report the medication regimens of their patients, it is unlikely to
have affected the neuropsychological test performance in our MS sample. Most of
our patients had either never taken immunomodulatory medications, and those that
had taken them had done so for less than a year.
48


Summary
The primary purpose of this study was to determine whether memory deficits
observed in MS could be attributed to acquisition or retrieval deficits. While there
has been considerable debate in neuropsychology about the specific nature of
memory impairment in MS, our results demonstrate that difficulties in both
acquisition and retrieval contribute to the memory deficits in multiple sclerosis.
49


APPENDIX
Effect Sizes by Individual Studies
Study Ne Nc Learning Recall Recog.
RelaDse-Remittine Patients Onlv:
Fisk, J. D. & Archibald, C. J. (2001) 20 35 0.31 0.22 n.r.
Ling, N. D., & Selby, M. J. (1998) 57 132 0.95 0.44 0.40
Mitchell, S. M. (2003)a 53 31 0.64 0.83 0.43
Pozzilli, C, et al. (1991) 17 17 n.r. 1.04 n.r.
Overall RR Effect 147 215 0.75 0.55 0.41
Progressive Patients Onlv:
Fisk, J. D. & Archibald, C. J. (2001) 15 35 0.92 1.09 n.r.
Ling, N. D., & Selby, M. J. (1998) 47 132 1.19 0.59 0.46
Nocentini, U., et al. (2001) 69 25 0.53 0.72 n.r.
Rao, S. M., et al. (1984) 35 15 0.84 0.68 0.70
Wallace, G. L., & Holmes, S. (1993) 4 4 n.r. 0.57 0.85
Overall CP Effect 170 211 0.93 0.70 0.53
50


Effect Sizes by Individual Studies (Cont.)
Study Ne Nc Learning Recall Recog.
Mixed Subtvne Groups:
Armstrong, C., et al. (1996) 67 22 0.70 0.53 0.44
Beatty, W. W., et al. (1995) 109 32 1.04 1.01 n.r.
Caine, E. D., et al. (1986) 30 15 1.04 1.10 0.61
Coolidge, F.L., et al. (1996) 30 30 0.76 0.64 0.04
DeLuca, J., et al. (1995) 12 20 0.64 0.36 0.53
Diamond, B. J., et al. (1997) 15 13 1.40 0.39 0.77
Iwaski, Y., et al. (1989) 24 16 1.01 0.83 n.r.
Jennekens-Schinkel, A., et al. (1990) 39 24 0.86 0.80 0.40
Krupp, L. B., & Elkins, L. E. (2000) 45 14 0.28 0.94 n.r.
Krupp, L. B., et al. (1994) 20 20 0.63 0.40 n.r.
Kujula, P., et al. (1996) 45 35 1.15 n.r. n.r.
Landro, N.I., et al (2000) 26 24 0.36 0.17 n.r.
Litvan, I., et al. (1988) 16 16 1.98 1.15 n.r.
Maurelli, M.,etal. (1992) 34 18 1.04 1.24 n.r.
Minden, S. L. (1990) 50 35 0.87 0.38 0.71
Rao, S. M., et al. (1991) 100 100 0.70 n.r. n.r.
Rao, S.M., et al. (1989) 37 26 0.85 0.92 0.07
51


Effect Sizes by Individual Studies (Cont.)
Study Ne Nc Learning Recall Recog.
Scarrabelotti, M., & Carrol, M. (1998) 50 41 0.51 n.r. 0.32
Sperling, R. A., et al. (2001) 28 28 0.86 0.98 n.r.
Thornton, A.E., et al. (2002) 49 49 0.72 0.56 0.37
van den Burg, W., et al. (1987) 40 40 0.53 0.37 0.47
Overall Mixed Subtype Effect 644 556 0.79 0.68 0.40
Note: Ne = total multiple sclerosis patients, Nc = total healthy controls, Learning =
leaming/acquisition, Recall = delayed free recall, Recog. = recognition, n.r. = data not
reported, a = Included in reanalysis only
52


REFERENCES
References marked with an asterisk indicate studies included in the meta-analysis.
Amato, M. A., Ponziani, G., Pracucci, G., Bracco, L., Siracusa, G., & Amaducci, L.
(1995). Cognitive impairment in early-onset multiple sclerosis: Pattern,
predictors, and impact on everyday life in a 4-year follow-up. Archives of
Neurology, 52,168-172.
American Psychological Association. (2001). Publication manual of the American
Psychological Association (5th ed.). Washington, DC: Author.
*Armstrong, C., Onishi, K., Robinson, K., D'Esposito, M., Thompson, H., Rostami,
A., et al. (1996). Serial position and temporal cue effects in multiple sclerosis:
two subtypes of defective memory mechanisms. Neuropsychologia, 34, 853-
862.
Atkinson, R. C., & Shiffrin, R. M. (1968). Human memory: A proposed system and
its control processes. In K. W. Spence & J. T. Spence (Eds.), The psychology
of learning and motivation (Vol. 2, pp. 89-195). Orlando, FL: Academic
Press.
Beatty, W. W., Goodkin, D. E., Monson, N., Beatty, P. A., & Hertsgaard, D. (1988).
Anterograde and retrograde amnesia in patients with chronic progressive
multiple sclerosis. Archives of Neurology, 45, 611-619.
Beatty, W. W., Krull, K. R., Wilbanks, S. L., Blanco, C. R., Hames, K. A., & Paul, R.
H. (1996). Further validation of constructs from the selective reminding test.
Journal of Clinical and Experimental Neuropsychology, 18, 52-55.
*Beatty, W. W., Paul, R. H., Blanco, C. R., Hames, K. A., & Wilbanks, S. L. (1995).
Attention in multiple sclerosis: Correlates of impairment on the WAIS-R Digit
Span Test. Applied Neuropsychology, 2, 139-144.
53


Beatty, W. W., Paul, R. H., Wilbanks, S. L., Hames, K. A., Blanco, C. R., &
Goodkin, D. E. (1995). Identifying multiple sclerosis patients with mild or
global cognitive impairment using the Screening Examination for Cognitive
Impairment (SEFCI). Neurology, 45, 718-723.
Beatty, W. W., Wilbanks, S. L-, Blanco, C. R., Hames, K. A., Tivis, R., & Paul, R. H.
(1996). Memory disturbance in multiple sclerosis: Reconsideration of patterns
of performance on the selective reminding test. Journal of Clinical and
Experimental Neuropsychology, 18, 56-62.
Beck, A. T., & Steer, R. A. (1987). Manual for the revised Beck Depression
Inventory. San Antonio, TX: The Psychological Corporation.
Benedict, R. H., Bakshi, R., Simon, J. H., Priore, R., Miller, C., & Munschauer, F.
(2002). Frontal cortex atrophy predicts cognitive impairment in multiple
sclerosis. Journal of Neuropsychiatry and Clinical Neuroscience, 14,44-51.
Buschke, H., & Fuld, P. A. (1974). Evaluating storage, retention, and retrieval in
disordered memory and learning. Neurology, 24,1019-1025.
*Caine, E. D., Bamford, K. A., Schiffer, R. B., Shoulson, I., & Levy, S. (1986). A
controlled neuropsychological comparison of Huntington's disease and
multiple sclerosis. Archives of Neurology, 43,249-254.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.).
Hillsdale, NJ: Erlbaum.
Colorado HealthSite (2003). MS definitions, facts, and statistics. Retrieved 2/8/03,
from http://www.coloradohealthsite.org/ms/ms_stats.html
Comi, G., Filippi, M., Martinelli, V., Campi, A., Rodegher, M., Alberoni, M., et al.
(1995). Brain MRI correlates of cognitive impairment in primary and
secondary progressive multiple sclerosis. Journal of the Neurological
Sciences, 132,222-227.
54


*Coolidge, F. L., Middleton, P. A., & Griego, J. A. (1996). The effects of interference
on verbal learning in multiple sclerosis. Archives of Clinical
Neuropsychology, 11, 605-611.
Cooper, H., & Hedges, L. V. (1994). The handbook of research synthesis. New York.
Craik, F. I. M., & Tulving, E. (1975). Depth of processing and the retention of words
in episodic memory. Journal of Experimental Psychology: General, 104,268-
294.
Delis, D. C., Kramer, J. H., Kaplan, E., & Ober, B. A. (1987). Manual for the
California Verbal Learning Test Adult Version. San Antonio, TX: The
Psychological Corporation.
Delis, D. C., Kramer, J. H., Kaplan, E., & Ober, B. A. (2000). Manual for the revised
California Verbal Learning Test Adult Version (2nd ed.). San Antonio, TX:
The Psychological Corporation.
DeLuca, J., Barbieri-Berger, S., & Johnson, S. K. (1994). The nature of memory
impairments in multiple sclerosis: Acquisition versus retrieval. Journal of
Clinical and Experimental Neuropsychology, 16,183-189.
DeLuca, J., Gaudino, E. A., Diamond, B. J., Christodoulou, C., & Engel, R. A.
(1998). Acquisition and storage deficits in multiple sclerosis. Journal of
Clinical and Experimental Neuropsychology, 20, 376-390.
*DeLuca, J., Johnson, S. K., Beldowicz, D., & Natelson, B. H. (1995).
Neuropsychological impairments in chronic fatigue syndrome, multiple
sclerosis, and depression. Journal of Neurology, Neurosurgery and
Psychiatry, 58, 38-43.
Demaree, H. A., Gaudino, E. A., DeLuca, J., & Ricker, J. H. (2000). Learning
impairment is associated with recall ability in multiple sclerosis. Journal of
Clinical and Experimental Neuropsychology, 22, 865-873.
55


*Diamond, B. J., DeLuca, J., Johnson, S. K., & Kelley, S. M. (1997). Verbal learning
in amnesic anterior communicating artery aneurysm patients and in patients
with multiple sclerosis. Applied Neuropsychology, 4, 89-98.
Doty, R. L., Li, C., Mannon, L. J., & Yousem, D. M. (1998). Olfactory dysfunction
in multiple sclerosis. Relation to plaque load in inferior frontal and temporal
lobes. Annals of the New York Academy of Sciences, 855,781-786.
Fischer, J. S., Priore, R. L., Jacobs, L. D., Cookfair, D. L., Rudick, R. A., Herndon, R.
M., et al. (2000). Neuropsychological effects of interferon beta-la in
relapsing multiple sclerosis. Multiple Sclerosis Collaborative Research
Group. Annals of Neurology, 48, 885-892.
*Fisk, J. D., & Archibald, C. J. (2001). Limitations of the Paced Auditory Serial
Addition Test as a measure of working memory in patients with multiple
sclerosis. Journal of the International Neuropsychological Society, 7, 363-
372.
Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975). "Mini-Mental State": A
practical method of grading the cognitive state of patients for the clinician.
Journal of Psychiatric Research, 12,189-198.
Fulton, J. C., Grossman, R. I., Udupa, J., Mannon, L. J., Grossman, M., Wei, L., et al.
(1999). MR lesion load and cognitive function in patients with relapsing-
remitting multiple sclerosis. American Journal of Neuroradiology, 20,1951-
1955.
Gaudino, E. A., Chiaravalloti, N. D., DeLuca, J., & Diamond, B. J. (2001). A
comparison of memory performance in relapsing-remitting, primary
progressive and secondary progressive, multiple sclerosis. Neuropsychiatry,
Neuropsychology and Behavioral Neurology, 14, 32-44.
Glass, G. V., McGaw, B., & Smith, M. L. (1981). Meta-analysis in social research.
Beverly Hills, CA: Sage Publications.
56


Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. San Diego,
CA: Academic Press, Inc.
*Iwasaki, Y., Kinoshita, M., Ikeda, K., & Takamiya, K. (1989). Cognitive function in
multiple sclerosis and its relation to functional abilities. International Journal
of Neuroscience, 48,219-223.
*Jennekens-Schinkel, A., van der Velde, E. A., Sanders, E. A., & Lanser, J. B.
(1990). Memory and learning in outpatients with quiescent multiple sclerosis.
Journal of the Neurological Sciences, 95, 311-325.
*Krupp, L. B., & Elkins, L. E. (2000). Fatigue and declines in cognitive functioning
in multiple sclerosis. Neurology, 55,934-939.
*Krupp, L. B., Sliwinski, M., Masur, D. M., Friedberg, F., & Coyle, P. K. (1994).
Cognitive functioning and depression in patients with chronic fatigue
syndrome and multiple sclerosis. Archives of Neurology, 51,705-710.
*Kujala, P., Portin, R., & Ruutiainen, J. (1996). Memory deficits and early cognitive
deterioration in MS. Acta Neurologica Scandinavica, 93, 329-335.
Laatu, S., Hamalainen, P., Revonsuo, A., Portin, R., & Ruutiainen, J. (1999).
Semantic memory deficit in multiple sclerosis; impaired understanding of
conceptual meanings. Journal of Neurological Sciences, 162, 152-161.
*Landro, N. I., Sletvold, H., & Celius, E. G. (2000). Memory functioning and
emotional changes in early phase multiple sclerosis. Archives of Clinical
Neuropsychology, 15,37-46.
*Ling, N. D., & Selby, M. J. (1998). Assessment of memory in multiple sclerosis
patients using the Memory Assessment Scale. Perceptual and Motor Skills,
86, 987-998.
Lipsey, M. W., & Wilson, D. B. (2000). Practical meta-analysis. Thousand Oaks,
CA: Sage Publications.
57


*Litvan, I., Grafinan, J., Vendrell, P., & Martinez, J. M. (1988). Slowed information
processing in multiple sclerosis. Archives of Neurology, 45,281-285.
*Maurelli, M., Marchioni, E., Cerretano, R., Bosone, D., Bergamaschi, R., Citterio,
A., et al. (1992). Neuropsychological assessment in MS: Clinical,
neurophysiological and neuroradiological relationships. Acta Neurologica
Scandinavica, 86,124-128.
*Minden, S. L., Moes, E. J., Orav, J., Kaplan, E., & Reich, P. (1990). Memory
impairment in multiple sclerosis. Journal of Clinical and Experimental
Neuropsychology, 12, 566-586.
Multiple Sclerosis Association of America. (2002). Multiple sclerosis: Managing
symptoms (3rd ed.). Cherry Hill, NJ.
National Multiple Sclerosis Society (2001, 04/02). Just the facts 2001-2002:
Frequently asked questions about multiple sclerosis and the National MS
Society. Retrieved 2/10/03, from
http://www.nationalmssociety.org/pdfrbrochures/JustFacts.pdf
National Multiple Sclerosis Society (2003). The National MS Society information
sourcebook. Retrieved 2/8/2003,2003, from
http ://www.nationalmssociety .org/sourcebook. asp
*Nocentini, U., Rossini, P. M., Carlesimo, G. A., Graceffa, A., Grasso, M. G., Lupoi,
D., et al. (2001). Patterns of cognitive impairment in secondary progressive
stable phase of multiple sclerosis: correlations with MRI findings. European
Neurology, 45,11-18.
Onwuegbuzie, A. J., & Leech, N. L. (2003). Meta-analysis research: Cautions and
limitations. Unpublished manuscript, University of Colorado at Denver.
Orwin, R. G. (1983). A fail-safe N for effect size. Journal of Educational Statistics, 8,
157-159.
58


Poser, C. M., Paty, D. W., Scheinberg, L., McDonald, W. I., Davis, F. A., Ebers, G.
C., et al. (1983). New diagnostic criteria for multiple sclerosis: Guidelines for
research protocols. Annals of Neurology, 13,227-231.
*Pozzilli, C., Passafiume, D., Bemardi, S., Pantano, P., Incoccia, C., Bastianello, S.,
et al. (1991). SPECT, MRI and cognitive functions in multiple sclerosis.
Journal of Neurology, Neurosurgery and Psychiatry, 54,110-115.
Rao, S. M. (1995). Neuropsychology of multiple sclerosis. Current Opinion in
Neurology, 8, 216-220.
Rao, S. M., Grafinan, J., DiGiulio, D., Mittenberg, W., Bemardin, L., Leo, G. J., et al.
(1993). Memory dysfunction in multiple sclerosis: Its relation to working
memory, semantic encoding, and implicit learning. Neuropsychology, 7, 364-
374.
*Rao, S. M., Hammeke, T. A., McQuillen, M. P., Khatri, B. O., & Lloyd, D. (1984).
Memory disturbance in chronic progressive multiple sclerosis. Archives of
Neurology, 41, 625-631.
*Rao, S. M., Leo, G. J., Bemardin, L., & Unverzagt, F. (1991). Cognitive dysfunction
in multiple sclerosis. I. Frequency, patterns, and prediction. Neurology, 41,
685-691.
*Rao, S. M., Leo, G. J., & St Aubin-Faubert, P. (1989). On the nature of memory
disturbance in multiple sclerosis. Journal of Clinical and Experimental
Neuropsychology, 11, 699-112.
Rey, A. (1964). L'examen clinique enpsychologie. Paris: Presses Universitaires de
France.
Rocky Mountain MS Center (2002). Facts about MS. Retrieved 2/8/03, from
http://mscenter.securesites.com/msfacts.htm
59


*Scarrabelotti, M:, & Carroll, M. (1998). Awareness of remembering achieved
through automatic and conscious processes in multiple sclerosis. Brain and
Cognition, 38,183-201.
Scarrabelotti, M., & Carroll, M. (1999). Memory dissociation and metamemory in
multiple sclerosis. Neuropsychologia, 37,1335-1350.
Schmidt, D., Krause, B. J., Mottaghy, F. M., Halsband, U., Herzog, H., Tellmann, L.,
et al. (2002). Brain systems engaged in encoding and retrieval of word-pair
associates independent of their imagery content or presentation modalities.
Neuropsychologia, 40,457-470.
*Sperling, R. A., Guttmann, C. R., Hohol, M. J., Warfield, S. K., Jakab, M., Parente,
M., et al. (2001). Regional magnetic resonance imaging lesion burden and
cognitive function in multiple sclerosis: A longitudinal study. Archives of
Neurology, 58,115-121.
Thornton, A. E., & Raz, N. (1997). Memory impairment in multiple sclerosis: A
quantitative review. Neuropsychology, 11,357-366.
*Thomton, A. E., Raz, N., & Tucker, K. A. (2002). Memory in multiple sclerosis:
contextual encoding deficits. Journal of the International Neuropsychological
Society, 8,395-409.
*van den Burg, W., van Zomeren, A. H., Minderhoud, J. M., Prange, A. J., & Meijer,
N. S. (1987). Cognitive impairment in patients with multiple sclerosis and
mild physical disability. Archives of Neurology, 44,494-501.
*Wallace, G. L., & Holmes, S. (1993). Cognitive-linguistic assessment of individuals
with multiple sclerosis. Archives of Physical Medicine and Rehabilitation, 74,
637-643.
Wishart, H., & Sharpe, D. (1997). Neuropsychological aspects of multiple sclerosis:
A quantitative review. Journal of Clinical and Experimental
Neuropsychology, 19, 810-824.
60


Wolf, F. M. (1986). Meta-analysis: Quantitative methods for research synthesis.
Newbury, London: Sage.
Zakzanis, K. K. (2000). Distinct neurocognitive profiles in multiple sclerosis
subtypes. Archives of Clinical Neuropsychology, 15,115-136.
Zakzanis, K. K. (2001). Statistics to tell the truth, the whole truth, and nothing but the
truth: Formulae, illustrative numerical examples, and heuristic interpretation
of effect size analyses for neuropsychological researchers. Archives of
Clinical Neuropsychology, 16, 653-667.
Zakzanis, K. K., Leach, L., & Kaplan, E. (1999). Multiple sclerosis. In
Neuropsychological differential diagnosis (pp. 109-125). Lisse: Swets &
Zeitlinger.
61