Citation
Readers' preference for process graphics and outcome graphics accompanying equipment assembly intructions

Material Information

Title:
Readers' preference for process graphics and outcome graphics accompanying equipment assembly intructions
Creator:
Sharp, D. Michael
Publication Date:
Language:
English
Physical Description:
56 leaves : illustrations ; 28 cm

Subjects

Subjects / Keywords:
Graphic arts ( lcsh )
Installation of equipment ( lcsh )
Consumers -- Attitudes ( lcsh )
Consumers -- Attitudes ( fast )
Graphic arts ( fast )
Installation of equipment ( fast )
Genre:
bibliography ( marcgt )
theses ( marcgt )
non-fiction ( marcgt )

Notes

Bibliography:
Includes bibliographical references (leaves 54-56).
General Note:
Department of English
Statement of Responsibility:
by D. Michael Sharp.

Record Information

Source Institution:
|University of Colorado Denver
Holding Location:
|Auraria Library
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
49641786 ( OCLC )
ocm49641786
Classification:
LD1190.L542 2001m .S42 ( lcc )

Full Text
READERS PREFERENCE FOR
PROCESS GRAPHICS AND OUTCOME GRAPHICS
ACCOMPANYING EQUIPMENT ASSEMBLY INSTRUCTIONS
by
D. Michael Shaip
B.A., St. Marys College of Maryland, 1985
A thesis submitted to the
University of Colorado at Denver
in partial fulfillment
of the requirements for the degree of
Master of Science
Technical Communication
2001


This thesis for the Master of Science
degree by
D. Michael Sharp
has been approved
by
Sonja Foss
Filipp Sapfenza
/hr. 11 j Otio/
Date


Sharp, D. Michael (M.S., Technical Communication)
Readers Preference for Process Graphics and Outcome Graphics
Accompanying Equipment Assembly Instructions
Thesis directed by Professor James F. Stratman
ABSTRACT
With this study I propose that graphics that accompany procedural instructions divide
into two categories, namely outcome graphics and process graphics. By using a think-
aloud protocol, I ask the following questions:
1. Can readers tell the difference between graphical types?
2. Do readers prefer one graphical type over another if they are given a choice?
With this study I was also able to collect information about what readers want and do
not want when they read instructions (n=25). From think-aloud data, 72% of subjects
indicated they recognized a difference between outcome graphics and process
graphics. Eighty percent of subjects preferred process graphics to outcome graphics.
With study I could also examine subjects vocalized reasons for their graphical
preferences, as well as examine reading behaviors and vocalized meta-reading
information. Finally, I suggest improvements in the studys method, as well as other
ways to continue research that involves outcome graphics and process graphics.
This abstract accurately represents the content of the candidates thesis. I recommend
its publication.
in


DEDICATION
I want to thank my folks for their encouragement while I've been working
on this degree. Your support always means a lot.
I also want to thank my grandmother. Maybe you were right.


ACKNOWLEDGEMENT
I'd like to thank my advisor, James Stratman, for his steady guidance
during this thesis. You've been extremely helpful in keeping me focused on
the forest.


CONTENTS
Figures ............................................................... ix
CHAPTER
1. INTRODUCTION....................................................1
Definitions.....................................................2
Graphic......................................................2
Process Graphic..............................................2
Outcome Graphic..............................................4
Action Element ..............................................5
Literature Search ..............................................6
A Case for Graphics..........................................8
A Case for Process Graphics..................................9
A Case for Outcome Graphics..................................12
Research Questions .............................................14
Scope ..........................................................14
Hypotheses......................................................14
Hypothesis 1.................................................15
Hypothesis 2.................................................15
Hypothesis 3.................................................15
vi


2. METHOD
17
Device Used........................................................18
Description of Experimental Conditions.............................18
Subjects ..........................................................23
Experimental Design ...............................................23
Procedure..........................................................24
Decision-Making Time ..............................................25
Post-Decision Time.................................................26
Post-Decision Conversation......................................26
Debriefing .....................................................27
3. RESULTS ...........................................................28
Recognizing Differences Between Graphical Types....................28
Graphical Preference ..............................................29
Reasons for Preferences.........................................31
Decision Changes................................................33
Reader Desires.....................................................38
Qualities Readers Said They Like................................38
Qualities Readers Said They Dislike.............................38
Reading Behavior...................................................39
Meta-Reading Comments ..........................................40
4. CONCLUSION ........................................................43
vii


Implications....................................................43
Improvements ......................................................47
APPENDICES
A: Flyer used to announce study.................................49
B: Study description read to and given to subjects, to set context.50
C: Consent form that subjects would read, fill and sign ...........51
D: Script read to subjects......................................52
E: Stop script read to subjects after they had given choice and ranking ... 53
BIBLIOGRAPHY ...................................................54
viii


FIGURES
Figure
1-1 ..............................................................................3
1-2 ..............................................................................3
1-3 ..............................................................................4
1-4 ..............................................................................5
1-5 ..............................................................................6
1-6 ..............................................................................10
1-7 ................................................................................10
1-8 ..............................................................................11
1-9 ................................................................................13
1- 10 .............................................................................13
2- 1 ............................................................................18
2-2
Page from manual with process graphics..............................................20
2-3
Page from manual with outcome graphics..............................................21
2- 4
Page from manual with both process and outcome graphics.............................22
3- 1 ..............................................................................29
IX


3-2
3-3
3-4
3-5
3-6
3-7
30
31
33
34
40
42
x


CHAPTER 1
INTRODUCTION
When I worked as an information designer for a company that made
superconductive telephony devices, an engineer asked me to change the
graphics I used with my installation manuals. Her concern arose because we
were shipping devices to non-English speaking countries, and she wanted to
help ensure that the devices were installed correctly. Whereas my graphics
were photographs that showed the results or outcomes of following the
installation steps correctly, she wanted to give emphasis to the actions that the
installer performed. She believed such graphics would be more helpful that
way, especially when installers might not be able to read the instructions. Her
request seemed reasonable and came from experience she had gained while
installing the devices overseas, so I obliged her.
I worried about the effect of this change, however, for a couple reasons.
First, this graphical format removed the visual outcomes of the steps, the only
way installers could compare their work to correct examples. Second, because
page space and picture size remain the same, including the installer or
installer's hands in the graphics required that fewer graphics could be close-
ups, thereby possibly losing important information.
From working with this experience, I coined my own terms for these
graphical types: process graphics and outcome graphics. And I began to observe
that the graphics accompanying procedural task instructions may be process
graphics or outcome graphics and sometimes a combination of both types. By
asking other information designers how they chose their graphics, I learned
1


that often the choice of graphical type seemed to be a matter of personal
preference, perhaps aesthetic. I became curious about whether one graphical
type was more helpful than the other, so I began to search for relevant studies
that compared and contrasted process and outcome graphics.
Definitions
Before continuing with this paper, it is important to define clearly some
terms that will appear frequently.
Graphic
Graphic, in the context of this study, refers to any pictorial image that may
accompany procedural task instructions. A graphic may be of various media,
including line drawings, detailed/shaded drawings, photographs of models,
and realistic photographs, all of which may be rendered in black and white or
color. A graphic is not a quantitative graph, chart or table.
Process Graphic
Process graphic refers to a graphic that depicts a process or action as it is
being performed. If accompanying a set of instructions, a process graphic can
depict the actions that the instructions require, possibly with the use of action
elements (discussed below). A process graphic emphasizes and explicitly
exhibits the activity that a reader will perform. Examples of process graphics
follow on the next page with Figures 1-1 and 1-2.
2


The image to the left shows examples of process
graphics from instructions for a camera. The
graphic has different action elements to indicate
what the reader will do:
An arrow indicates the reader will depress the
rewind button, in the top frame.
A rotational arrow indicates how the reader
will turn the rewind knob, in the middle
frame.
An arrow indicates how the reader will pull
the rewind knob to open the back cover, in the
bottom frame.
Figure 1-1
The graphic to the left is an
example of a process graphic.
A hand is holding a
screwdriver and tightening a
bolt, as described in the
written procedural
instructions that this graphic
accompanied.
Figure 1-2
3


Outcome Graphic
Outcome graphic refers to a graphic that depicts the (desired) outcome of
performing a process or action. If accompanying a set of instructions, an
outcome graphic can depict the result or outcome of following those
instructions. An outcome graphic implies that the reader performed some
action and emphasizes and exhibits the result of the reader's action. Examples
of outcome graphics follow with Figures 1-3 and 1-4.
The graphic to the left is an
example of an outcome
graphic. The written
instruction tells the reader
how to properly connect and
place the battery pack into a
telephone handset.
4


The graphic to the left is an
example of an outcome
graphic. The written
instruction tells the reader to
draw chalk marks on both
sides of fabric; the figure shows
the result.
Action Element
Action element refers to elements of a process graphic that help indicate
what the reader will do, or what action is being performed. For example,
lines may indicate motion or an impact (such as with a mallet),
arrows may indicate direction of a force, and
rotational arrows may indicate which way to turn a screwdriver.
Examples of action elements follow on the next page with Figure 1-5.
5


The graphic to the left
(composed of two figures that
accompanied written
procedural instructions) is an
example of a process graphic.
Among other action elements
that show a process to
perform are the following:
' Lines to indicate swinging a
mallet
' Lines to indicate impact of
the mallet
' Arrows to show directions of
force
Rotational arrows to indicate
which way to turn a
screwdriver
Figure 1-5
Literature Search
To begin searching for literature on what I had termed "outcome" and
"process" graphics, I used the following resources, among others:
University of Colorado at Denver Library
Applied Science Technology Abstracts
Dissertation Abstracts
Education Abstracts
the ERIC database
First Search
Interlibrary Loan
OVID
6


I used various combinations of the following keywords:
graphic
cognit [sic]
instruct
learn
outcome
process
procedur [sic]
task
Each word would lead to variants, for example, "procedur" would also
reach "procedure," "procedures," and "procedural." By sifting through
hundreds of articles that my searches produced, I learned that most of the
results written after the mid-1980s were concerned with animated graphics
and the role they may might play in instruction. This emphasis in the research
coincides with the increase in personal computer (PC) usage. As PCs became
more powerful and better able to render animation, interest in research with
only still graphics seemed to wane.
Categories of research included the following conclusions:
Graphics with text improve recall.
Anglin and Stevens (1986) made such a conclusion when they studied the
effects of text recall (n=42). They gave subjects either text only or text and
graphics treatments. In immediate recall conditions, the text and graphics
condition tested significantly higher than the text only condition.
Pictures alone do not help with retention.
Ausel and Bieger (1989) state such a conclusion after they tested subjects
on an assembly task (n=47), using three sets of instructions: graphics only,
text only, and text and graphics. Initially, subjects who had graphics-only
instructions assembled the product most quickly. However, when subjects
were asked to repeat the assembly one month later without instructions,
those subject who had used the text-only instructions performed the task
most quickly. From this, Ausel and Bieger suggest that the effects from
graphics are not long-lasting. Ellis, Whitehill and Irick (1996) also found
that graphics with instructions reduce assembly time but do not lead to
retention, even as soon as one week later (n=89).
7


There is a need to identify different types of graphics to create taxonomies.
In such a direction, Macdonald-Ross (1989) wants to arrive at a scenario in
which the relations between graphic structure, problem structure and
reader uses are all considered. He also indicates that some communicators
are more effective than others at producing certain types of graphic
information and that it is important to learn ways to integrate these
different communicators within a staff. Working with a broader concept of
graphics, Lohse, Biolsi, Walker and Rueter (1994) develop a classification of
graphics into eleven major divisions, of which "picture" is one.
Graphics are effective, but there is still a need to know why.
Working with perceptual and cognitive processes involved in the
organization, interpretation and understanding of graphics, Winn (1994)
suggests that more must be known about graphics to continue improving
their use in communication and instruction. He raises questions about
whether graphics act consistently with syntactical rules and if it is possible
to develop prescriptions for graphical use.
Text enhances graphics.
The book Comprehension of Graphics in Text (DeCorte, ed.,1993) indicates
that graphics by themselves do not directly affect learning but they must
be processed accurately with text.
Graphics enhance text.
Examples of this conclusion follow.
A Case for Graphics
As the last two bulleted items above suggest, there were studies conducted
to observe how graphics and text might work together. Booher (1975)
conducted a study in which three groups of subjects performed the same tasks
with the aid of instructions that differed by being text-only, graphic-only, or a
text and graphic combination. Subjects who used the text and graphic
combination instructions performed their tasks faster and more accurately
than the other two groups (n=90). His results indicate that graphics facilitate
operational tasks, especially if the graphics convey spatial information about
the location of the objects the subjects would manipulate.
Even quite recent studies show that readers want graphics and prefer
instructions accompanied by graphics to text-only instructions. Fukuoka,
8


Kojima and Spyridakis (1999) conducted a study to compare Japanese and
American reader preferences toward instructions, using four variations of the
same instructional set:
instructions with one graphic per step
instructions with one graphic per every three steps
instructions with one graphic per page
instructions with no graphics (text-only)
Fukuoka, Kojima and Spyridakis concluded that subjects believed that more
graphics, rather than fewer, make instructions easier to follow. Furthermore,
they found that subjects prefer a combination of text and graphics, which they
believed would be more effective than the text-only instructions, which 0%
preferred (n=29).
A Case for Process Graphics
From my literature search, no articles used the term process graphic, but I
did find a few studies that investigated ways to improve graphics that
accompany procedural tasks. In a 1979 article, Szlichcinski referred to "non-
pictorial elements," such as arrows and lines1 (p. 263), and said they are useful
because they help the graphic to "show not just what something looks like, but
how to do something (p. 264)." Szlichcinski said these non-pictorial elements
help the instructions "establish a dialogue with the reader and influence his
actions (p. 264)."
1. It would seem that arrows and lines are rather recent accompaniments to graphic
images. In his article Pictorial Instructions, image historian Ernst Gombrich
indicates that arrows used as pointers and/or indicators of movement are not used
before the 18th century. The earliest example he finds is in a French treatise,
Architecture Hydraulic, from 1737. Jonathan Miller, in his article Moving Pictures,
explains how graphic artists learned to use lines and blurs to indicate movement,
by imitating the streaks that appeared in photographs between the 1840s and
1870s, due to the slow photographic emulsions.
9


Szlichcinski's objective was to find a syntax for pictorial instructions: he
essentially attempted to make sentences by using illustrations and non-
pictorial elements in specific orders. Szlichcinski designed an experiment
(1980) using a production technique, in which he asked people to draw (or
produce) their own graphics to accompany instructions. He first asked
subjects to operate a range of manual controls, such as rocker switches, toggle
switches and rotary dials, to learn how to operate the controls. Next, subjects
were required to draw pictures to explain how to operate those controls. In
this experiment, 96% (n=80) of subjects used arrows to represent actions, and
44% used arrows and fingers or hands, "suggesting that they regarded these
symbols as interchangeable (p. 117)." Among these subjects, the most often
used illustration was a combination of switch and action element, as shown in
Figures 1-6 and 1-7. Szlichcinski stated that this "is an exact analogue of its
written equivalent (p. 122)."
In many of the illustrations, it was possible to determine the state of the
switch. More than 80% of subjects drew the switch in its initial position paired
with an action element, as with Figure 1-6. The remaining subjects drew the
switch in its final position with an action element, as with Figure 1-7. Figures
1-6 and 1-7 align with what I refer to as process graphics and outcome graphics,
respectively. The outcome image and action element combination used in
Figure 1-7 actually can lead to some confusion for readers because the
10


combination requires "that the action be deduced rather than showing it
explicitly (p. 122)." In my division between process graphics and outcome
graphics, action elements only accompany process graphics.
In a second line of research, Szlichcinski (1982) examined subjects'
comprehension of sequences of pictorial instructions pertaining to operating
manual controls similar to the ones used with the 1979 production technique.
The pictorial instructions received various treatments, including arrows only,
hands and arrows, and insets showing a close-up of the control being
operated. In this experiment, Szlichcinski always paired action elements, such
as arrows, with the switches in their original position. Figure 1-8 shows an
example of the treatment that produced the best performance.
11


Szlichcinski's 1982 study results indicated that subjects performed the
tasks best when the instructions offered the following:
information about the control's location
close-up of the control being operated
Szlichcinski concluded that showing detailed information about both
appearance and location of controls was important and that "presentation
style for instructions should not be made simply on aesthetic grounds
(p. 463)." This point urges the importance for information designers to have a
method of choosing what type of graphics to use with their instructions.
A Case for Outcome Graphics
Guthrie et al. (1990) and Burnham (1992) strongly recommend using
outcome graphics with procedural tasks. Specifically, Guthrie et al. (1990)
promote the idea that using outcome graphics enables readers to compare
immediately their results to images of correct outcomes. Readers then can self-
correct if their results do not match the correct outcome images. Two examples
from the appendix of Burnham's paper follow on the next page with Figures
1-9 and 1-10.
12


Figure 1-9
Neither Guthrie nor Burnham suggest, however, a specific alternative to the
outcome graphic, so it is not clear if they hold the outcome graphic to be
superior from among a taxonomy of graphical types.
From these researchers' discussions and promotions of graphics as
instructional aids, I will use this qualitative study to examine whether readers
can tell a difference between process graphics and outcome graphics and whether
readers have a preference for one graphical type over the other. Furthermore,
13


from think-aloud data, I will attempt to learn criteria that readers want and
expect from instructions when they perform a new procedural task. Such
research could establish a useful offshoot of industry-worthy information that
could suggest when and why to use a type of graphic with procedural task
instructions.
Research Questions
The research questions for this study are,
Can readers tell the difference between process graphics and outcome
graphics?
Do readers prefer one graphical type over another if they are given a choice?
The answers to these questions may be important for information designers
when they consider using graphics with equipment assembly instructions. At
a time when technology soon may make it practical to create individualized
instruction, knowing reader's preferences may become an important part of
effective instructional design.
Scope
In this study, I collect and examine qualitative data from readers about
their subjective preference for graphical type. I do not examine whether a
graphical type or a reader's graphical preference actually leads to superior
performance. Such issues require further study.
Hypotheses
In this study, subjects were asked to pick one instruction manual from
among four versions of the same information. One manual was text only. Of
14


the other manuals, one had a process graphic for each step, one had an
outcome graphic for each step, and one had both kinds of graphic for each
step, thereby having two pictures for each step.
Hypothesis 1
Concerning the manual without graphics, given the novelty and
complexity of the task, I predict that no readers will choose to use it. Research
from Fukuoka, Kojima and Spyridakis (1999) supports this prediction, when
they received a 0% preference for text-only information versus text containing
various amounts of graphics (n=29).
Hypothesis 2
Concerning the manual with both types of graphics, due to the novelty of
having an apparent action-result image sequence with the instructions, I
predict that all subjects will give first preference to this manual version. This
prediction also gains some support from Fukuoka, Kojima and Spyridakis'
research, which suggests that readers believe that more graphics, rather than
fewer, make instructions easier to follow (1999).
Hypothesis 3
In Szlichcinski's study (1979), when people were asked to draw pictorial
instructions on how to operate a switch, only 14 out of 80 people (17.5%) drew
a control switch in its final position (outcome). The remaining 66 subjects drew
the switch in its initial position, paired with an action element (to indicate a
process that would lead to the desired result). Based on these results, if
subjects were forced to choose between the manual with process graphics and
15


the manual with outcome graphics, I predict a majority of subjects will prefer
the manual with process graphics.
16


CHAPTER 2
METHOD
Based upon the difficulty of a task, readers may desire less or more
explicitness in the instructions that accompany the task. These subjective,
qualitative data can be measured. If readers are assigned an unfamiliar or
complex task, they will desire more explicit, graphic instruction than they
would for tasks that require easy deduction (Kern, 1985; Smillie, 1985).
By using a task that is very novel to subjects, subjects should desire more
explicit graphical accompaniment. Using a novel task also should help
prevent readers from reviewing graphics superficially, as they may be prone to
do with more common tasks (Weidenmann, 1989,1994; Peeck, 1994).
In this study, I used a benign deception that led subjects to believe they
would build a complex electro-mechanical device for which they would
choose an assembly manual; in fact, subjects did not build the device after they
had chosen a manual to use. I wanted to observe subjects' decision-making
when they believed they would perform the task.
17


Device Used
The hardware for this study consisted of the tools and parts required to
build a complex electro-mechanical device with an electric motor and software
driver. The product is shown below.
Figure 2-1
Description of Experimental Conditions
The experimental conditions for this study were four versions of a manual
with complete, step-by-step instructions that led to a completed product:
a version with no graphics (text-only), 4 pages,
a version with process graphics, 14 pages,
a version with outcome graphics, 14 pages, and
a version with both process and outcome graphics, 14 pages.
The text was the same for all manuals. The first page of all manuals was
the same, consisting of two short paragraphs and a picture showing all the
necessary components for building the device. The first paragraph described a
two-part process necessary to build the device successfully and warned that
18


improper assembly could lead to malfunction. The second paragraph referred
to the picture of components and recommended that the subject become
familiar with the components before proceeding.
The manuals with either process graphics or outcome graphics each had
one graphic per step; the manual with both graphical types had two graphics
per step. Samples of process graphic and outcome graphic versions are shown
on the following pages in Figures 2-2 and 2-3.
19


Meknik Moving Carriage Crank Arm Assembly
8. Slide the connected plates and bushing housing onto the acme nut assemblies, so
the top plate's crank link bearing nut is further from the gear motor.
9. Loosely fasten the acme nut assemblies to the top and bottom plates with
a. 4 1 /4 -20 socket head cap screw's, and
b. 4 lock washers.
Figure 2-2
Page from manual with process graphics
20


Meknik Moving Carriage Crunk Arm Assembly
8. Slide the connected plates and bushing housing onto the acme nut assemblies, so
the top plate's crank link bearing nut is further from the gear motor.
9. Loosely fasten the acme nut assemblies to the top and bottom plates with
a. 4 V.f -20 socket head cap screws, and
b. 4 lock washers.
Figure 2-3
Page from manual with outcome graphics
21


Concerning the use of more than one picture with the same step: the
reader may have a dilemma of deciding what the relationship between them is
(Szlichcinski, 1979). To remove this problem, I placed the outcome graphic to
the right of the process graphic, mimicking an action-result sequence. This
also conformed to left-right cultural convention.
Vfeknik Moving Carriage Crank Arm Assembly
8.
Slide the connected plates and bushing housing onto the acme nut assemblies, so
the top plate's crank link bearing nut is further from the gear motor.
9. Loosely fasten the acme nut assemblies to the top and bottom plates with
a. 1 /,i -20 socket head cap screws, and
b. 4 lock washers.
Figure 2-4
Page from manual with both process and outcome graphics
22


The graphics were placed below the step to which they pertained. In this
way, all three manuals with graphics were the same length to prevent a
preference related to document length. In addition, each graphical image had
a box around it, which Szlichcinski indicates would help readers to view the
pictures sequentially (1982), in the case of the version with both graphical
types.
Subjects
Twenty-five subjects volunteered for participation in the study (7 females
and 18 males). I recruited subjects from the University of Colorado at Denver's
Mechanical Engineering Department specifically to acquire subjects who were
less likely to examine the information design of the manuals critically, as
might students of English or Technical Communication. Two subjects were
graduate students; 23 were undergraduates. Subjects ranged from 21 to 44
years of age, with the average age being 25 years. Years of work ranged from 0
to 25 years, with the average being less than 2 years. Subjects were paid $15 for
their participation.
Experimental Design
The four manuals were used to study two factors:
Can readers tell the difference between process graphics and outcome
graphics?
Do readers prefer one graphical type over another, if they are given a choice?
Dependent measures included behavioral compliance with the instructions,
reading and thinking aloud, amount of manual read or skimmed, and time
spent viewing the manuals.
23


Procedure
To preclude subjects from guessing the purpose of the study, I recruited
subjects under the guise that my objective was to examine the ergonomics of
the product and its assembly process from a mechanical engineering
perspective. I took the individual subjects to a room in which the assembly
was supposed to take place. The experiment began with me reading a project
background and consent form to all subjects (Appendix B). Then subjects saw
all the necessary parts and tools required for the assembly. The experiment
began with reading the following instructions to the subject:
You are about to begin a think-aloud protocol so that I may
gain your input concerning the ergonomics of a new device and
how appropriately the materials and components work together
for the intended function of the device. I want to emphasize that
the protocol is examining this new device and not your personal
ability or skill with the tools and materials used in this build. In
other words, this protocol is not testing you. It is important that
you remember this fact, should you encounter any difficulties in
the tasks that you perform. If you encounter any difficulties
while performing your task, you should continue to think aloud
about these difficulties as you try to work through them.
I will start the tape recorder, and I will give you a short
practice task to help you get used to thinking out loud. In this
case, you will make a choice among analgesic medicines by
gathering information from the labels. If you already have a
preference among these medicines, please behave as a consumer
trying to make a new decision. Make your choice based upon
information from the labels. I want you to practice such decision-
based think-aloud activity because I want your opinions and
preferences that arise while building the device.
24


After the think-aloud practice, I will have you begin the task
of building the device, starting with choosing an assembly
manual. The task will end when you announce that you have
finished the build or can no longer continue. If you have no
further questions, let's begin. Here is your practice task.
The objective of the instructions was twofold. First was to place the subject
into a context of performing a new task for which his or her mechanical
engineering skills were necessary. Second was to lead subjects to believe that
the practice think-aloud was only to get them practiced at thinking aloud,
while I specifically wanted to have them practice making a choice. In this way,
each subject became practiced with thinking aloud specifically during a choice
task.
After each subject had become used to thinking aloud while being tape-
recorded, I instructed the subject to begin the actual think-aloud protocol,
which began with choosing a manual to use for building the device. Tape
recording also allowed me to measure the subjects' decision times. Manuals
were always presented as a stack of four manuals, randomly mixed.
Decision-Making Time
While subjects spoke and thought aloud with the tape recorder placed
near them, I remained in the room but appeared to attend to other tasks of my
own. By appearing to attend to other tasks, I did not appear overly interested
in the subject's performance but was able to hear when the subject had
decided on which manual to use. I refer to all events after that decision as
occurring post-decision.
25


Post-Decision Time
When the subject had chosen a manual and was preparing to build the
device, I interrupted the subject, while leaving the tape recorder running, to
ask which manual he or she had chosen. After the subject gave the first choice,
I asked the subject to rank the remaining manuals (most to least preferred). In
this way, I tried to make this preferential ranking seem to be a continuation of
the same task, thereby changing the subjects' thought patterns as little as
possible.
Post-Decision Conversation
When I asked subjects to give their next choice of manual, after the one
they chose for the task, frequently they would become conversational in their
answers. This occurred with 48% of subjects (12). In response to being asked
what their next choice of manual would be, they would become descriptive of
the manuals, the pictures and text, and their reading behaviors. Seemingly, by
virtue of my involvement with the subjects at that time, the subjects exited the
think-aloud frame of mind and entered a conversational frame of mind.
I had not anticipated that conversation would follow subjects' decisions,
but I believed that conversational data could reveal important information
about readers' wants also. Although subjects may have still been in the context
in which they believed they would build the device, this was no longer think-
aloud nor decision-making data. I believed I had to differentiate it from the
decision-making data, so I refer to it as post-decision conversation.
26


Debriefing
I then debriefed the subject with a debriefing script:
OK, please stop. I am actually only collecting think-aloud
information concerning your decision-making when choosing
from among the manuals. Now that you have chosen a manual,
the protocol is ended. I have no need for further information. I
will pay you as you have expected. As mentioned on your
consent form, you may obtain information about this study's
outcome.
I then paid the subject and also asked the subject not to report what he or she
experienced to other potential subjects, in an attempt to not shape
expectations of other potential subjects.
27


CHAPTER 3
RESULTS
Recognizing Differences Between Graphical Types
From the think-aloud data during subjects' decision- making, 72% of
subjects (18 subjects) indicated that they recognized differences between the
graphical treatments. Initially, some subjects also expressed difficulty
recognizing differences between graphical choices, although they usually did
recognize differences eventually. There were four types of difficulty in
recognizing differences between graphical sets:
16% of subjects (4 subjects) expressed difficulty recognizing the difference
between the combined and process graphical sets. (This was the most
common difficulty.)
4% (1 subject) expressed difficulty recognizing the difference between the
combined and outcome graphical sets.
4% (1 subject) expressed difficulty recognizing the difference between the
process and outcome graphical sets.
4% (1 subject) expressed difficulty recognizing the difference between the
outcome graphical set and the manual with no graphics.
This pattern of results may indicate that some subjects were more likely to
observe process graphics than outcome graphics. When looking at two
graphical sets that had process graphics in them, these subjects seemed only to
see the process graphics initially. In support of this supposition, it is
interesting to note that one subject commented that the outcome graphical set
had fewer graphics than the process graphical set.
28


Graphical Preference
Overall, the majority of subjects chose the manual with both graphical sets
(Figure 3-1), although not 100% as predicted. During decision-making, 72% of
subjects (18 subjects) chose to use the manual with both graphical sets. Of
those who chose the manual with either process or outcome graphics as their
first choice, there was no strong difference. Because some subjects changed
their preference during post-decision conversation, initial decision-making
graphical preference is shown beside post-decision graphical preference.
The decision data do not add to 100% because 1 subject (4%)
chose two manuals for first choice (one with both graphical sets
and one with the process graphical set).
Figure 3-1
29


After subjects chose which manual they wanted to use to build the
electromechanical device, they were asked to rank the remaining manuals in
order of preference. The results for second choice show a very strong
preference for process graphics over outcome graphics (Figure 3-2). Eighty
percent of subjects (20 subjects) preferred the process graphics over the
outcome graphics.
Second Choice
The decision data do not add to 100% because 5 subjects (20%) chose both
the manual with the process graphical set and the manual with the outcome
graphical set as their second choice.
Figure 3-2
30


Continuing the preference ranking, a majority picked the manual with
only outcome graphics as their third choice (Figure 3-3). One-hundred percent
of subjects ranked the manual without graphics as their last choice.
V)
o
0)
'S
3

o
m
O)
ra
c
4)
O
i.
0)
Q.
100
90
80
70
60
50
40
30
20
10
0
Third Choice
L Decision
0STDEVP = 27.86
Post-Decision
STDEVP = 34.22
0% 0%
-----------------(
No
Graphics
84%
8% 8%

Both
Graphics
The decision data do not add to 100% because 5 subjects (20%) chose both
the manual with the process graphical set and the manual with the outcome
graphical set as their second choice, while catling the Text-only option their last choice of the four.
Figure 3-3
These preference results match very closely with the predictions made
earlier. Because the choice data show rather distinct preferences among the
graphical choices, I examined the decision-making think-aloud data to learn if
subjects gave any distinct reasons for choosing the way they did.
Reasons for Preferences
From the decision-making think-aloud data, I was able to learn reasons
why subjects made the graphical choices they did.
31


Preference for Manual with Both Graphical Sets. Although the manual
with both graphical types was the first choice of the majority of subjects,
subjects gave only three reasons why they preferred it:
48% (12 subjects) said they liked the "during and after" effect of the two
graphics with each step.
28% (7 subjects) said they preferred more pictures.
4% (1 subject) made the comment that more pictures provided more details.
Preference for Process Graphics Over Outcome Graphics. Subjects showed
a strong preference for process graphics over outcome graphics, a preference
for which they revealed five reasons (Figure 3-4):
56% (14 subjects) said that process graphics seemed more helpful than
outcome graphics.
12% (3 subjects) said they liked seeing hands in the pictures.
8% (2 subjects) said process graphics seemed more descriptive to them.
4% (1 subject) specifically did not like outcome graphics.
4% (1 subject) enjoyed a sense of human involvement given by hands.
32


Vocalized Reasons for Choosing
Process Graphics Over Outcome Graphics
100

4-
o
0)
S'
3
CO
o
0)
U)
CD
+4
c
0)
o
0)
Q.
90
80
70
Seemed Liked hands. Seemed Did not want Liked human
more more outcome involvement,
helpful. descriptive. graphics.
Figure 3-4
Preference for Outcome Graphics Over Process Graphics. Although a
minority of subjects (4 subjects) preferred outcome graphics, think-aloud data
gave three reasons for the preference:
4% (1 subject) wanted outcome graphics.
4% (1 subject) found the hands distracting.
4% (1 subject) thought the text was better in the manual with outcome
graphics.
Decision Changes
I had not anticipated that subjects might change their order of preference
after they had made their decision. Of the seven subjects who changed their
order of preference, I found no correlation with other variables, such as age,
gender, years of education or years of work experience. I grouped the decision
times to see if there might be a correlation to decision changes (Figure 3-5).
Grouping the decision times revealed three points:
33


No subject made a decision in less than a minute.
32% (8 subjects) of subjects, the largest time group, made their decision in
between 1 to 2 minutes.
80% (20 subjects) of subjects made their decision in between 1 to 4 minutes.
Preference Changes and Decision Times
(A
o
a>
!a
3
O
a)
ai
ra
-*-<
c
<0
o
k_
a)
a.
100
90
80
70
60
50
40
30
20
10
0
1 + to 2 2+ to 3 3+ to 4 4+ to 5 5+ to 6
minutes minutes minutes minutes minutes
4 4
------
6+ to 7 7+ to 8
minutes minutes
l l Number of
Preference Changes
Percentage of
Subjects in Decision Time
Figure 3-5
The small number of preference changes across time groups prevents
observing any significant differences, and think-aloud data did not reveal why
seven subjects changed the order of their preference. Post-decision data
revealed two kinds of preference changes:
34


Change of First Choice Preference.
Two subjects (Ss #6 and 7) changed their first choice preference from the
manual with the outcome graphical set to the manual with both graphical
sets.
One subject (S #10) originally chose both the manual with the process
graphical set and the manual with both graphical sets.
Subjects #6 and 7, although running through the protocol separately,
behaved very similarly. Each happened to view the text-only version manual
first. In each case, by comparison to the text-only version manual, the subject
strongly preferred the first graphical version of the manual to appear and
chose it immediately, without viewing the other two manuals. By chance, each
subject next viewed the manual with the outcome graphical set. By not
viewing the other two manuals, the subjects preferred the other manuals with
graphical sets equally over the text-only version of the manual. Upon more
complete inspection of the manuals during the post-decision period, each
subject preferred a) the manual with both graphical sets and b) the manual
with process graphical set to the manual with the outcome graphical set.
Subject #10 decided to perform the build using two manuals (those with
both graphical sets and the process graphical set). In post-decision time, when
asked to rank the manuals, the subject preferred a) the manual with both
graphical sets and b) the manual with process graphical set to the manual with
the outcome graphical set.
35


Change of Second Choice Preference.
Three subjects (Ss #3,16 and 17), who initially rated the manuals with single
graphical sets equally after first choice preference, eventually rated the
manual with the process graphical set higher than the manual with the
outcome graphical set.
One subject (S #26) initially rated the manual with the process graphical set
as second preference but changed preference to the manual with the
outcome graphical set.
Subjects #3,16 and 17 based their preference on which manual had the
most graphics. Thus, these subjects did not differentiate between process and
outcome graphical sets at first, and they gave equal rank to the manuals with a
single set of graphics. When asked to rank the manuals during post-decision
period, all three ranked the manual with the process graphical set higher than
the manual with the outcome graphical set.
Subject #26 initially preferred the manual with the process graphical set to
the manual with the outcome graphical set but, in post-decision discussion,
preferred the manual with the outcome graphical set for a second choice.
Decision Confidence. Of note, no subjects who took more than four
minutes to decide which manual to use changed their decisions. All seven
decision changes occurred within the 80% of subjects who made their decision
between one and four minutes. This may indicate a correlation between how
much time a person puts into a decision and the confidence that person has in
that decision.
Decision Changes Explained. Borrowing from the field of decision
research, I suggest that the subjects did not approach the choice task with a list
36


of decision-making operators "that act on the contents of short-term memory
to create new knowledge or.. move the decision maker from one point in the
problem space to another" Carroll and Johnson, p. 73). In other words,
subjects did not approach the manuals with a pre-existing set of criteria, such
that when they choose instructions to do a procedural task, they compare a)
text-size, b) graphic-to-step ratio, and c) graphical content, for example.
Decision research suggests that when people make an intentional decision,
such as buying a car, they bring with them decision operators by which to
compare and eliminate alternatives, so they may arrive at a best choice
decision. Such a goal, therefore, "requires an in-depth understanding of the
task" (Carroll and Johnson, p. 81). In this study, it is quite likely subjects did
not have such a set of operators for two reasons:
The context made them anticipate building something instead of making a
choice.
The population was specifically not specialized in information design.
As a result, subjects only gained criteria for making a decision between
manuals as they progressed through the manuals for the first time. Because
decision makers need to place criteria into a hierarchy, what Carroll and
Johnson call rank ordering "the attributes in terms of importance" (p. 64), it is
likely that subjects who changed their decision were still organizing this
hierarchy, even after their initial decisions. Therefore, it also may be that
subjects who took more time to decide were creating their hierarchy during
that time, whether or not thinking so aloud, and felt no need to adjust the
hierarchy after their decision.
In the case of the two subjects (Ss #6 and 7) who first observed the text-
only version of the manual and chose the first graphical version they viewed,
perhaps any graphical addition to the text seemed an important enough
37


criterion to choose that manual immediately, especially if feeling rushed, as
one of the two subjects indicated in post-decision conversation. As a result, the
subjects may not have felt the need to build a hierarchy any further during the
decision-making part of the protocol.
Reader Desires
This study had the potential to collect information about what readers say
they want and do not want in procedural instructions. The data reveal a wide
variety of qualities that readers like and dislike.
Qualities Readers Said They Like
The think-aloud data revealed 12 qualities that readers said they look for
or like to have with instructions:
80% (20 subjects) said they like pictures.
64% (16 subjects) indicated they like process graphics.
36% (9 subjects) said they like more pictures, when available.
28% (7 subjects) said they want rich descriptions, whether textual or
graphical.
24% (6 subjects) indicated they want to see outcome graphics.
24% (6 subjects) said they want to see "during and after" graphics.
16% (4 subjects) said they like to see hands in the graphics.
16% (4 subjects) said they want instructions that are easy to read.
12% (3 subjects) said they want be able to accomplish the task quickly.
12% (3 subjects) said they like to see an inventory list with picture(s) of parts.
4% (1 subject) verbalized a desire for larger pictures.
4% (1 subject) verbalized a desire to see more white space with text.
Qualities Readers Said They Dislike
The think-aloud data revealed five qualities that readers say they avoid or
dislike having when using instructions:
38


76% (18 subjects) indicated they disliked or avoided instructions with few or
no pictures.
16% (4 subjects) said they dislike too much detail.
4% (1 subject) verbalized a dislike for unclear pictures.
4% (1 subject) disliked hands, believing they obstruct a graphic message.
4% (1 subject) disliked reading.
From these data, subjects appear to be more concerned with graphics than
text, so I then examined think-aloud data to try to perceive reading behavior
and learn what subjects say about reading.
Reading Behavior
Although the protocol led subjects to believe that they would perform a
novel and complex task, the think-aloud data reveal that subjects typically did
not read much text while making their decision about which manual to use
(Figure 4-7). Reading usually subsided quickly in favor of looking at and
comparing graphics. It is possible that subjects planned to read the
instructions during the build; however, few read much of the instructions
while making their choice decision:
40% (10 subjects) read the whole introduction to the manual but no further.
28% (7 subjects) read past the introduction, although not deeply into the
instructions, and compared only a few sentences and pictures.
4% (1 subject) read the entire manual.
39


Figure 3-6 reflects the above-mentioned bulleted points.
Reading Patterns
100
90
w
o 80
0)
70
3
W 60
O
o 50
O)
40
c
30
20
10
0
48
28
Read whole introduction. Read past introduction.
Patterns
Figure 3-6
4
Read entire manual.
From think-aloud data it seems that perhaps 60% of the subjects (15 subjects)
did not read the whole introduction, and fewer subjects read any amount
beyond the introduction.
Meta-Reading Comments
Observing that few subjects seemed to read much of the instructions
before they decided which manual to use to build the device, it made sense to
look for meta-reading comments by looking at the think-aloud data and post-
decision data in such a way to understand how subjects read or what subjects
said about reading. Keeping in mind that the context of the protocol led
subjects to believe that they would perform a complex and novel task, there
40


may be reason to expect that careful reading to choose a manual might be
considered desirable.
The think-aloud data seem to indicate a frequent occurrence where
subjects would start to read the beginning of the manual and then drift
through the manual, skim-reading parts while flipping through it. Still other
subjects went directly to skim-reading parts while flipping through it, without
reading the beginning paragraphs. I labeled these behaviors as a "reluctance
to read" prior to building, that is, before starting the task. Clearly, it is
unrealistic to expect subjects to read a build guide before they actually begin
to build. The think-aloud data also showed some subjects talked about looking
at the pictures (not the text).
From the post-decision discussion, several subjects confessed to reading
little to no text at all. And one subject expressed a desire to not read at all,
except as a backup to the graphics, if necessary. Together, think-aloud data and
post-decision data indicate the following about what subjects may think about
reading (Figure 3-7):
76% (19 subjects) evidenced a reluctance to read prior to building, even if
they had started to read.
48% (12 subjects) did not read out loud, despite instructions to do so, while
deciding which manual to use.
36% (9 subjects) talked about looking at pictures (not the text).
28% (7 subjects) confessed to reading little or none of the text.
4% (1 subject) expressed only wanting text as a backup to pictures.
41


Percentage of Subjects
Vocalized Information
About Reading
100
90
80
70
60
50
40
30
20
10
0
76
48
----- 36
28
Evidenced
reluctance to
read.
Did not read
while deciding.
Talked about
looking at
pictures.
Confessed to
reading little or
none.
Figure 3-7
4
_____1 I
Wanted text as
backup to
pictures.
42


CHAPTER 4
CONCLUSION
Implications
As mentioned earlier, at a time when technology soon may make it
practical to create individualized instruction, knowing a reader's preferences
may become an important part of effective instructional design. Because
readers exhibit different learning styles, graphical preference may play an
important role in determining a person's learning style. From knowing an
individual's learning style, documentation can help increase a person's
productivity (Smart, Whiting and DeTienn, 2001).
The results of this research have a number of implications and/or
recommendations for information designers, particularly when designing
assembly instructions. First, the majority of readers, 72% (18 subjects), can tell
a difference between outcome and process graphics during think aloud.
Second, the research also indicates that readers largely prefer process
graphics to outcome graphics (80%, or 20 subjects), when they are given a
choice. The research does not suggest that subjects' choices necessarily
correlate with performance.
Third, the results of this research are supportive of Fukuoka, Kojima and
Spyridakis' study, in which no readers preferred a text-only choice set of
instructions (n=29). With this current research there is evidence that readers
want to have graphical information with their instructions and may refuse to
use instructions without graphics. Additionally, readers want more graphics,
43


when available, which is also supportive of Fukuoka, Kojima and Spyridakis's
study.
Fourth, of further importance to information designers, the research
results reveal some specific likes and dislikes that readers have when choosing
a manual. A collection of such information might be useful for information
designers when they consider what to include when they write for their
audiences.
Finally, the results of this research may indicate that readers are reluctant
to read assembly instructions before they actually begin an assembly task. This
is important for information designers to consider if there is valid expectation
that readers should read the assembly instructions prior to assembly, for
instance to become familiar with precautionary information, or novel
components or functions beforehand. For such purposes, a more effective
approach may be to list and describe precautions, novel components and
functions separately from assembly instructions. Another approach may be to
have a temporal division between familiarization and assembly. In this study, I
led subjects to believe they would build the device quite soon, so their
attention may have been focused on the future event rather than the
immediate decision making in which they were involved. However, it does
appear that readers will study graphical aspects of assembly instructions prior
to an assembly task.
Regarding subjects' reluctance to read much of the assembly manual prior
to actually assembling the electromechanical device, very likely it is unrealistic
to expect readers to behave differently than they did. I looked into how people
use manuals, which led me to work by Schriver and Smart, Whiting and
DeTienn.
44


Schriver (1997) states that "the first decision people make when confronted
with a document is whether or not to read" (p. 164) and goes on to say that
"anecdotal evidence suggests that many people prefer not to read at all unless
they have to." The realistic context in which subjects believed they were
working demanded that they behave as realistically as possible. Inasmuch as
the manuals serve effectively the rhetorical function of accurately leading to
the assembly of the device, subjects believed they were in a realistic context. It
is probably safe to believe that subjects' decision-making and reading
behavior is similar to real life consumer behavior.
To check how consumers use manuals, Schriver (1997) asked 201 people
how they read manuals they had encountered in the past. Schriver's research
describes a context into which my subjects' vocalized and meta-reading
comments fit similarly. About 80% of Schriver's respondents reported
scanning manuals or using them for reference, compared to 76% of my
subjects who quickly or immediately began to skim read parts of a manual
while flipping through it. Only 15% of Schriver's respondents reported
reading the manuals fully and 4% claimed to never read manuals.
Furthermore, Schriver then asked those 201 people to report how they
used their most recently acquired instruction manuals for four different
products. To this question, across all products, an average of only 23%
reported reading the manual before using the product. A much higher average
percentage, 42%, reported reading the manual while working with the
products. The present results suggest that the expectation readers will read
instructions while working with a product, and not before, to be much more
realistic. Schriver's respondents went on to indicate that 17% reported using
the manual as a reference and 19% reported not using the manual at all.
45


Schriver's respondents describe a context into which my subjects' vocalized
and meta-reading comments can fit.
In another study (2001), Smart, Whiting and DeTienn compared using
printed manuals to using online help (print vs. electronic form) for users of a
word processing application. In this study (n=400), they learned that more
users used the printed version than the online help within the last month (92%
compared to 54%). Inversely, fewer users said they never used printed
documentation than those who said they never used online help (1%
compared to 35%).
To try to better understand the context in which a subset of those users
(n=18) chose to use printed or online documentation, Smart, Whiting and
DeTienn used an ethnographic method called contextual inquiry (Cl), as
detailed by Beyer and Holtzblatt (1998) and Raven and Flanders (1996). Cl
gathers data through interviews over a period of days or weeks, during which
the interviewer observes a user in the context of the user's work environment.
The observers are then able to ask questions that lead users to perform tasks
and discuss work as they perform it. In this way, the interviewers aren't
relying only on the users' memories but are helping to recreate the activity.
Although the manuals used by Smart, Whiting and DeTienn and those
studied in this paper were of different sizes, comments sound similar to those
of the subjects in this paper. One person talked about scanning the manual.
Only one claimed to read the manual actually cover to cover, to be able to
understand tasks before installing the software. Five users talked about using
the manual, not to accomplish their tasks but if only they had problems doing
so. In spite of the difference in subjects' tasks between Smart, Whiting and
DeTienn's study and the present study, there are similar vocalizations.
46


Improvements
As mentioned earlier, subjects only gained criteria for making a decision
between manuals as they progressed through the manuals for the first time. Of
note, what subjects claimed they desire from the documentation probably
reflects what they would use as criteria. Looking at subject desires and
decision times, it is likely that decision times would have been shorter if
subjects had come to the task with previous criteria with which to choose an
assembly manual. The average decision time was 3.14 minutes (3 minutes, 8
seconds). Because decision makers need to place criteria into a hierarchy, what
Carroll and Johnson call rank ordering "the attributes in terms of importance
(p. 64)," it is likely that subjects would have been able to make their decisions
more quickly. Furthermore, if subjects had come to the task with previous
criteria with which to choose an assembly manual, it is likely that fewer would
have changed their decisions later.
Unfortunately, this study's design may have led some subjects to change
their decision during post-decision. While I attempted to remain neutral about
their preferences, and indeed felt I had no preference that subjects choose one
graphical type over another, subjects may have felt there was a "correct"
choice and may have changed their decision in an effort to please the
researcher (possibly even unconsciously). To remove such a possibility, an
improvement in this study's design could be to have two people run the
subject protocols: one to set the necessary context with the benign deception,
explain the instructions and read the subject rights, and another person to start
and stop the subjects' think-alouds. By only appearing for the think-aloud, the
47


second person may be viewed as someone more neutral, from whom subjects
would be less likely to seek approval.
Also, with this study I used process graphics that showed hands, for
which subjects may have a psychological and anthropomorphic preference. In
fact, during one subject's think-aloud, the subject said the hands gave him
comfort and made him feel like someone would help him perform the task.
Perhaps a fairer graphical comparison would have been to use action elements
instead of hands in the process graphics.
Finally, I used photographic images for the graphics in this study. I am
inclined to conduct further research to learn whether readers show similar
results using instructions with a graphical media other than photographs.
It is important to remember that, although I was able to observe that
readers have graphical preferences, the results do not indicate whether
process graphics or outcome graphics are more likely to lead to better task
performance. In addition to the above-mentioned alternatives, further study is
necessary to determine whether one graphical type leads to better task
performance than the other graphical type, in both time and quality.
48


ARTIFACTS OF THE STUDY
APPENDIX A
Attention Mechanical Engineering Students!
Paid Volunteers Needed
to test the ergonomics and
material/component appropriateness of
a new mechanical design.
Earn $15 for participating 30 minutes or less in this study.
To sign up for this study, please contact Michael Sharp:
msharp@maroon.cudenver.ed u
303-832-9184
Flyer used to announce study
49


APPENDIX B
Fase of Assembly for Moving-Carriage Crank Arm;
Background and Consent
Background
1 am interested in learning about the ergonomics of a new device and how
appropriately the materials and components work together for the intended
function of the device. Everything you say aloud and think aloud will be tape
recorded and may be transcribed by someone other than myself.
Your participation in this research today will help to better understand the
appropriateness of using these materials and components together. Your
participation is voluntary, and will last approximately 30 minutes. When you
complete the task described below, or feel that you can no longer continue with the
task, I will pay you $15. Please read all of the information on this page carefully
before completing and signing the attached consent form.
1. Shortly, you will be asked to think-aloud while you build a moving-carriage
crank arm. 1 want to emphasize that the protocol is examining this new device
and not your personal ability or skill with the tools and materials used in this
build. In other words, this protocol is not testing you. It is important that you
remember this fact should you encounter any difficulties in the tasks that you
perform. If you encounter any difficulties while performing your tasks, you
should continue to think aloud about these difficulties as you try to work
through them.
2. Before starting to build this device, you will think aloud while you choose an
instruction manual that describes how lo complete the build. As you make your
choice, please read out loud all material that you happen to read. As von read.
please sav out loud everything you happen to be thinking about as you proceed.
If you need to re-read something, no matter how often, continue reading aloud
and thinking aloud. Try to not fall silent.
3. When you have chosen an instructional manual, proceed to the table with the
components and begin the build, while thinking aloud. As von build, please sav
out loud everything you happen to be thinking about as you proceed. If you need
to re-read something, no matter how often, continue reading aloud and thinking
aloud. Try to not fall silent.
Consent Form to Protect Your Confidentiality
Before you start your building task, I would like you to fill out the data and consent
form (on following page), and sign it. It asks for your age, gender, occupation,
program of study and your highest degree completed at present. You may keep a
copy of the consent form. When you are finished the task, or feel that you can no
longer continue, 1 will pay yon 515. and you may ask any questions about the
research at that time. If you have any questions about your rights as a participant n
this research, please contact the Office of Academic Affairs. CU Denver Building.
Suite 700. 202-556-2550. You may also reach me (Michael Sharpl at 303-B32-91S4.
Study description read to and given to subjects, to set context
50


APPENDIX C
Consent Form
I, the undersigned, am a voluntary participant in Meknik Engineering Ease of
Assembly for Moving-Carriage Crank Arm project. I understand that I will he paid
5/5.00 upon completion of my participation in this research. The results of the
research will be used to help improve the design and usability of the moving-
carriage crank arm. /is a participant, I understand that the identities of the
participants involved in this research trill not revealed in any publications, data or
communications resulting form the research. I further understand that the identity
of participants will not he revealed to any person other than the Principle
Investigator in charge of the project (Michael Sharp).
Protection methods against toss of confidentiality have been explained to me. I
realize that atphnumeric encoding and locking data in a cabinet do itof ensure total
protection against loss of cofidetttialily. Furthermore, / understand that I may
withdraw from this study at any time, and request that data collected from me be
destroyed and not used, without forfeit of payment.
I understand all of the above instructions and agree to participate under the terms
described above:
Prim name: _______________________________________
Signature: ________________________________ Date:
Age: ----------
Gender: Male_____ Female_____
Occupation: _________________
Program of Study: _________________
Years of Study: ____________________________________
Highest Degree Completed at Present: ________________
Consent form that subjects would read, fill and sign
51


APPENDIX D
Script
Yuu are about to begin a think-aloud protocol so that I may gain your input
concerning the ergonomics of a new device and how appropriately the maLcrials
and components work together for the intended function of the device. 1 want to
emphasize thaL Lhe protocol is examining this new device and not your personal
ability or skill with the tools and materials used in this build, In other words, this
protocol is not testing you. It is important that you remember this fact, should you
encounter any difficulties in the tasks that you perform, If you encounter any
difficulties while performing your tasks, you should continue to think aloud about
these difficulties as you try to work through them.
1 will start the tape recorder, and I will give you a short practice task to help you get
used to thinking out loud. In this case, you will make a choice among analgesic
medicines by gathering information from the labels. If you already have a preference
among these medicines, please behave as a consumer trying to make a new decision.
Make your choice based upon information from the labels. I want you to practice
such decision-based think-aloud activity because I want your opinions and
preferences that arise while building the device.
After the think-aloud practice, 1 will have you begin the task of building the device,
starling with choosing an assembly manual. The task will end when you announce
that you have finished the build or can no longer continue.
You may contact Lhe Office of Affairs, CU Denver Building, Suite 700, 303-556-2550, if
they have further questions about your rights as a research subject. Do you have any
questions? if not, let's begin. Here is your practice task.
Script read to subjects
52


APPENDIX E
Stop Script
... w/icn subject is ready to begin budding the device,
OK, please stop. In the beginning of this project I led you to believe that you would
actually build the moving-carriage crank arm. I never intended that you would
actually perform the build. 1 led you believe you would perform the build so that
you would carefully consider the instructions.
I am actually only collecting Lhink-aloud information concerning your decision-
making when choosing from among the different manuals. Now that you have
chosen a manual, Ihc protocol is ended. 1 have no need for further information. I
will pay you $15.00 as you have expected. As mentioned on your consent form, you
may obtain information about the outcome of this study.
Furthermore, measures are in place to keep the identities of all subjects in this study
confidential. Additionally, all information and data collected from you will be
paired with an alphanumeric code to attempt to remove any connection between
you and the data.
Stop script read to subjects after they had given choice and ranking
53


BIBLIOGRAPHY
Anglin, G. J., Stevens, J. S., 1986. Prose-relevant pictures and recall from
science text. Perceptual and Motor Skills, Vol. 63, (3), 114145.
Ausel, D., Bieger, G. R., 1989. The durability of picture text procedural
instructions for individuals with different cognitive styles. Research report:
Indiana University of Pennsylvania. ERIC # ED30807
Beyer, H., Holtzblatt, K., 1998. Contextual design: Defining customer-centered
systems. San Francisco: Morgan Kaufman.
Booher, H. R., 1975. Relative comprehensibility of pictorial Information and
printed words in proceduralized instructions. Human Factors, 17(3): 266-
277.
Burnham, C., 1992. Improving written instructions for procedural tasks.
Berkeley: National Center for Research inVocational Education.
Carroll, J. S., Johnson, E. J., 1990. Decision research: Afield guide. Sage
Publications, Inc.
DeCorte, E. (Ed.), 1993. Comprehension of graphics in text. Learning and
Instruction, Vol. 3 (3), 151-249.
Ellis, J.A., Whitehill, B. V., and Irick, C., 1996. The effects of explanation and
pictures on learning, retention, and transfer of a procedural assembly task.
Contemporary Educational Psychology, Vol. 21, 129-148.
Fukuoka, W., Kojima, Y., and Spyridakis, J., 1999. Preference and effectiveness
with Japanese and American readers. Technical Communication (2nd
quarter, 1999): 167-176.
Gombrich, E., 1990. Pictorial Instructions. In Barlow, H., Blakemore, C., and
Weston-Smith, M (Eds.) Images and understanding (pp. 26-45). Cambridge
University Press.
54


Guthrie, J. T., Bennett, S., and Weber, S., 1990. Processing procedural
documents: A transformational model. University of Maryland.
Kern, R. P., 1985. Modeling users in their use of technical manuals. In Duffy
and Waller (Eds.) Designing usable texts (pp. 341-375). Orlando, FL:
Academic Press.
Lohse, G. L., Biolsi, K., Walker, N., and Rueter, H. H., 1994. A classification of
visual representation. Communication of the ACM, Vol. 37 (12), 36-49.
Macdonald-Ross, M., 1989. Towards a graphic ecology. In Mangl, H. and Levin
R. (Eds.) Knowledge acquisition from text and pictures (pp. 145-154). Elsevier
Science Publishers B.V. North Holland.
Miller, J., 1990. Moving pictures. In Barlow, H., Blakemore, C., and Weston-
Smith, M (Eds.) Images and understanding (pp. 180-194). Cambridge
University Press.
Peeck, J., 1994. Enhancing graphic-effects in instructional texts: Influencing
learning activities. In Schnotz, W. and Kulhavy, R.W (Eds.) Comprehension
of graphics, (pp. 291-301). Elsevier Science Publishers: Amsterdam.
Raven, M. E., Flanders, A., 1996. Using contextual inquiry to learn about your
audiences. The Journal of Computer Documentation, 20 (1), 1-13.
Schriver K. A., 1997. Dynamics in Document Design. John Wiley & Sons, Inc.
Smart, K. L., Whiting, M. E., and DeTienne, K. B., 2001. Assessing need for
printed and online documentation: A study of customer preference and
use. Journal of Business Communication, 38 (3) pp. 285-314.
Smillie, R. J., 1985. Design strategies for job performance aids. In Duffy and
Waller (Eds.) Designing usable texts (pp. 341-375). Orlando, FL: Academic
Press.
Szlichcinski, K. P., 1979. Diagrams and illustrations as aids to problem solving.
In Instructional Science 8 (3) pp. 253-274. Amsterdam: Elsevier Scientific
Publishing Company.
55


Szlichcinski, K. P., 1980. The syntax of pictorial instructions. In Kolers, P. A.,
Wrolstad, M. E,. & Bouma, H., (Eds.) Processing of visible language, Vol. 2.
(pp. 113-124). New York: Plenum.
Szlichcinski, K. P, 1982. Factors affecting the comprehension of pictographic
instructions. In Pictorial Instructions (pp. 449-467). British Telecom Research
Department, Martlesham.
Weidenmann, B., 1989. When good pictures fail: An information-processing
approach to the effect of illustrations. In Heinz, M. and Levin, J. (Eds.)
Knowledge acqidsition from text and pictures (pp. 157-170). Elsevier Science
Publishers B.V. Amsterdam.
Weidenmann, B., 1994. Codes of instructional pictures. In Schnotz, W. and
Kulhavy, R.W. (Eds.) Comprehension of graphics (pp. 29-42). Elsevier Science
Publishers Amsterdam.
Winn, W., 1994. Contribution of perceptual and cognitive processes to the
comrehension of graphics. In Schnotz, W. and Kulhavy, R.W. (Eds.)
Comprehension of graphics (pp. 3-27). Elsevier Science Publishers
Amsterdam.
56