Citation
The effects of generating semantic networks on knowledge synthesis as measured by expert system creation

Material Information

Title:
The effects of generating semantic networks on knowledge synthesis as measured by expert system creation
Creator:
Marra, Rose Marie
Publication Date:
Language:
English
Physical Description:
xiii, 198 leaves : illustrations ; 29 cm

Subjects

Subjects / Keywords:
Expert systems (Computer science) ( lcsh )
Knowledge representation (Information theory) ( lcsh )
Semantics -- Network analysis ( lcsh )
Expert systems (Computer science) ( fast )
Knowledge representation (Information theory) ( fast )
Semantics -- Network analysis ( fast )
Genre:
bibliography ( marcgt )
theses ( marcgt )
non-fiction ( marcgt )

Notes

Bibliography:
Includes bibliographical references (leaves 193-198).
General Note:
Submitted in partial fulfillment of the requirements for the degree, Doctor of Philosophy, Administration, Supervision and Curriculum Development.
General Note:
School of Education and Human Development
Statement of Responsibility:
by Rose Marie Marra.

Record Information

Source Institution:
University of Colorado Denver
Holding Location:
Auraria Library
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
36448774 ( OCLC )
ocm36448774
Classification:
LD1190.E3 1996d .M37 ( lcc )

Downloads

This item has the following downloads:


Full Text
THE EFFECTS OF GENERATING SEMANTIC NETWORKS ON
KNOWLEDGE SYNTHESIS AS MEASURED
BY EXPERT SYSTEM CREATION
by
Rose Marie Marra
B.S., Rockhurst College, 1984
M.S., University of Kansas, 1986
A thesis submitted to the
University of Colorado at Denver
in partial fulfillment
of the requirements for the degree of
Doctor of Philosophy
Administration, Supervision and Curriculum Development
1996


This thesis for the Doctor of Philosophy
degree by
Rose Marie Marra
has been approved
by
R. Scott Grabinger
Allison A. Carr

Laura Goodwin?
Elizabeth Kozleski
me
Date


Marra, Rose Marie (Ph.D., Administration, Supervision and Curriculum
Development)
The Effects of Generating Semantic Networks on Knowledge Synthesis as
Measured by Expert System Creation
Thesis co-directed by Professors R. Scott Grabinger and Alison A. Carr
ABSTRACT
Semantic networking tools and expert system shells have both been
used as knowledge generation and synthesis tools. When learners are
required to generate semantic networks and their own expert systems (using
expert systems shells which make the process accessible to non-
programmers), both tools have been shown to facilitate deeper processing,
and knowledge synthesis that educators and industry find desirable (K.A.
Fisher, 1992; D.H. Jonassen & S. Wang, 1993a; R.C. Lippert, 1988). While prior
research has shown the effectiveness of these tools used separately, this study
examines how combining both tools affects subjects' ability to generate expert
systems that demonstrate increased knowledge synthesis and deeper
processing. Subjects were divided into two groups. One group used SemNet
a proven semantic networking tool (Fisher, K.M. & J. Faletti & C.N. Quinn,
1989) to create semantic networks on their expert system topics before
producing expert systems. Subjects in the second group produced expert
systems only. Resulting expert systems, semantic networks, and subjects
performance on preliminary and ending essays were compared. There were
significant differences between groups for the number of expert system rule
types (F=6.837, p = 0.015) and rules (F= 3.323, p = 0.081), however the
remaining five expert system variables were not significant. A content


analysis of the ending essays indicates that all subjects experienced notable
synthesis during the project and suggested reasons for the predominant lack
of statistical significance. The lack of significant differences may be
attributable to the minimal scaffolding of the semantic networking to expert
system generation process. SemNet group subjects were trained on how to
produce semantic networks, but not on how to create networks that would
specifically structure their knowledge into the conditional relationships
required by an expert system's if-then knowledge base. A future study is
proposed to correct this problem.
This abstract accurately represents the content of the candidate's thesis. We
recommend its publication.
IV


DEDICATION
This work is dedicated to David my professional mentor,
my friend and partner in life.


ACKNOWLEDGEMENTS
Many kind individuals helped to make this accomplishment possible.
Endless thanks to Scott Grabinger for the constructive feedback that made
this thesis a reality, and to Ali Carr who supported this research with a venue
in which to perform it, and bottomless encouragement. To my parents, I owe
thanks for always supporting me in all my educational endeavors. Many
thanks also to Dave, Carol and Cristen who had to live with me while I
closeted myself away to study and write. And lastly, to Spencer who sat
patiently by my side waiting for me to take a break for a game of ball, or a
walk around the block.


CONTENTS
FIGURES......................................................xii
TABLES.......................................................xiii
CHAPTER
1. OVERVIEW...................................................1
Introduction............................................1
General Background......................................2
Comparison of Semantic Networks and Expert Systems.....10
Problem Statement......................................14
2. LITERATURE REVIEW.........................................16
Introduction...........................................16
Semantic Networking and Concept Mapping Overview.17
Semantic Networks Research Literature Review.....20
Assumptions and Uses........................20
Concept Mapping and Semantic Networking
Research....................................22
Summary.....................................28
Student Expert System Development Research.............29
Expert Systems Background and Components.........29
vii


Expert System Uses...............................34
Expert System Shells.............................38
Expert System Generation Benefits and Limitations......40
Expert System Generation Research......................42
Synthesis of Two Research Areas........................51
Research Hypotheses....................................58
Summary......................................................58
3. RESEARCH DESIGN.................................................60
Introduction.................................................60
Variables....................................................60
Independent Variables..................................60
Dependent Variables....................................60
Method.......................................................61
Participants...........................................61
Study Materials........................................62
Software.........................................62
Assessment Materials...................................68
Essays...........................................68
SemNet...........................................72
EXSYS............................................74
Reliability and Validity...............................76
Procedure..............................................79
Training.........................................80
vrn


Software Access..................................81
Expert System Topics.............................82
Activities.......................................82
Summary......................................................85
4. RESULTS.........................................................86
Introduction.................................................86
Data Analysis Overview.......................................86
Data Analysis Results........................................90
Subject Attrition......................................90
Hypothesis One.........................................91
Preliminary Essay Results........................91
Expert System Data MANOVA........................93
Hypothesis Two.........................................98
Ending Essay Data Categorization.................98
Hypothesis Three......................................100
Semnet and Expert System Data Correlation
Matrix..........................................100
Hypothesis Data Analysis Summary......................102
Additional Data Analysis..............................103
Activity Log Data...............................103
Summary.....................................................106
5. DISCUSSION.....................................................108
Chapter Overview............................................108
!
DC


Review of Study's Rationale...................................108
Review of Statistical Results.................................113
Hypothesis One..........................................113
Hypothesis Two..........................................115
Hypothesis Three........................................116
Reconciling the Results and the Rationale
Rule Type Variable Significance.........................120
Semantic Network Scaffolding............................123
Expert System Scaffolding...............................127
Human Computer Interface Differences....................128
Other Factors...........................................132
Insufficient Treatment Time......................132
Intolerance of Ambiguity.........................133
Limitations of Conducting Research in a "Real"
Classroom........................................135
Explanation Summary.....................................136
A Future Study................................................137
Conclusions...................................................141
APPENDIX
A INFORMED CONSENT FORM.......................................144
B EXSYS TRAINING MATERIALS....................................146
C ACTIVITY LOG................................................156
x


D SEMANTIC NETWORK LINKS.....................157
E SEMNET GROUP SUBJECTS DATA.................159
F NON-SEMNET GROUP SUBJECTS DATA.............182
REFERENCES........................................193
XI


FIGURES
Figure
1.1. Instructional theory structural knowledge example.................4
1.2. Instructional model semantic network example......................6
1.3. Semantic network representation of expert system decision........13
2.1. Instance example using SemNet....................................17
2.2. Key concepts.....................................................22
2.3. Expert system components.........................................30
2.4. Expert system acting as tutor....................................35
2.5. Mac Smarts knowledge base using examples.........................40
2.6. Cognitive processing available for target task using semantic
networks and expert system shells....................................57
3.1. SemNet instance example..........................................63
3.2. EXSYS rule creation screen.......................................67
3.3. Sample portion of an essay showing tallied data attributes.......70
3.4. Sample semantic network assessment criteria worksheet............74
3.6. Sample expert system assessment criteria worksheet...............76
4.1. Semantic network data items.....................................101
5.1. Sample expert system qualifier (factor) and its values..........118
5.2. If-then rule showing use of multiple qualifiers and values......119
E.l. Semantic networking excerpt on classroom management.............160
E. 2. Expert system report..........................................181
F. l. Non-semnet subject expert system report.......................192
xii


TABLES
Table
1.1 Expert System User Question Examples.............................7
1.2. Expert System and Semantic Network Comparison..................11
2.1. Expert System Interface Questions..............................31
2.2. Expert System Knowledge Base Example...........................33
3.1. Subjects Categorized by Treatment Group........................62
3.2. Categories Tallied In Biding Essay Data........................72
3.3. Preliminary Essay Reliability Correlations.....................77
4.1. Subject Attrition..............................................91
4.2. Preliminary Essay Descriptive Data.............................92
4.3. Preliminary Essay Data MANOVA Multivariate and Univariate
Statistics..........................................................93
4.4. Expert System Descriptive Data.................................95
4.5. Expert System MANOVA...........................................97
4.6. Category Counts from Ending Essays.............................99
4.7. Semantic Network and Expert System Correlations (n=13)........102
4.8. Activity Logs.................................................104
4.9. MANOVA........................................................105
4.10. Correlation Matrix...........................................106
5.1. Expert System and Semantic Network Comparison Highlights......110
5.2. SemNet and EXSYS Interface Differences........................130
XUl


CHAPTER 1
OVERVIEW
Introduction
Both semantic networking tools and expert system shells are examples
of knowledge representation formalisms and mindtools (Jonassen, 1993).
Jonassen defines mindtools as "generalizable tools that are intended to
facilitate cognitive processing" (p. 99). Similarly, a formalism is a means of
expressing some item in a systematically defined way. Language is a
common formalism, while semantic networks and expert system shells are
more specialized formalisms. Knowledge representation formalisms and
mindtools allow learners to externally represent a model of their internal
knowledge structures. Though neither tool is intended to provide an accurate
physical image of our internal memory structures, both help learners
explicitly define how their body of knowledge in a domain fits together into a
meaningful whole (Fisher, Faletti, & Quinn, 1989). In short, when used for
knowledge representation, both semantic networks and expert system shells
promote knowledge synthesis.
There are at least two strong forces which challenge educators and
theorists to examine ways to encourage knowledge synthesis: 1) the adoption
of constructivist learning strategies, and 2) industrial demands for employees
who can apply knowledge to novel problem-solving activities.
Construed vis tic learning means that learners build their own knowledge
l


representations (Fisher, Faletti, Thornton, Patterson, Lipson, & Spring, 1987;
Jonassen, 1991). Knowledge representation and synthesis formalisms, such as
the ones used in this study, help learners crystallize and externalize their
constructions that may be somewhat fuzzy when strictly left in their internal
thought processes.
Industry also demands the benefits of encouraging knowledge
synthesis. Many educators, researchers and theorists believe that knowledge
synthesis activities versus rote learning tasks can help learners apply their
learning to problem-solving and critical thinking activities (Allen, 1991;
Fisher, et al., 1987). Due to this concern, companies are interested in changing
their internal training in order to develop these problem solving skills.
Additionally, companies that fund external educational institutions
(universities, public schools) via fellowships and grants look for instructional
settings that promote these skills that help produce more problem-solving
capable learners. Given the links between knowledge construction activities
and problem solving, knowledge representation tools may help fulfill these
industry needs. This study responds to these industry needs by examining
the effects of combining two knowledge representation formalisms to
encourage knowledge synthesis and problem solving.
General Background
While semantic nets and expert systems can be used in many ways,
this study uses both formalisms as knowledge representation tools. A
2


knowledge representation tool is any external means of representing an
individual's knowledge. These tools are based on the idea that there is an
underlying structure or form to what we know, structural knowledge
(Jonassen, Beissner, & Yacd, 1993a). Jonassen and Grabowski (1993) define
structural knowledge as describing "an individual's organization of ideas
about different content domains" (p. 434). For instance, the information in
Figure 1.1 might represent a portion of an instructional technology student's
structural knowledge about instructional models. Note that this example is
only meant to illustrate the idea of concepts and relationships between them,
and not be a complete representation of instructional design theories and
models. The central concept, instructional theory, has several links. One link
indicates that instructional theories have attributes such as making
predictions and generating hypotheses. Other links show that there are
several instances (examples) of instructional theories and another indicates
that instructional models influence paradigms, models and instructional
practices.
3


Instances ----------------------- Instructional
Theory
connectionism I
schema theory I
systems I
motivation I
Attributes
principles
hypotheses
makes predictions
definition
Influences
paradigms
models
practices
Figure 1.1. Instructional theory structural knowledge example.
Structural knowledge is not a concrete concept that can be proven, but
rather a model for thinking about how memory may be organized. In
Jonassen, Beisner, and Yacd (1993a), several means of eliciting structural
knowledge are discussed including free or controlled word associations. In a
free word association, subjects are asked to write down as many words or
concepts that they associate with a given term within a specified time limit.
In a controlled word association, subjects must not only write down words
associated with the given term, but rank each word's relationship strength to
the given term. Once the knowledge has been elicited via one of these
methods or others, tools such as semantic networks or even a simple
outlining tool can be used to externally represent and organize this
knowledge. Semantic networks and expert system generation are both
knowledge representation tools.
4


Fisher (1992) defines a semantic network as having three parts:
concepts, relations, and instances. A concept is an information piece that can
be represented as a word or phrase. For example, in Figure 1.2, instructional
theory and schema theory are examples of a concepts. A relation or a link
describes how concepts are connected to one another. In Figure 12,
"influences" and "has instances" are relations or links. Finally, an instance is a
concept-relation-concept unit. One instance from Figure 1.2 is "instructional
theory has instances schema theory". Overall, Figure 1.2 depicts in a
semantic network from the structural knowledge pictured more informally in
Figure 1.1. Note that this is only one level of how this semantic network can
represent this domain. Users can build an infinite number of relationships
from what is pictured here. For instance, I can create more relations and
concepts starting with the concept paradigm, or models.
5


connectionism
schema theory
systems
motivation
has instances
Constructional TheonT>. inf1uences
paradigms
models
practice
has attributes

principles
hypotheses
makes predictions
definition
Figure 1.2. Instructional model semantic network example.
Unlike semantic networks which focus on graphically representing
individual knowledge structures, an expert system exists to answer user
questions. Expert systems are special software programs designed to
6


simulate the reasoning process of a human expert in a particular domain
(Grabinger, Wilson, & Jonassen, 1988). To create an expert system, experts
encode their knowledge into the part of the expert system called the
knowledge base, and determine what information they must gather from
novice users in order to answer a particular question in the domain. For
instance, an expert system about instructional theories could answer a
question such as 'Which instructional theory should I apply to the current
instructional situation?"
In order for an expert system to solve a problem, the user must provide
specific information about this problem. Even though the expert system may
always answer the question of which instructional theory the user should use,
that answer will depend on the user's particular instructional circumstances.
The expert system gathers these drcumstances or variables through a series of
questions, such as those depicted in Table 1.1.
Table 1.1
Expert System User Question Examplesl.l Expert System User Question
Examples
Question________________________________________________________Variable
What are the learning objectives for the situation? objectives
How will you assess students' grasp of these objectives? assessment
7


The answers to the questions shown in Table 1.1 then interact with the expert
knowledge base (traditionally provided by the expert) to produce the goal, or
decision. More details on expert systems are provided in Chapter two.
A semantic network is a formalism because a system of graphical links
is used to express relationships between concepts in some domain. A
semantic network is clearly a formalism both for the creator and die viewer of
the network. Both experience the graphical links showing relationships
between concepts in the domain. While the author will necessarily have a
more intimate connection and understanding of these links than a reader,
none the less, both parties can glean meaning from these links.
An expert system, on the other hand, is not necessarily seen as a
formalism by its users, but rather only by its creators. A finished expert
system simply presents users with a series of questions (see Table 1.1) and
ultimately a proposed answer to the situation. This does not appear to be a
formalism as the form of both questions and advice may vary widely.
However, the expert system creator, in order to build a system that knows
what advice to recommend under what circumstances, must create a
knowledge base of if-then rules that represent conditional relationships
between pertinent factors in die domain and potential pieces of advice or
solutions in the domain. For instance, in the instructional theory domain, the
expert system author must create a series of if-then rules, called the
knowledge base, that connect factors such as objectives and assessment to an
applicable instructional theory. This underlying knowledge base of rules is
what comprised the formalism in an expert system. End users are not
8


necessarily aware of the existence or structure of this knowledge base and
thus do not experience expert systems as a formalism. However, the creators,
who author this knowledge base do see and experience the systematic
formalism of the base's if-then rules.
Novices traditionally used completed expert systems in order to receive
advice in the expert system's domain. However, just as semantic networks
are used by novices to create their own knowledge representations of a
domain, novices can create expert systems for the same purpose. When
novices create expert systems as a knowledge synthesis activity, they create
the knowledge base that the ES decisions are based upon. This activity has
been recently made possible (since the mid-1980s) with the advent of expert
systems shells (Wilson & Welsh, 1986). Expert system shells provide non-
programmers a way to create an expert system. An expert system shell
enables people to create an expert system by simply representing the
domain's knowledge base within the shells rule editor. It is this act of
creating the knowledge base that engages the creator in novel knowledge
representation formalism. Just as building a semantic network requires the
creator to externally organize and clarify their understanding of relationships
between concepts in a domain, an expert system author must also understand
conditional relationships within a domain to create the if-then rules. In both
cases, users must organize and synthesize their knowledge in a domain. It is
this synthesis activity which can then, in turn, lead to better problem-solving
abilities in that domain.
9


Comparison of Semantic Networks and Expert Systems
Even though semantic networks and expert system shells both allow
users to represent their structural knowledge, there are several differences,
depicted in Table 12, between them. Items 1 through 3 in Table 1.2 are fairly
self-evident comparing semantic networks' relationships, concepts and
graphical organization to expert systems' if-then rule knowledge base,
domain factors and verbal-based representation. Additionally, item 4
highlights, as previously discussed, that semantic networks are clearly
formalisms to both users and authors while expert systems only appear as
formalisms to their authors. These differences are dear on the surface, but
another more important difference lies in the substance of the knowledge
representation the tools promote, hi this study, I hypothesize that expert
system generation as a knowledge representation and synthesis activity may
be enhanced by combining it with semantic networking.
10


Table 1.2.
Expert System and Semantic Network Comparisonl.2. Expert System and
Semantic Network Comparison
Item # Semantic Networks Expert Systems
1 has links and relationships has if-then rules
2 iconic/graphical word-based
3 has concepts or nodes has decision making factors
4 formalism to users and authors formalism to authors only
5 flexible knowledge structured knowledge
representation strategy representation strategy
6 shows any inter-relationships of represents logic behind decision
concepts in domain making in a knowledge domain
7 expresses initial, simple expresses refined, conditional
relationships relationships
Items 5 through 7 in Table 1.2 focus on the more substantive
differences between expert systems and semantic networks. In a semantic
network, one can represent any sort of relationship between concepts in a
domain. These relationships may focus on examples, attributes,
characteristics, or any other thing the author sees as relevant. See Appendix
D for a list of commonly used semantic network relationship labels. This
flexibility, as referred to in item 5 from Table 12, allows semantic networks to
be effective for any content domain and any level of knowledge
representation for that domain. Expert systems, however, express
relationships purely in conditional terms via if-then rules (item 6 from Table
1.2). Thus, while the expert system is not as generalized a knowledge
li


representation formalism as a semantic network, it is more exact and suited
for expressing conditional relationships within a domain.
The value in combining semantic networks and expert system
knowledge representation formalisms, as done in this study, may come from
these differences in the kinds of relationships normally expressed in semantic
networks and expert systems. For instance, when users interact with an
expert system, they must answer questions, such as those in Table 1.1, to
provide information that is necessary for the system to supply a decision. The
question, "what is your ... objective?" is intended to gather data about the
concept instructional objective. The designer of the expert system believes
that the instructional objective is a key determinant of what sort of
instructional theory, if any, to use. Thus the relationship between the concept
instructional objective and instructional theory choice must be clearly defined
in one or more if-then rules before the expert system can be completed.
Semantic networks, then, may facilitate expert system creation by forcing
learners to clearly define all sorts of concept relationships in a domain prior to
specifying the conditional if-then relationships necessary for an expert system.
An expert system builds upon the values of the key concepts defined
in the semantic network and their interrelationships to make decisions. For
example, the key concept instructional objective has the values memorize,
problem solve and synthesize. The concept values are plugged into a
knowledge base of rules in the expert system that use these values to make
decisions. For instance, in the case of the instructional theory expert system,
there may be a rule that says, 'Tf your instructional objective is to memorize
12


something, then adopt a behaviorist instructional theory". This rule is
semantically represented in Figure 1.3. This rule takes die input from the
instructional objective question and the provides advice. From a semantic
network perspective, an expert system rule is a semantic network instance
that leads to a particular decision based on significant concepts.
Figure 1.3. Semantic network representation of expert system decision.
has value
imp ies

[ behaviorism |
13


Essentially, the parts of an expert system the key concepts gathered
in questions, the underlying rules and the resulting advice can all be
represented via a semantic network. Both formalisms deal with relationships
between key concepts in a domain. However, the expert system requires that
users specify causal relationships in if-then rules in order to provide solutions
to a particular question rather than simply representing any and all concepts
in a domain. The important point is that the components of an expert system
require a knowledge of the underlying conceptual relationshipswhich is
exactly what a semantic networking tool does. Thus, starting off an expert
system generation exercise by semantically networking the domain in
question may provide an explicit framework, or scaffold1 the more finely
constructed relationships of die expert system. This study is based on the
idea that building inter-concept relationships can scaffold and enhance the
knowledge synthesis activities necessary for creating an expert system and
developing problem-solving skills.
Problem Statement
This study determines whether semantic networks and expert systems
generation may be complementary knowledge representation activities.
Specifically, it examines how using a computer-based semantic networking
1 The term "scaffold" in this instructional setting is similar to its meaning at a construction
site. Just as the physical scaffold supports some object, in an instructional setting, certain
pedagogical strategies can be used to support or scaffold learning.
14


knowledge representation tool prior to expert system generation affects both
the resulting expert systems and overall domain synthesis as measured by
before and after essays. Thus the research question is:
How does a concept mapping task affect learners' ability to (1) produce
expert systems using a simple expert system shell, and (2) overall
domain synthesis?
To examine this question, a control group uses a standard expert
system shell that requires users to express knowledge relationships in if-then
rules within the shell's rule editor. In contrast, the experimental group
generates semantic networks as a "warm-up" or advanced organizer activity
prior to actually using the same expert system shell software to also generate
an expert system. So, upon completion, members of both groups have
produced an expert system. The resulting expert systems and individuals
overall domain synthesis were measured by essays written prior to the
activities and then again at their completion are evaluated to determine the
effectiveness of both treatments. Two measures were used to evaluate the
effectiveness of the treatments. First, students in both groups wrote essays
about their expert system topics prior to the studys activities and again upon
completion. Second, the expert systems that students produced were also
evaluated.
15


CHAPTER2
LITERATURE REVIEW
Introduction
Chapter one introduced the main themes and problem statements of
this study. Essentially, this study examines the effects of subjects using two
mindtools, semantic networks and expert system generation, in combination
with one another. Recall that Jonassen (1993)defines mindtools as
"generalizable tools that are intended to facilitate cognitive processing" (p.
99). I hypothesize that by building semantic networks prior to generating
expert systems, subjects will have an opportunity to do a preliminary
knowledge organization on their domains, which will ultimately produce
expert systems that show deeper processing1 and more knowledge synthesis.
This chapter builds the rationale for the problem statement from chapter one
via a literature review of both semantic networks and expert system
generation.
1 Deeper processing refers to when a learner encodes knowledge so that there are many rich
links associated with the new knowledge Visually, a concept ina semantic network that has
been deeply processed would have many links directly and indirectly related to it
Knowledge that has been deeply processed is more likely to be able to be applied to problem
solving situations. Simply stated, it is the opposite of isolated and inert knowledge.
16


Semantic Network and Concept Mapping
Semantic Networking and Concept Mapping Overview
In explaining the computer-based semantic networking tool, SemNet,
Fisher (1992) defines a semantic network as having three parts: concepts,
relations and instances. A concept is an information piece that is represented
as a word or phrase. A relation describes how concepts are connected to one
another. An instance is a concept-relation-concept unit. Figure 2.1 shows an
instance example based on the concepts Italian food and garlic. The link or
relation, hasa, indicates that 'Italian food has garlic (at least when this author
cooks it).
CONCEPT RELATION CONCEPT
Figure 2.1. Instance example using SemNet.
hi SemNet, links can be symmetrical or asymmetrical. A symmetrical
link is one that uses the same link label in both directions. For instance the
link "is opposite" as in "top is opposite bottom", does not need to be
changed to be correct for both concepts. Most links, however, are
asymmetrical. For example in the relation "instructional model has type -
17


elaboration theory", the link "has type should be inverted to "is a type of for
the relation "elaboration theory is a type of instructional model" to make
sense. See Appendix D for an extensive list of symmetric and asymmetric
link labels.
Semantic networks are closely related to another knowledge
representation formalism called concept maps. Concept mapping was
developed at Cornell University in the late 1970's by Joseph Novak and his
colleagues (1990a). Novak developed concept maps while considering
Ausubel's assimilation theory (Ausubel, 1963). Assimilation theory is based
upon the assumption that cognitive structures are organized hierarchically.
Most new learning occurs when the new information is subsumed under
existing propositions and ideas in the hierarchy. Based upon assimilation
theory, Ausubel (1963) states his most important assumption about learning,
that the single most important factor that influences new learning is what the
learner already knows. Further, for instruction to be effective, teachers
should discover the learners prior knowledge and teach accordingly. In
considering assimilation theory and its accompanying assumption, Novak
and his colleagues developed concept maps to represent the learner's current
and evolving knowledge hierarchies.
In spite of the fact that this study uses a semantic networking tool,
SemNet, this review examines both semantic networks and concept maps
because the cognitive benefits of the tools are very similar. Briefly, both tools
require that users analyze and concretely define their knowledge structures as
they externally represent their knowledge in a semantic network or a concept
18


map. Both semantic networks and concept maps define relationships via
labeled links between nodes that represent concepts. Researchers, however,
differentiate between semantic networks and concept maps. Concept maps,
according to Fisher, Faletti, and Quinn (1989) are generally smaller
(containing 20 30 concepts) and can be viewed on a single page. Computer-
based semantic networks, in contrast, may contain thousands of nodes and
thus are viewed one section at a time by moving between concepts via the
links. Fisher et al. argue that the very act of browsing through a semantic
network may further promote knowledge synthesis.
The other difference is organizational. Both formalisms use nodes and
labeled links, however concept maps are organized hierarchically starting
from one concept. Semantic nets may be heterarchically organized with
many links between concepts. In spite of the differences between the tools,
both have the possibility of promoting knowledge synthesis. The differences
concerning size and ordering are not as important as the stated similarity.
For this study, however, the researcher chose to use a semantic networking
tool rather than a concept mapping tool for the following reasons: (a)
semantic networks have the advantage of being able to support larger and
more complex networks (and thus larger and more complex domains), (b)
semantic networks avoid the limitation of strictly hierarchical representations,
and (c) there is a significant body of research on evaluating semantic
networks which does not exist for concept maps (Fisher, 1992; Fisher, et al.,
1989; Fisher, et al., 1987; Goldberg & Bach, 1991). The next section briefly
reviews the features of the specific semantic network software used. I follow
19


this with a review of the research on both semantic networks and concept
maps to show that both tools promote knowledge synthesis.
Semantic Networks Research Literature Review
Assumptions and Uses. Semantic networking is based on the
assumption that the human mind may be organized and operate in a manner
that is analogous to the linked nodes used in semantic networks
(Schvaneveldt, Durso, & Dearholt, 1987). Semantic networking and concept
mapping tools are intended to "mirror" (even if crudely) the concept linking
and construction processes that occur internally. Thus this type of knowledge
representation formalism should provide a "natural", and non-intrusive
external interface (Fisher, 1992) for representing knowledge.
It is important to note that terms such as semantic networks and
concept maps are used in various disciplines with different interpretations.
Kitchin (1994) discusses several perspectives commonly used for cognitive
maps. The interpretations vary from a literal interpretation that a cognitive
map is actually a map (or maps) that exists in the human brain to the position
more commonly espoused in the educational literature that concept maps and
semantic networks are hypothetical constructs that provide a useful way to
model how humans may internally represent knowledge.
Advocates of semantic networks and concept maps acknowledge that
the constructs may simply be hypothetical, but argue that the active
representation of knowledge structure relationships required by semantic
networking encourages a deeper encoding of knowledge that is more focused
20


on relationships than isolated facts (Fisher, et al., 1989; Reader & Hammond,
1994). Reader and Hammond (1994) break the benefits of semantic
networking into two groups: those associated with the process of producing
the network and those associated with the finished product. This research
deals with the former. The processes of organizing and establishing the
relationships necessary to produce a concept map or semantic network are
viewed as promoting knowledge synthesis and ultimately enhancing transfer
and problem-solving abilities (Allen, 1991; Fisher, et ai., 1989; Jonassen &
Wang, 1993b).
Semantic networks and concept maps have many educational uses.
Jonassen, Beisner, and Yacd (1993a) discuss them both in terms of knowledge
acquisition and knowledge representation. Within representation, there are
several more distinct applications. For instance, Markham, Mintzes, and
Jones (1994) examined the use of concept maps as an assessment tool. They
verified that concept maps differentiate between novices and experts. Maps
generated by major and non-major biology students were clearly different in
structural complexity and organizational patterns of their knowledge bases.
The finished maps can also be used to evaluate learner knowledge structures
or simply as a basis for discussion and rationalization between peers and
instructors. Novak (1990b), however, strongly supports students developing
concept maps as the best use of the maps. His observations and research led
him to conclude that "the primary benefit of concept maps accrues to the
persons who construct the maps" (p. 37). The following review of research for
both concept maps and semantic networks elaborates on this idea.
21


increased time on task
map generation
---------1----------
has max benefits
Figure 2.2. Key concepts: semantic networking and concept mapping
research.
Concept Mapping and Semantic Networking Research. Figure 2.2
highlights the key concepts from the semantic networking and concept
22


mapping research reviewed below. First, the research focuses on the
following benefits for these tools: they promote knowledge synthesis and
deeper processing and can also improve recall. Further, research shows that
these benefits are maximized when users create their own networks, rather
than browsing through completed ones. Finally, there is research to support
use of these tools as advanced organizers for other learning tasks (Tomey-
Purta, 1991; Willerman & MacHarg, 1991). These points are pertinent because
they establish that (a) semantic networks and concept maps can promote
knowledge synthesis, which can also lead to improved transfer and problem-
solving abilities, (b) network or map creation, as used in the current study,
accrues the maximum knowledge synthesis benefits, and (c) networks and
maps can be used as an advanced organizer task, as is proposed in the current
study. The research review follows.
The first studies reviewed support point (a) from abovenamely that
these tools can promote more knowledge synthesis and deeper internal
processing, and other intellectual benefits such as improved recall, than many
rote-oriented learning strategies. For instance, Fisher (1989) hypothesized
and found that SemNet engaged students in an external knowledge
representation process that encouraged parallel development internally.
Mitchell (1991) also describes research where learners who engaged in
concept mapping techniques achieved a higher level of understanding (in this
case, coherence with the expert) than those who did not use the concept
mapping strategy.
23


This implied increased depth of processing is borne out in other
concept mapping research. Reader and Hammond (1994) predicted that
providing students who are browsing through a hypertext environment with
a concept mapping tool would be more effective in terms of scores on a
posttest than those students who used a computerized note taking tool. The
researchers found that the concept mapping group did perform significantly
better on the post test. Perhaps even more interesting is that the concept
mappers, while spending no more time browsing the hypertext, did spend
more time developing their maps than their counterparts did developing
notes. This seems to indicate that relationship building requires deeper
processing than the factual nature most notes embody.
Jonassen and Wang (1993a) also found evidence to support the use of a
semantic networking task as a means of integrating, synthesizing, and
restructuring knowledge. In a series of experiments where adult learners
browsed a hypertext and were tested on their recall, and other higher order
learning such as the ability to generate analogies, the researchers did not find
any significant differences between subjects using a full-linked, and
structured hypertext with a graphical browser and subjects using an
unstructured set of nodes. A second experiment varied the treatments
slightly and still no significant differences were found. A third experiment,
however, introduced the requirement of one group creating a semantic
network to represent what they had learned from the hypertext. The learners
who created semantic networks performed significantly better than learners
who were only instructed to "study" the hypertext based materials.
24


Semantic networks and concept maps have also been evaluated for
another proposed benefit their ability to improve recall. Holley, Dansereau,
McDonald, Garland, and Collins (1979) gave students five hours of concept
mapping training in representing text passages. Subsequent testing showed
these students were better able to recall the text's main ideas, but not the
details. Berkowitz (1986) found that students that generated their own
concept maps, even when not completely and totally accurate, had a
significant recall advantage over students who did not produce concept
maps. Berkowitz's study does not address whether recall was different for
main ideas versus details, but both studies do show that concept maps
generation improved at least some sort of recall. The Holly et al. (1979) study
seems to support the idea that concept maps (as well as semantic networks)
focus synthesis activities on the broader and more general relationships in a
domain rather than the details. This in turn supports the idea that these tools
can promote the synthesis activities that help to improve transfer. Transfer is
based not so much on the details of a situation but the general theories and
principles that can be applied to another situation as well.
Reader and Hammond found that another strength of semantic
networks is effective use of time on task (Reader & Hammond, 1994). While
time on task can be misleading as learners may spend large portions of
supposed "on task" time thinking about other things, semantic network
research from Fisher, Faletti, Thornton, Patterson, Lipson, and Springs (1987)
provides more support to Reader and Hammond. Fisher et al. reported that
during early trials of SemNet, student conversations were almost entirely
25


about how to best represent the concept and its relationships, thus supporting
that the tool promotes significant time spent on the desired cognitive
activities.
The previously mentioned studies point to the intellectual and
knowledge synthesis, recall, and effective use of time on task. The next
important point to emphasize is that there is a difference, in terms of
knowledge synthesis benefits between learners using an already-created map
versus creating one for themselves (point (b) from above). Novak (1990b) in
his discussion of concept mapping, supports the knowledge synthesis claim,
but clearly states that the benefits from concept mapping accrue to the person
doing the mapping rather than a persons who simply browse a completed
map. Although a great supporter (as well as the creator) of concept mapping,
Novak expresses a fear that someone will misuse concept mapping asking
students to memorize maps rather than generate them.
Willerman and MacHarg's (1991) study supports the use of these tools
as advanced organizers (point (c) from above). They didn't ask students to
memorize maps, as Novak, had feared, but did use completed teacher-
generated concept maps as an advanced organizer for eighth grade physical
science students. Even students using completed maps scored significantly
better than die control group on a teacher generated multiple choice test that
did not include questions at the synthesis and evaluation levels. These
researchers concluded that because the concept maps were teacher generated
and thus more accurate than student concept maps, they were a better anchor
26


for subsequent learning. The significant differences, however, may be
explained via other reasons. For instance, the increased exposure to the
material via the advanced organizer may have positively affected scores.
Also, it is not dear that this strategy would be effective when performance is
measured at the synthesis and evaluation level activities from Bloom's
taxonomy (1956). In fact, Markham, Mintzes, and Jones (1994) report that
correlations between commonly used psychometric instruments such as
multiple choice tests and concept maps are rdatively low. Markham et al.
suggest that multiple choice tests do not often assess performance at the
synthesis and evaluation levels, which is of course what concept map
generation encourages.
Finally, as is the case for most knowledge formalisms, there is evidence
that the more time learners invest in using the tool, the greater the benefits.
Novak (1990b) specifically found this to be true in looking at the rdationship
between student problem-solving abilities and their use of concept maps. In
a longer term study where junior high students used concept mapping tools
over the entire school year, Novak found that the students using the concept
mapping tools outperformed their counterparts by a wide range on a test of
novd problem solving. Conversdy, other concept mapping and semantic
networking studies have hypothesized that their lack of significant
differences may have been due to not enough time spent with the tool (Fraser
& Edwards, 1985; Lehman, Custer, & Kahle, 1985). This dement of time spent
with the tool is reflected on in chapter three in the discussion of the design of
this study.
27


Summary: Semantic Networking and Concept Mapping Research.
While the previous semantic networking and concept mapping studies are
quite variable, in general they all focus on shifting from rote learning of
concepts and facts out of context to the deeper understanding required when
learners must externally represent via semantic networks and concept maps
how concepts are related to one another (Allen, 1991; Fisher, 1992; Fisher, et
al., 1989; Fisher, et al., 1987). The expected benefit of such increased depth of
processing is a more elaborate memory of the established relationships
leading to the ability to apply knowledge in new situations (Fisher, et al.,
1987). Additionally, the computer based nature of such tools as SemNet
allow users to easily and dynamically change their knowledge
representations which accommodates the need to edit an existing network as
new knowledge is constructed and synthesized. While not all studies used
the tools uniformly they showed, in general, a trend in which students who
used semantic networking and concept mapping activities significantly
outperforming those who did not.
Several points from this research review are especially pertinent to the
current study. First, semantic networking and concept mapping activities
foster knowledge synthesis and deeper processing. Secondly, this benefit is
maximized when users create their own networks or maps, as is the case in
the current study, rather than using completed ones. Finally, the research
establishes a successful precedent (Willerman & MacHarg, 1991) for using
maps or networks as advanced organizer tools. Significant results may be
even more probable in the current study as learners will create their own
28


networks rather than browse completed ones as occurred in the Willerman
and MacHarg study.
Student Expert System Development Research
Expert Systems Background and Components
After examining the effectiveness of semantic networks and concept
maps as knowledge representation systems, we now examine another specific
system student expert system development. Expert systems are special
software programs designed to simulate the reasoning process of a human
expert in a particular domain. Expert systems are a particular application of
the larger field called artificial intelligence. Artificial intelligence (AI) is a
field within computer science which studies ways to simulate via computer
hardware and software the way humans think, reason and solve problems
(Grabinger, et al., 1988). An expert system is a software program specifically
designed to simulate the way a human expert would solve a problem in a
particular domain. For instance, an expert system about instructional theories
might answer questions such as "Which instructional theory should I use in a
particular instructional situation?"
29


Figure 2.3 shows the components of an expert system. In order for an
expert system to give a user advice, users must have a means of interacting
with the system to communicate their particular problem parameters. The
user/machine interface provides this communication path. The interface asks
the user questions to help the user clarify and express exactly what they want
from the expert system. For instance, in the current example the interface
might pose the questions as listed in Table 2.1:
30


Table 2.1.
Expert System Interface Questions
Expert System User Questions
What type of instructional objective applies to your learning situation?
1. intellectual
2. verbal
3. psychomotor
4. attitudinal
What type of instructional assessment do you plan on using for your
learning situation?
1. problem-solving questions
2. multiple choice
3. short answer
4. essay
What type of instructional activities are you used to performing?
1. worksheets
2. small groups
3. lecture
The answers to these questions comprise the "current problem information"
for providing a solution to this expert system use. This current problem
information is used in conjunction with what the expert system "knows"
about the domain to provide advice to the user.
An expert system can not exist without some means of encoding the
knowledge of the expert. This information is encoded in the "knowledge
base". The 'expert editor' depicted in Figure 2.3 is used in a fashion similar to
31


a word processor to physically enter and create the knowledge base which
serves as a representation of the experts knowledge. Ultimately, the answers
users provide via the expert system user interface interact with the expert's
knowledge in the knowledge base to provide a recommended solution. The
knowledge base contains facts and rules relevant to the domain being
simulated. Facts are statements of declarative knowledge and rules are if-
then relationships. Continuing the instructional theory example, Table 2.2
represents a subset of facts and rules that may be in the knowledge base.
32


Table 22.
Expert System Knowledge Base Example
FACT: Choosing the proper instructional theory to guide instructional
development can increase your instructional effectiveness.
FACT: The type of instructional objective is a major determinant of the
proper instructional theory to choose.
FACT: The type of instructional assessment desired is a major
determinant of the proper instructional theory to choose.
FACT: The type of instructional theory you choose should depend on
your current teaching style.
FACT: Learning to implement an instructional theory that is new to
you takes significant time.
RULE 1: If (your teaching style is freeform) OR
(you have little time to learn new material)
THEN use no formal instructional theory operate as you
normally do
RULE 2: If (your instructional objectives are memorization)
THEN implement a behaviorist instructional theory.
Given the user's answers to the expert system questions and a
knowledge base of rules and facts for the domain, how does an expert system
provide a solution? The "inference engine" is the part of the expert system
that links the current users problem information to the expert's encoded
33


knowledge in the knowledge base. The inference engine takes input that
users provide via answering the expert system's prompting questions and
indexes into the knowledge base to provide solutions.
Expert System Uses. Using expert systems in the classroom is not a
new idea. Expert systems classroom applications can be classified according
to the amount of control the end user has during the experience (Jonassen,
Wilson, Wang, & Grabinger, 1993b). Earlier experiences with expert systems
focused on objectivistic uses where learners had little control. The most
common example is having learners act as novices querying the expertise in a
completed expert system. This application embodies the objectivistic learning
paradigm as learners received knowledge that has already been refined into
the expert system. The underlying assumption is that knowledge is a set of
completed entities that learners acquire. Passively using an expert system
that contains encoded pieces of knowledge supports that assumption. The
goal for the learner, then, is to have knowledge structures that mimic, as
much as possible, the expert's as it is represented in the expert system
(Jonassen, et al., 1993b). In this case the expert system acts a tutor and the
learner a tutee.
An example of such an expert system tutor is taken from Sener (1991)
and is depicted in Figure 2.4. The specific purpose of this expert system is to
tutor its users on how to classify soils. Sener clearly indicates that he hopes to
use the "colorful expert system shell environment as an interesting and
motivating medium of instruction" (p. 8). Students in an undergraduate
course on soil testing used the system to classify their soil samples according
34


to a particular soil classification scheme. Figure 2.4 shows a sample of the
questions the system asked the user and the answers the user provided. Note
that other than the measurements that the student has taken prior to using the
system, the user is the recipient of the expert system's "knowledge".
What is the percentage of the soil sample passing through the No. 10 sieve?
(The answer without the % sign, type ? mark if not known).
67
What is the percentage of the soil sample passing through the No. 40 sieve?
(The answer without the % sign, type ? mark if not known).
45
What is the percentage of the soil sample passing through the No. 200 sieve?
(The answer without the % sign, type ? mark if not known).
37
What is the liquid limit for this soil sample? (The answer without the %
sign, type ? mark if not known).
45
What is the plastic limit for this soil sample? (The answer without the %
sign, type 'NP' mark if non-plastic).
17_________________________________________________________________
Figure 2.4. Expert system acting as tutor.
Marcoulides (1988) describes a use of an expert system in the
classroom that is slightly less objectivistic. He uses an expert system as a
35


"coach" to help learners understand and use basic statistical analysis tools
such as t tests, z tests, and analysis of variance. Rather than simply having
learners use an expert system to answer specific questions about statistical
analyses, Marcoulides' system prompts them to think about each factor they
must consider in order to make an accurate decision about the statistical
analysis to use. He found that such a coaching and reflection increased
accuracy of selecting the appropriate statistical tool and retention of the
material as measured on the final examination. His findings are congruent
with other research that indicates activities that require self reflection
encourage retention and transfer (Schon, 1987).
Grabinger and Pollock (1989) also conducted a study using expert
systems in a more student-thought provoking way. They used an expert
system as a self reflective feedback tool for students developing instructional
media products. The expert system was available during labs for students to
get immediate and thorough feedback on the quality of their projects. Expert
system users produced final products slightly, but not significantly, better
than students who received feedback directly from the instructor. However,
in a subsequent media evaluation task, expert system users generated
significantly more new evaluation details than did the externally generated
feedback group. Thus, students who used the expert system feedback
mechanism were better able to apply the expert system criteria from one
project to another, supporting the idea that the reflective nature of self
generated feedback may have encouraged transfer. It is critical to note that
these researchers do not attribute the success of the reflection to the medium
36


itself (i.e. the expert system) but rather that the system provides a structured
way to assure that users examine a particular criteria before moving on.
The studies just reviewed have students interacting with completed
expert systems in a tutee role. Constructivist uses reverse these roles. Here,
die expert system acts as a tutee and learners tutors it with their knowledge
representations. Just as Novak posits that the greatest benefit of concept
maps accrue to those how develop them, Jonassen, Wilson, Grabinger, and
Wang (1993b) realized that those developing the expert system benefited the
most from that process. Codifying the knowledge into the form the expert
system requires forces the developers to engage in knowledge synthesis and
explicitly state the relationships of the concepts in the knowledge domain.
Morelli (1990) describes the constructivist use of expert systems in the
classroom in a slightly different way saying it is a way to build a precise model
of the target domain. The resulting knowledge base model is intended to
represent as much as possible that of the experts'. Both sets of researchers,
however, advocate turning the expert system shell over to the learners to
create their own expert systems and thus allow them to experience the
knowledge representation, synthesis and model building benefits that only
the experts experienced before.
To summarize, expert systems can be used in a variety of ways in class
room settings. The previous review shows that the range of uses is from a
completed expert system acting as a tutor to learners who wish to receive
pre-processed information from that expert system, to a use where the
learners are acting as experts who are "teaching" the expert systems by
37


!
creating knowledge bases on a particular topic. It is proposed that the active
nature of learners acting as experts building their own expert systems can
help these learners organize and synthesize their knowledge in that domain.
In order for non-computer programmers to have the capability to create
expert systems, they must use an expert system shell. The next section
addresses expert systems shells.
Expert System Shells. An expert system can be created via a high level
programming language such as Lisp, PROLOG or C, or via an expert system
shell (Grabinger, et al., 1988). Programming languages have the advantage of
providing maximum flexibility in terms of how many rules and facts a
knowledge base may contain and the also the complexity of the inference
engine that accesses the knowledge base to provide solutions. The drawback,
of course, of programming languages is that one must know how to program.
These languages have dauntingly complex syntax and semantics that many
potential expert system creators, which included many educators, may not
wish to learn.
Expert systems shells have opened the door to allow non-
programmers to easily create expert systems. Just as an expert system
contains a rule editor for building a knowledge base, so does an expert
system shell. An expert system shell allows people to create an expert system
by representing the domain's knowledge base within the shell's rule editor.
Users input the facts, rules and user questions pertinent to their knowledge
domain and the problems they wish to solve with this expert system. The
expert system shell then provides all other software portions of the expert
38


system including the all-important inference engine. While expert system
shell users must still learn some syntax for stating the rules and facts, it is
much easier than learning the syntax of a full-blown programming language
in order to create expert systems.
Most expert system shells allow input of the knowledge base in one of
two ways (or in some cases both). A commonly used method is to specify the
relationships of pertinent concepts and facts in a domain via if-then rules.
Shells that use this method are called rule-based expert systems shells (see
Table 2.2). While the particular syntax from one shell to the next may vary, in
general an if-then rule is in the form of IF condition THEN decisionl ELSE
decision2. Also operating under boolean logic, the condition may be actually
several conditions linked together via AND or OR connectors. Rule 1 from
Table 22 illustrates the use of an OR connector.
Another means of representing a knowledge base in an expert system
shell is to enter a series of comprehensive examples that represent the factors
and solutions of the problem domain. The shell takes the examples and then
induces rules that can be used to solve the problem. Figure 2.5 shows a
sample knowledge base from MacSmarts (Corporation, 1988), an example-
based expert system shell. This expert system provides advice on the type of
instructional media to use in a learning situation.
39


distribution motion outcome media
wide no affective print
wide yes affective dramatic TV
wide no verbal information dramatic TV
wide no intellectual skill informative TV
single yes intellectual skill computer
single yes verbal information print
single no affective slide-tape
single no intellectual skill slide-tape
Figure 2.5. Mac Smarts knowledge base using examples.
Expert System Generation Benefits and Limitations
What does it mean for novices to create expert systems as a knowledge
synthesis activity? Using the parts of the expert system as just explained and
represented in Figure 2.3, it means that novices or learners, rather than
"experts", create the knowledge base that the expert system solutions are
based upon. Jonassen et al. (1993b) report that the greatest benefits in terms
of knowledge acquisition, synthesis and problem-solving skills accrues to the
individuals who develop the knowledge base. Other researchers (Morelli,
1990; Roberge, 1990; Trollip & Lippert, 1987; Wideman & Owston, 1993) agree
citing the following of benefits:
1. precise mastering of target knowledge due to unforgiving
syntactical nature of shells,
40


2.
increased enthusiasm for learning,
3. development of independent learning skills,
4. demonstrated active and aggressive pursuit of knowledge
during knowledge engineering process, and
5. deeper knowledge processing and learning even when only
simple systems were developed for complex domains (see
description of Trollip and Lippert (1987) below).
The latter benefit is the one most frequently touted and tested in expert
system generation research. This knowledge organization involves linking
new ideas into previously existing schemata. Jonassen and Wang (1993b)
define learning as the building of new knowledge structures by constructing
new nodes and connecting them with existing nodes. Unfortunately, some of
the learning that often occurs in schools and in industry is not of this active
and constructive sort, but rather of a rote nature.
While rote learning may promote memorization (usually temporary) of
facts, today's workers need to be able to synthesize an ever increasing number
of knowledge sources in their decision making processes and have
independent learning skills in order to tackle the learning and problem-
solving situations encountered frequently. These problems are often ill-
structured and the solvers must determine the problem parameters as well as
a means to solve it. Rote learning does not address these metacognitive
needs. Further, when students do learn to problem solve it is often via
formulae that they do not understand and therefore can not transfer to new
situations. Generating expert systems as well as generating semantic
41


networks both promote the skills that encourage learners to understand how
and why they reach a solution as well as the solution itself.
Even with these proposed benefits, readers may wonder what the
limitations of constructing expert systems are. Morelli (1990) discusses the
basic skills users must possess in order to successfully use most expert
systems shells: basic computing skills (i.e. keyboarding and word
processing), oral and written communication skills for interviewing experts,
formulating questions, working in groups, and logic and reasoning skills for
building relationships between items in the knowledge base and for
formulating if-then rules for the shell. Beyond these basic skills, one must
also choose an appropriate topic for expert system development. The topic
must be defined well enough to pose a specific question to answer, but also
should be broad enough to include complex relationships that learners must
synthesize in order to represent within the expert system. (See Grabinger,
Wilson, and Jonassen (1988) for more details.)
Expert System Generation Research
A review of the literature supports Jonassen's, et al. (1993b) and Trollip
and Lipperts (1987) position that developing expert systems using expert
systems shells enhances the knowledge representation process, encourages
knowledge construction and synthesis and develops higher order thinking
skills. Studies vary greatly within several factors including expert system
shell software used, knowledge domain represented, supporting activities
accompanying the expert system generation, degree of higher order thinking
42


skills necessary for synthesis of the domain, and the age of the subjects.
Given that the current study proposes to use semantic network generation as
a warm-up activity for generating expert systems, the following research
descriptions focus on the activities that supported the expert system
generation.
Knox-Quinn (1988) describes ways expert systems can be used in the
classroom and reported initial research on using a Macintosh-based expert
systems shell for student knowledge base development by junior high
students. Knox-Quinn states that her purpose was to explore the feasibility
of using expert system generation as a classroom task that can engender
deeper knowledge processing. Given the exploratory nature of the study, she
allowed students to chose expert system domains they were interested in and
more importantly were already familiar with rather than assigning a topic
that would require student research. For instance, a group of students
developed an expert system to advise men and women on what sorts of shoes
to buy. The knowledge base they constructed came from many sources and
included rules on foot shape, health and style. Knox-Quinn observed that the
quality of the resulting expert systems was affected by how well students
represented their knowledge base rules prior to entering them into the
system. Students used a variety of means to organize their knowledge
including decision trees and flow charts, but still in some cases required
further coaching in order to achieve the knowledge organization necessary
for expert system generation. These findings support the importance of
helping learners organize their knowledge prior to expert system generation.
43


Tamashiro and Bechtelheimer's (1991) conducted similar research with
young school children. Learners used an example based shell developed by
Tamashiro called Knowledge Works. The researchers found that with proper
accompanying scaffolding that expert system development can work on
simple knowledge domains with elementary students. Teachers used die
expert system shell in a variety of ways. Some teachers used the a completed
expert system as a student tutor. Others allowed students to generate expert
systems in conjunction with lessons on outlining and other knowledge
organization topics. As teachers witnessed the benefits of expert system
development and observed how well their students grasped and enjoyed the
activity, some chose to introduce other knowledge formalisms such as
semantic networks. While Tamashiro and Bechtelheimer do not report any
formal research results, anecdotal observations showed students as young as
second graders learned to follow and predict the reasoning paths used in the
expert systems and gradually move from concrete to abstract problem
solving. Even though this progress required significant coaching and
modeling from the instructors, it is evidence that these tools can promote the
kinds of higher order thinking skills previously discussed.
Wideman and Owston (1988) conducted a qualitative study where 37
seventh graders working in small groups developed expert systems to
classify living organisms. Students used a rule-based expert system and were
instructed on the shell via demonstrations and practice development of expert
systems. While teachers did present some information in class on biological
classification (via direct instruction and examples), groups were coached on
44


finding their own information sources and constructing the required
knowledge interrelationships. To help structure their knowledge, students
were required to develop charts representing their knowledge of the domain
prior to writing tire formal rules for the expert system shell. All but one
group of the seven was able to complete a knowledge base, but various
problems that are typical of novices in a domain emerged during the process.
One particularly noticeable problem was how to use a "variable" to hold a
value that would later determine an organism classification. Rather than
defining a single variable or parameter that represented a factor such as
whether a Pisces class member was in the sturgeon or salmon family, students
wished to have separate parameters for each family. Ultimately, students
made significant progress in their dealings with both the subject matter and
the knowledge base construction. The researchers concluded that with the
proper amount of time invested and appropriate coaching and modeling,
students at this grade level could produce expert systems and engage in
knowledge structuring and other cognitive tasks not normally demonstrated
at that age.
In a more recent study, Wideman and Owston (1993) compared the
cognitive skill gains and transfer for three groups of eighth grade students.
One group developed weather prediction expert systems; another group used
software designed to develop problem-solving skills, and a control group
received no special treatment. The students in the expert system group were
instructed via a practice expert system on how to use the rule-based shell
prior to actually developing their weather prediction systems. While there
45


were no significant differences between die groups, there was an interesting
effect for the subset of students who scored higher than the grand median on
a standardized test of abstract reasoning. Within that subset, there were
significant main effects for the expert system group on tests of formal
reasoning and transfer. The researchers also concluded via qualitative
observations that the expert system development task was the most cognitive
demanding of the three activities as it required modeling, coaching, practice
and monitoring before the average student could successfully abstract file
pertinent factors into the relationships that an expert system requires.
Additionally, Wideman and Owston speculated that their overall lack of
significant differences among groups may have been due to insufficient
treatment time (approximately sixteen hours) to develop growth in the skills
they measured (formal reasoning and transfer tasks).
The next two studies share something in common with the Wideman
and Owston (1993) study just described. Just as Wideman and Owston
compared expert system generation and a general problem-solving
formalism, Jonassen's (1993) study compares two knowledge representation
or "mindtools". Jonassen defines mindtools as "generalizable tools that are
intended to facilitate cognitive processing" (p. 99). The purpose of his study
was to compare the effectiveness of semantic networks and expert system
generation. Subjects used these tools to represent their knowledge on
analyzing learner characteristics and relating them to learning outcomes and
instructional techniques. Jonassen expected to find that the expert system
generation subjects would develop more causally-oriented knowledge
46


structures because of the if-then format of the expert system shell used, and
that the semantic networking formalism would result in knowledge
structures that more closely resemble a representation of the expert's
knowledge. The study used pre- and posttest Pathfinder Networks
(Goldsmith, Johnson, & Acton, 1991) to assess the knowledge structures of
graduate students in a course on instructional technology. During the course,
subjects had either used semantic networks or expert system generation as a
means to synthesize course content. Results from evaluating the post test
Pathfinder networks showed no significant differences between the two
groups but did show better knowledge organization than those who did not
use either formalism.
Like Wideman and Owston, Morelli's (1990) junior high subjects also
generated expert systems for a classification system. Unlike Wideman and
Owston, however, Morelli's study emphasizes to a greater extent the process
of gathering the knowledge to populate the expert system's knowledge base.
In this study, students were required to actively gather knowledge base
information from a subject matter expert who was carefully coached to not
provide all the synthesis and relationship information that would make the
rule formulation too easy. Morelli wanted his students to struggle with the
hardest part of the activity synthesizing the knowledge into a meaningful
body. As with several of the reviewed studies, Morelli's reports his study in a
case study format and does not include any formal evaluation but focuses on
the students processes rather than end results. The point Morelli makes is
that the knowledge engineering process is a good model for how students
47


learn science. The knowledge engineer model requires an active and self-
directed learner (the knowledge engineer) to interact with a qualified and
articulate expert (or teacher in a classroom situation). The learner is put in
control of the learning situation and it is up to them, with some modeling
from the "expert", to focus on the portions of the domain that will integrate it
into an understandable model. These activities encourage building
conceptual and causal relationships and deeper processing. Both of these
activities can have a positive effect on problem solving in familiar and
unfamiliar situations which is an important characteristic for a scientist.
Lippert (1988b) conducted a qualitative study gathering 30 hours of
data each for four honors college freshman students developing expert
systems in projectile motion. The students themselves, while none felt they
developed an expert system that completely represented their domain
knowledge, remarked that the value of the project was in their organization of
knowledge, and ultimately it was the structure of their knowledge and not
the content that changed. Knowing "when and why" to apply knowledge
seemed to be a new and important discovery for Lippert's students (p. 33).
Students also reported that their problem-solving strategies shifted away
from rote manipulation without understanding to strategies that were based
on the underlying theories and conceptual relationships within the domain.
While Lippert's study was preliminary and intended to determine how more
quantitative studies in the future might be constructed, it is this structural
change that may lead to enhanced problem-solving skills and increased
transfer, which is in turn the reason for the current study.
48



One of the significant aspects of Lippert's (1988b) and Jonassen's (1993)
studies is that students were required to engage in more prediction level
knowledge synthesis than some of the other studies that dealt with
classification (Morelli, 1990; Wideman & Owston, 1988). hi other words, the
knowledge synthesis in Jonassen's and Lipperts studies had students
answering a "what will happen if question while the classification studies
dealt with the concrete "what is it" question. While even studies that had
students tackle classification problems seemed to enhance knowledge
synthesis, the classification domains are less ambiguous and more clearly
have right or wrong results than the so-called "what if' domains. Jonassen
(1993) states that when formulating the if-then rules for an expert system
shell, learners must explicitly "articulate the causal reasoning that is implicit
in the decision-making in the content domain" (p. 100). Further, the more
complex and ambiguous the domain, the more causal reasoning is necessary
to represent it. Thus the significance of Lipperts research on projectile
motion, an inherently complex domain, is enhanced.
Trollip and Lippert (1987) conducted a qualitative study which asked
college students in a course on intelligent CAI to work in small groups to
prototype an expert system to help CAI developers design the layout of their
title screen displays. Several project characteristics emerged. The project
generated a large amount of peer interaction; students had to negotiate about
what questions to ask experts, what factors to include in the knowledge base,
and how to approach this purposefully ill structured assignment. The task
also forced students to hone their communication skills in order to extract
49


knowledge from experts. And finally, the analysis of the subject matter
material required to develop the rules and relationships for the expert system
shell was so deep and so incisive that learners developed a greater
comprehension of the domain. There is a certain irony in this last statement
in that the project required that students develop simple expert systems (due
to time and reasoning constraints) about a complex domain. The irony is that
in spite of the feet that the resulting expert systems were simple, students still
engaged in deep knowledge processing in order to produce these systems.
Recall that lipperts students expressed the same irony as their expert
systems did not fully represent their grasp of the domain (Lippert, 1988b).
The potential power of this mind tool is impressive given that even with
"incomplete" expert systems, learners experienced increased knowledge
synthesis and changes in problem-solving strategies.
These studieswhile ranging in many factors such as subject age,
knowledge domains, expert systems shells, treatment times and evaluation
methods all used expert system generation which some degree of success
to promote knowledge synthesis and deeper processing. Promoting these
activities during the learning process may in turn, as some of these studies
showed (Jonassen, 1993; Lippert, 1988b; Wideman & Owston, 1993), increase
knowledge transfer and in general develop the metacognitive skills necessary
for more effective problem solving. Many of these studies indicate that
learners used other formalisms such as trees and maps to help organize their
knowledge. None of the studies, however, formally look at the effects of
using (or not using) another formalism in conjunction with expert systems
50


generation. This research attempts to address this gap by looking at how
semantic networks work together with expert systems generation. In the next
section, the trends, patterns and gaps including the one just mentioned
in this literature and the semantic networking literature review are discussed
to form a rationale for the current study.
Synthesis of Two Research Areas
Taken together, what commonalties, overlap, and gaps do these
research areas have? It is clear from the literature reviews of semantic
networking and expert systems generation that both knowledge
representation formalisms can be used effectively for encouraging deeper
level processing and knowledge synthesis (Allen, 1991; Fisher, 1992; Jonassen,
et al., 1993b; Knox-Quinn, 1988). Other researchers further point to a
relationship between deeper processing and knowledge synthesis and
increased learner transfer and problem-solving skills (Jonassen, 1993; Lippert,
1988a; Wideman & Owston, 1988).
The central idea behind these tools is based on an understanding of
how schemata and problem-solving information relate. As learners
externally represent a domain via either expert system or semantic network
generation, they also internally strengthen and add to existing knowledge
schemata. In essence, they generate new knowledge by attaching it to the old.
Wittrock (1974) describes this as the "generative" nature of learning. This
generative process helps to synthesize and strengthen the schemata.
Schemata, however, can store more than simply content but also knowledge
51


about specific problems including procedures that may be used to solve that
problem. So, using tools such as semantic network and expert system
generation that can support learning and schema development can in turn
support this schematic representation of the "problem space" which
ultimately can help learners be more effective problem solvers (Tomey-Purta,
1991, cf).
Given this prior research and background on how these processes help
promote desirable learner characteristics such as knowledge synthesis, what
questions are left unanswered and how can the prior research be advanced?
Recall that the current research examines the use of two formalisms, expert
system generation and semantic networks, to promote knowledge synthesis
beyond what may be promoted by using expert system generation alone.
There is some precedent for combining knowledge formalisms although the
specific effects of doing so have not been reported on in the research.
Jonassen compared these two formalisms and found no significant differences
(Jonassen, 1993). Also, several of the expert system generation studies
reviewed told how learners, some on their own accord and some via
instructor guidance, used various other knowledge formalisms to help
represent the relationships and structure of a domain prior to representing it
within the expert system shell. Lippert (1988b) describes how learners used
both hand drawn decision tables and decision trees to represent their
domains. Knox-Quinn's (1988) and Wideman and Owston's (1988) students
also used decision trees and flow charts to organize their knowledge bases.
All six of Trollip and Lippert's student groups reported that they voluntarily
52


used pictorial representations, trees, and graphs to help them organize their
knowledge (Trollip & Lippert, 1987). Also, a sixth grade teacher who saw
how well her students mastered a simple expert system shell introduced
other "tools" such as semantic "webs" and tree diagrams (Tamashiro &
Bechtelheimer, 1991). All of these researchers, instructors and learners
recognized die potential benefits of students using various formalisms, some
more familiar and comfortable to use than others, to maximize the knowledge
synthesis experience. This supports the idea of using semantic networks in
conjunction with expert system generation as proposed in this study.
Another researcher reports on an even more novel association of
semantic networks and expert systems shells. In Kuczora's (1990) discussion
of expert systems, he describes knowledge engineering as the weakest link in
the process. While not an educational application of the two formalisms, he
describes a system, VEGAN (Visual Editor for Generation of Associative
Networks), that makes the knowledge representation easy and intuitive.
VEGAN is an interactive graphical editor that allows knowledge engineers to
generate and manipulate semantic networks to represent a knowledge
domain. VEGAN then translates the semantic network into if-then
production rules. VEGAN'S existence supports this study's idea of using a
semantic networking exercise as a "warm-up" activity to expert system
generation in two ways. First, by providing a semantic networking interface
to an expert system shell, VEGAN shows a dear precedent for using semantic
networks in dose assodation with expert systems shells. Secondly, VEGANs
designers propose that the semantic networking formalism can affect the
53


expert system outcome, and directly hypothesize that users tend to work
more effectively with the semantic net representations than the production
rules because the "rule encoding mechanism lacked intuitive appeal in
comparison with the graphical semantic network editor" (Kuczora & Eklund,
1990, p. 171-172).
Given this precedent for using semantic networks or concept maps in
conjunction with expert system generation, and prior research results
showing that students preferred to use a variety of knowledge representation
formalisms, the next question is how can semantic network development
support and even enhance the knowledge generation and synthesis process
that can occur during expert system generation? La very general terms,
Novak states that (1990b) concept mapping used with other educational
strategies can lead to superior achievement, but this doesn't give us a specific
suggestion on what role they can play in expert system generation. The
following possibilities are suggested.
First, developing a semantic network could serve as an "warm up"
knowledge organization activity for developing the expert system rules and
the overall structuring of the knowledge domain. An advanced organizer is a
means of conveying and bridging the gap between what is already known
and what is to be learned (Ausubel, 1978). An advanced organizer is meant to
activate pre-existing schema in a particular domain so that new learning may
be more effectively synthesized and tied to the old. This serves to tie new
learning in with what is already known and thus strengthen both the new
and the old. Tomey-Purta (1991) suggests that presenting concept maps are a
54


good way to provide an advanced organizer. However, in keeping with the
idea of giving the students the tools to organize their own knowledge, this
research suggests having the students develop concept maps as their own
advanced organizers for ultimately developing an expert system in the
domain.
Second, Willerman and MacHarg (1991) used teacher-completed
concept maps as advanced organizers for students studying science. Students
did not create concept maps but rather copied them. Even with this level of
concept map usage, which goes against Novak's recommendation that
students will benefit most from creating their own concept maps (Novak,
1990b), students outperformed the control group on the exam.
Third, given this evidence for using concept maps and semantic
networks as advanced organizers, what other reasons are there for them
supporting expert system generation? A concept map or semantic network
requires that learners specify relationships within a knowledge domain.
Unlike an if-then rule for an expert systems shell, it does not require that
learners specify the conditions under which those relationships hold true.
Asking students to jump immediately to encoding such if-then rules may be
too large a leap for many learners. So the third reason that semantic
networking may support expert systems generation is that engaging in the
semantic networking exercise first can serve to organize the knowledge and
then allow learners to use that organized knowledge to formulate if-then
rules.
55


Fourth, semantic networking tools support the dynamic nature of
knowledge construction. Learners developing semantic networks can study
their networks and decide to create further instances or change something
already represented. Given the dynamic nature of our internal knowledge
structures and the constructivistic nature of learning (Jonassen, 1991), the
ability to easily edit or change represented knowledge structures is important.
Knowledge constructions may change due to new inputs, or in the case of a
graphical semantic network, from the user actually seeing their
representations (Fisher, et al., 1987). This ability to easily "edit" knowledge
structures further promotes a successful expert system development
experience by creating the clearest and most accurate picture of the learners'
knowledge representation.
A final proposed reason that semantic networking exercise may
improve the quality of the expert systems is that the semantic networking tool
is easier to use than the expert system shell. Fisher et al. (1987), while
discussing the knowledge representation and depth of processing benefits of
semantic networks, explain that
The theoretical benefits described above are a unlikely to be widely
realized as long as the process of drawing networks is inordinately
tedious, cumbersome, and time consuming. The computer-based
semantic network software alleviates tiiis constraint permitting the
user to concentrate almost entirely on the structure of the knowledge
rather than the mechanics of the representation, (p 10-11)
Because users find the SemNet program (and interface) easy to use, they can
focus their cognitive energies on the knowledge synthesis task rather than
56


how to use the tool (Fisher, et al., 1987). Figure 2.6 metaphorically represents
the relative cognitive load for using SemNet versus an expert system shell like
EXSYS. If one thinks of a user's finite amount of cognitive effort to invest in a
target task like the finite amount of change in one's pocket, then Fisher et al.
argue that the semantic networking interface allows users to apply a larger
portion of that "pocket change" towards the target task rather than towards
manipulating interface of the tool.
Semantic Networking Expert System Shells
Figure 2.6. Cognitive processing available for target task using semantic
networks and expert system shells.
This research study uses semantic networks and expert systems in
conjunction with one another. To examine this question, two groups
complete expert systems using a rule-based expert system shell. One group,
however, also creates semantic networks prior to generating their expert
systems. This second group, the semantic networking group, uses the
semantic networking activity as a "warm-up" or advanced organizer prior to
57


actually using the same expert system shell interface combines the formalisms
discussed above with a semantic networking interface for creating an expert
system. So, upon completion, members of both groups have produced an
expert system. The resulting expert systems and individuals' performance on
a preliminary and ending essays are evaluated to determine the effectiveness
of both treatments. This leads to the following research hypotheses. Note
that all assessment issues implied by these hypotheses are completely
explained in Chapter 3.
Research Hypotheses
Learners in the semantic networking group will produce better expert
systems than those in the non-semantic networking group, where better is
defined by an expert system set of assessment criteria (completely
explained in chapter 3) based upon number of rules, factors and decisions
in the expert system.
Learners in the semantic networking group will perform better on a the
ending essays than those in the non-semantic networking group. Ending
essays are evaluated on their ability to represent completely the
complexity of the expert system topic domain.
Within the semantic networking group, learners who generate more
expansive/complete concept maps (# of nodes, linked-ness) will produce
better expert systems where better is defined by the expert system
evaluation criteria.
Summary
This chapter discussed the general background, and prior research for
the two knowledge formalisms, expert systems generation and semantic
58


networks, used in this study. The research for both of these formalisms
shows that learners can engage in deeper processing and knowledge
synthesis than when using more rote, memorization-oriented methods of
instruction. The variety of ways instructors and researchers supported and
scaffolded the expert systems development are of particular interest to the
current research. Several studies showed both instructors and the learners
themselves choosing to use more than one knowledge representation
formalism to organize their knowledge prior to generating the if-then rules
for the expert system. This prior use of several knowledge representation
formalisms supports the formal evaluation of using semantic networks as a
knowledge organization "warm-up" activity before generating expert systems
using a shell. Other reasons for engaging in this study are based on the idea
that having completed a semantic network or concept map of the knowledge
domain could actually enhance the expert system generation process because
the concept map serves to supplement short and long term memory
concerning the domain. With the knowledge externally represented in the
concept map, learners can focus more of their finite cognitive capacity on the
process of generating the causal relationships that an expert system requires.
59


CHAPTER 3
RESEARCH DESIGN
Introduction
This chapter describes the study's methodology. Given the diverse
nature of the study participants and the expert systems developed,
particular attention is paid to assessment instruments used in the study.
Variables
Independent Variables
There was one independent variable in this study: the treatment
variable. The treatment group variable has two levels: semantic
networking group or non-semantic networking group. Subjects in the
semantic networking group produced semantic networks in conjunction
with expert system generation, while subjects in the non-semantic
networking group only produced expert systems. Both groups used the
same rule-based expert system shell editor to individually produce an
expert system for a knowledge representation task.
Dependent Variables
60


The dependent variables are the evaluations of the expert systems
and semantic networks produced by the participants, and each
participant's performance on preliminary and ending essays analyzed to
determine learner's changes in knowledge in the domain. See the
assessment section in this chapter for specific information the specific
dependent variables produced from each of these assessment instruments.
Method
Participants
The participants were 30 undergraduate education students at the
Pennsylvania State University enrolled in a course on the use of
technology in educational settings. While gender was not an independent
variable in this study, the gender distribution was typical for
undergraduate education courses; of the thirty subjects, 19 were women
and 11 men. Subjects participated in the study as part of the normal
course requirements. On the first day of the semester, in order to both (a)
form groups in a pseudo-random fashion, and (b) allow students to "mix"
and meet one another, subjects arranged themselves in a circle
alphabetically by their first names. To form the two groups, the instructor
and I split this circle down the middle; one half became the semantic
networking group and the other the non-semantic networking group. See
Table 3.1 for subjects initial distribution between the two treatment
groups.
61


Table 3.1.
Subjects Categorized by Treatment Group
SemNet___________Non-SemNet_______Total
15 15 30
Study Materials
Materials used to conduct this study included the software
packages for developing semantic networks and expert system shells, and
instruments for evaluating the resulting expert systems, semantic
networks and knowledge synthesis essays.
Software. SemNet was chosen as the semantic networking tool
software. SemNet is a Macintosh program that enables users to
graphically define concepts (as nodes) and the relationships between
concepts as labeled links (Fisher, 1992; Fisher, et al., 1987).
SemNet supports creation of a semantic network via instances. An
instance is a concept-relation-concept unit. Figure 3.1 shows a sample of
several instances represented in SemNet. In this example, "instructional
theory" is the central concept; it is linked to the concept "schema theory"
via the relation or link "has instances".
62


connectionism
schema theory
systems
motivation
has instances
Constructional TheonT> inf1uences
paradigms
models
practice
has attributes
________i________
principles
hypotheses
makes predictions
definition
Figure 3.1. SemNet instance example.
Like a word processor, SemNet is a content-free tool. Thus users
can represent any content they choose within SemNet. SemNet was
chosen for its relative ease of use. Fisher reports that the software can
usually be mastered within about an hour (Fisher, 1992). Users are not
63


forced into any particular order for creating their networks. They can
begin with any combination of concepts, relations and instances.
The software also contains features that help users to create better
networks. For example, when a user defines an instance such as the
"instructional theory has instances schema" shown in Figure 3.1,
SemNet automatically prompts the user to create the relation link in the
other direction if it is not a symmetrical link. In this case, the inverse
relation for "has instances" would be "instance of. Features such as this
help users to have a more complete set of links available as they create
their networks. Also, since SemNet prompts users with a list of created
relations when one is needed, having created both "directions" of an
asymmetric link and being reminded of those links via SemNet's prompts,
may prevent users from creating other links with the same meaning.
The expert system shell, EXSYS (EXSYS,) is the other software
package used in this study. EXSYS was chosen for its balance of
functionality and ease of use. Achieving the functionality side of this
balance means that a shell must be capable of representing the rules and
factors needed to make decisions in a complex knowledge domain.
However, this functionality must not be at the expense of ease of use. If
the shell is so difficult to use that learners spend all their energy on the
software mechanics, then they will not have as much cognitive effort left
for knowledge building, which is the ultimate goal of the activity.
Creating an expert system using EXSYS revolves around three main
components: qualifiers, choices, and if-then rules. Qualifiers are
64


statements that are completed by choosing from several provided values
associated with the statement. In essence, a qualifier is a means of
gathering data from file expert system users about a factor that is pertinent
for making decisions in that expert system's domain. For instance, the
sample expert system that accompanies EXSYS recommends a particular
new car to purchase based on several factors considered pertinent in that
domain. The expert system author must determine the factors needed in
the domain and create an EXSYS qualifier to gather user data about each
of these factors. In Figure 3.2, one qualifier is "driving on unimproved or
dirt roads will be", and the values are "frequently", "never", or
"occasionally". This qualifier addresses the factor of frequency of driving
on unimproved roads. Qualifiers in EXSYS expert systems have two
purposes. First, they are used to gather data about factors that are
necessary for the expert system's domain. Secondly, because qualifiers are
presented to the final user of the expert system, they constitute the user
interface of the completed expert system. Clearly phrased qualifier and
qualifier values can allow end users to more accurately provide data for
the expert system decision making process.
Choices are the second part of an EXSYS expert system. Choices
are simply the possible decisions or pieces of advice that the expert system
can produce. Depending on the expert system software used, what EXSYS
calls "choices" may in other packages be called "goals" or "advice" or
"decisions", hi the sample expert system that recommends a type of car to
65


purchase, the choices are particular makes and models of cars such as:
Saab 9000, Ford Escort, or Honda Accord Sedan.
The final and most difficult portion of an EXSYS expert system is
the if-then rules. All rule-based expert system shells require users to
understand and be able to construct logical if-then rules. There is a
difference amongst expert system shells, however, in how many detailed
syntactic rules users must remember. Some shells, for instance, require
users to remember and use shell keywords in a restricted manner, in
addition to punctuating shell if-then rules in certain ways. While these
syntactical rules can certainly be mastered and these shells used
effectively (many research studies have in fact used these shells (Jonassen,
1993; Lippert, 1988a)), the course's professor, and I wished to minimize the
amount of cognitive effort students applied towards the mechanics of the
shell. I hoped that this minimization would both maximize the cognitive
effort subjects spent on constructing the rule relationships for the content,
and increase subjects' confidence in their ability to produce expert
systems.
66


IF:
Driving on unimproved or dirt roods will be frequently done
or Vinter driving in snow is frequent
THEN:
Driving in situations with difficult traction is frequent
KH

m
5
IF Part
0 THEN Part
0 ELSE Part
Insert
AND OOR
Q New OR
OK
Change
Delete
And/Or
*
Cancel
Prev
Next
Note
Reference
Name
Figure 3.2. EXSYS rule creation screen.
The most important way EXSYS keeps users from having to spend
cognitive effort on memorizing keywords and syntax is via its support of
if-then rule construction. Users click on radio buttons (see Figure 3.2)
such as "qualifier", "choice", "if', or "then", to choose the portion of the if-
then statement they wish to construct. EXSYS guides users through
making selections to fill in the required portions of the statement. EXSYS
then takes the user's selections and automatically combines them with the
required key words to create the accepted syntax for EXSYS if-then rules.
67


Finally, the course's professor and I chose EXSYS because it
consistently implements Macintosh interface standards. Since the
students used other Macintosh software packages in the course, the
course's professor thought this consistency would help learners become
proficient with the expert system shell.
Assessment Materials
Assessment materials were designed based on previous research
using expert systems and semantic networks (Faletti & Fisher, 1991;
Fisher, et al., 1989; Jonassen, et al., 1993b; lippert, 1988b). Figures 3.3,3.4,
and 3.5 list the criteria for evaluating the preliminary and ending essays,
the semantic networks and the expert systems, respectively. All portions
of every tool produce numeric data. Most of the numeric data are tallies
of particular occurrences in the data (for instance, the number of concepts
used in the semantic networks). Each set of assessment criteria and the
specific data items they produced are discussed below. For a discussion of
validity and reliability of these measures, see the following section,
Reliability and Validity.
Essays. The essay assessment criteria (see Figure 3.3) gathered data
on the attributes students had been asked to focus on in their preliminary
and ending assignment description. Use of this activity before and after
the subjects generated expert systems was to allow: (1) the subjects to
express in a less structured way than required by the expert system their
understanding of their content areas, and (2) the researcher to see if their
are significant differences in the essay data attributes from preliminary to
68


ending essays. Both the professor and I knew that the learners would
need to understand the factors that were critical to decision making in
their expert system topic domain, the possible choices or decisions their
expert systems might produce and the relationships among these factors
and the choices (i.e. given a set of factors, which choice makes sense in this
domain). Thus the essays were evaluated for counts of the factors, choices
and relationships mentioned in each essay.
A factor was defined as any concept mentioned in the essay that the
author indicated was needed for making a decision in their expert system
domain. Specific examples of factors mentioned in the essay text were
also counted as a factor, but only if the general factor that the example
represented had not already been counted. Choices were any concept or
action that the author indicated could be an outcome or decision of their
expert system. Finally, relationships were defined as either relationships
between factors and choices, factors and factors or choices and choices
(although the last option never occurred).
Precedence for analyzing text in this way comes from "content
analysis" studies. Weber (1990) discusses the use of content analysis for
many types of research. He defines content analysis as a research method
that uses procedures to draw inferences from texts. He further indicates
that the way a researcher draws these inferences, whether from analyzing
text at a word, paragraph, or another level, varies based on the purpose of
the research.
69


When coding the essay data for relationships, the researcher looked
for any statements of causality between these items, such as explicit or
implied "if' conditions; statements that indicated that one item leads to or
causes another item; and clauses that indicated that there existed a time
dependency between items. All of these were counted as relationships,
but only when they applied to items that had previously been identified as
choices or factors. Essays were scored using a color-coded highlighting
scheme with a separate color for factors, choices, and relationships. Figure
3.3 shows a passage from a subject's preliminary essay on exercise science.
Factors are underlined: choices are in bold, and relationships are in italics.
The first step I would take in designing a personalized exercise
program would be to find out why the individual wants to start on
an exercise program in the first place. The person may want to
increase cardiovascular fitness, flexibility, muscle strength or
coordination......They may want to increase their strength
through weight training, or try to increase ones heart efficiency
through cardiovascular or aerobic fitness-----A person who has not
exercised in a long time or is not used to exercise would have to start out
from scratch and do basic exercises zvith little intensity. A person who has
been exercising and wants to take their workout to a higher level would
have a totally different workout from the beginner.
Figure 3.3. Sample portion of an essay showing tallied data attributes.
Preliminary and ending essays were expected to be evaluated using
identical criteria since both assignment descriptions instructed students to
address the factors critical to their domain, the choices in their domain and
the relationships between any combination of these two. Subjects
accurately followed assignment directions for the preliminary essays thus
70


allowing for evaluation using the above-stated criteria. However, for the
ending essays, in spite of assignment instructions which asked students to
address these same topics, students consistently did not include
information in their ending essays regarding factors, choices and
relationships for their expert system's domain. Thus, this ending essay
data could not be evaluated using these criteria and the planned repeated
measures ANOVA could not be performed on the preliminary and ending
essay data.
Rather than not use the ending essay data at all, I opted to perform
a category-based content analysis on the ending essay data (Weber, 1990).
The categories are listed in Table 3.2. Categories were generated from
general topics that were frequently addressed in the ending essays. This
data, while not what was originally intended, did prove useful in
evaluating the statistical results reported in chapter four.
71


Table 3.2.
Categories Tallied In Ending Essay Data
Increased Content Knowledge
Increased Complexity Understanding
No content knowledge increase
Described Their Knowledge Synthesis Process
SemNet to EXSYS Difficult
SemNet to EXSYS Helpful
Frustrated system didn't represent their knowledge
Frustrated hit EXSYS 50 rule limit
Wanted more coaching
Specified how to improve their expert system
Not used to project ambiguity
Increased awareness of using computers in classrooms
Increased confidence with using computers
Preferred SemNet over EXSYS
SemNet. The semantic networking assessment tool is based
predominantly on work done by Fisher, Faletti, and Quinn (1989).
Quantitative descriptive data collected for the networks included some
very basic network descriptions as well as more complex ones. All are
described below and are taken from previous research which used these
attributes to evaluate the overall content representation of networks
(Faletti & Fisher, 1991; Fisher, et al., 1989):
total number of relationships or links used in the network
(relationships)
number of concepts in the network (breadth)
number of concept-relation-concept instances in the network
(extent)
72


number of concepts that participate in three or more instances
(enmeshed concepts)
maximum concept embeddedness of the network (maximum
embeddedness) where embeddedness is defined as the count of
all possible paths to a concept from two nodes away.
a count of the concepts that have 25% or more of paths the
maximum embedded concept (embedded_25)
number of singly connected concepts (fringe concepts)
The SemNet software provides easy and accurate reports for
collecting this data. When viewing a network, a user can select various
reports that provide the data items described above. To collect this data,
the researcher simply opened each subject's network and referred to the
appropriate reports for each data item.
73


Subject John Doe
Domain: Exercise Science
Semantic Networking Scores:
Count
Breadth:
Extent
Enmeshed Concepts >= 3
Number of Relationships
Fringe Concepts
Maximum Embeddedness
Embedded 25
65
51
15
20
75
16
105
Figure 3.4. Sample semantic network assessment criteria worksheet.
EXSYS. Expert systems were assessed with quantitative descriptive
data collected for each expert system. Data items collected for each expert
system are described below.
number of rules in the system
number of rule types in the system where rule types were a
single rule type is defined as a single combination of qualifiers
used in a rule. For instance, if a rule is
IF grade = k-3 AND behavior = doesnt listen AND behavior
count = 1-2
this rule uses three qualifiers combined with one another. If
another rule is
74


IF grade = k-3 AND behavior = doesn't listen AND behavior
count = 3-4
this rule is of the same rule type because it checks the same three
qualifiers. It checks different values for one of those qualifiers
but none the less it is simply a variation on the first rule
pictured. Finally, the next rule exemplifies a rule that would be
a different rule type because it checks a different combination of
qualifiers:
IF grade = k-3 AND age = 10 -15...
number of qualifiers (or factors) considered while making
decisions in the domain
number of choices (or possible decisions) that the expert system
could produce
maximum depth of the rules in the system, where depth is
defined as the count of the number of factors or qualifiers used
in the condition portion of an if statement (i.e. in the rule "if
FACTOR A and FACTOR B and FACTOR C, then recommend
CHOICE D", three factors are considered thus this rule has a
depth of three)
the average depth of the rules in the system
a rating between 1 and 10 of the sensitivity of the system, where
sensitivity was defined by how well the system covered the
domain (both the choices, the qualifiers and the logic
represented in the if rules), and how well its qualifiers could
75


gather the information necessary for producing an expert
system outcome
Subject Jack Counselor
Domain: Special Education Referrals
Expert System Scores:
Count
Rules: 30
Qualifiers: 6
Choices: 5
Maximum Depth 8
Average Depth 4
Sensitivity 7
Figure 3.6. Sample expert system assessment criteria worksheet.
Reliability and Validity
Reliability and validity of the assessment measures is addressed in
the following ways. For both the semantic networking and expert systems
assessment criteria, reliability is ensured via the precise definitions of the
"countable" data attributes being collected for both tools. For instance, in
EXSYS, one can choose a menu item that lists all the rule, qualifiers and
choices that are in the expert system. To collect the data, one simply
counts and records the total number of each item other than the
counting, there is no room for interpretation errors. SemNet ensures even
76


more reliability by providing reports which specifically list the data
attributes I collected, and their descriptive statistics. The semantic
network criteria described are all taken from these reports, and are
completely objective. Thus, the researcher did not have to exercise any
judgment to determine the maximum embeddedness of the network, but
simply look at the proper SemNet report. Thus reliability is ensured for
the data attributes for the expert systems and the semantic networks via
the clear definitions of these attributes the tools' ability to clearly display
these attributes.
Reliability for the preliminary and ending essays is more of a factor
than with the other two measures. Recall that essays were scored for the
number of factors mentioned that the expert system would use in order to
make decisions in the domain, the number of choices the expert system
would recommend, and the relationships mentioned between these items.
To estimate the reliability of this scoring system, a sampling of five of the
total 5 preliminary essays were also scored by someone other than the
researcher. Table 3.3 shows the correlations between the two sets of
scores. The simplicity of the definitions of these data attributes seems to
have led to fairly high reliability correlations for this measure.
Table 3.3.
Preliminary Essay Reliability Correlations fn = 26)
Factors Choices Relationships
Reliability Correlation .91 .76 .96
77


Because of the fair amount of prior research conducted using
SemNet, validity of the assessment data criteria used in this study is also
fairly well established. Fisher, Faletti, and Quinn (1989) specifically
outline the attributes used for assessing SemNets in the current study as a
means of assessing the overall connectedness and structure of a semantic
network. They describe these attributes as indicators of the depth of
processing that may have occurred during network generation. Further,
the validity of these items as indicators of the creator's depth of processing
and overall synthesis of the knowledge domain is also supported via the
SemNet tool itself which provides easy ways of gathering these data items
for any network. Faletti and Fisher (1991) and Fisher (Fisher, 1992) both
describe the various SemNet reports as being designed so users and
researchers can see, for instance, which are the most developed concepts,
if there are duplicate concepts, and what sort of relationships the network
uses in its instances. Thus, this prior research provides a basis for the
validity of the semantic network attributes used to assess the network's
overall representation of the knowledge domain.
The expert system and essay data attributes are less tested. I
performed an extensive literature search for known and practiced ways
for evaluating expert systems in terms of the content knowledge depth
and synthesis they represented, and could find no research or guidelines.
Given that the purpose of the essays was to gather before and after
"snapshots" of how subjects viewed the portions of their expert systems
topics that were most pertinent to their expert system development (i.e.
78


the factors used to make decisions in the domain), it seemed logical to ask
subjects to write about the three main components of their expert systems:
factors (called qualifiers in EXSYS), choices, and the relationships between
these (or rules). These items are a subset of the items used in the overall
expert system evaluation. For the expert systems, I chose data attributes
that are the most basic to expert systems that use if-then rules; thus I
collected data on rules, choices and qualifiers as well as some other
variables that were available using the EXSYS package.
Procedure
At the beginning of the semester, the course professor and I told
participants that their main semester project would be to generate expert
systems on an assigned topic relevant to their majors. This project
replaced a research paper from previous semesters, thus students were
informed that they may have to do extra research in order to complete
their expert systems. We asked participants to provide permission that
the assignments turned in throughout the semester related to their expert
systems project could be used anonymously as data for a study. We also
informed participants of their rights pertaining to the use of their data in
this study via an Informed Consent Form (see Appendix A). Finally, we
explained to the class that they would use software, called an expert
system shell, that allowed them to create expert systems without having to
be computer programmers. For motivation, participants read articles on
using mindtools in the classroom, and specifically uses of expert system
79


generation and semantic networking generation tasks in classes. Given
that most of the participants planned careers in elementary education,
articles focused on uses of shells and networks for younger learners. Both
the professor who taught the course and I believed that providing
participants with such materials would positively motivate them by
providing relevant to their future occupations.
Training. Other studies using expert system shells as knowledge
representation tools indicated that proper and thorough training is
necessary (Knox-Quinn, 1995; Lippert, 1988b). The course's lab assistant
and I trained the participants to use the expert systems shell, EXSYS.
Training took place during two of the courses required 1.5 hour lab
sections and included on-line demonstrations of completed EXSYS-
generated expert systems, and a tutorial where participants were guided
through generating their own expert system. Additionally, I provided
participants with the following written materials to support their use of
the EXSYS expert system shell and the knowledge generation process:
1. an EXSYS instruction set created by this researcher (see
Appendix B) to support using the most necessary EXSYS
functions,
2. and access to the complete EXSYS documentation set.
As the semester progressed, certain course lab times were
designated for help on student expert systems. The lab instructor and I
80


were on hand for those sessions to help students with any difficulties they
were having using the expert system shell. Finally, participants had
access to the researcher for questions via electronic mail and also by
appointment as needed. Data about the types of difficulties experienced
was collected and is reported qualitatively in chapter four.
Similarly, participants who used SemNet in addition to EXSYS
were trained on SemNet. SemNet training occurred only for those in the
SemNet group in a specially designated lab session. Non-SemNet group
members were not present for this training session. Participants were
guided through creating a semantic network on an "easy" topic
deciding on a restaurant to patronize and then provided several
examples of completed networks and a hard-copy job aid for using
SemNet. As with the expert system, this researcher and lab instructor
were available both during lab times and via electronic mail to answer
participant questions.
Software Access. All participants received a copy of the EXSYS
software on a diskette. Participants used this diskette for running EXSYS
on the universitys laboratory Macintosh systems. On the other hand, only
participants in the SemNet group received copies of the SemNet software.
Given that the non-SemNet group had no need for the SemNet software,
there was no need or motivation for the groups to share this software.
The class was divided into expert system only and semantic
networks plus expert system via pseudo-random assignment. The class
division was 15 in the expert system only group and 15 in the semantic
81


networking group. Since the students in the semantic networking group
were required to turn in both the expert system and the semantic network,
these students were excused from a course assignment on computer
graphics. This equalized the work load for both groups.
Expert System Topics. This course's students are predominantly
general education, exercise science, and special education majors. Given
that the majority of the students fell into these three major categories, we
assigned expert system topics to each student based on their major. The
researcher hoped to increase subject intrinsic motivation with a relevant
topic. Exercise science students generated expert systems on designing an
exercise routine for an individual, special education students on
recommending an educational diagnosis for a student exemplifying
learning difficulties, and students from general education generated
expert systems on classroom management. Though participants did not
all generate expert systems on the same topic, the same evaluation rubric
was used for all three expert system domains. I realized the inherent
difficulty in introducing three rather than one expert system domain, but
believed that providing a more authentic and motivating topic for subjects
outweighed the research drawbacks.
Activities. To evaluate the participants' prior knowledge, for each
of the three topic groups, we asked participants to write a short (defined
as one to two double-spaced pages) essay describing their view of their
content domain. Our instructions told the participants to focus their
essays on the key decision-making factors in their domains and how those
82


factors relate to one another in order to draw conclusions and make
decisions in the domain. We required participants to complete this same
essay twice once prior to the knowledge representation project and
again afterwards. For the preliminary essay, we had not instructed the
participants on any knowledge representation mindtools, so these essays
can be assumed to represent a) the participants' own view of the
knowledge domain prior to the knowledge representation project, and b) a
representation of how their ability to represent a knowledge domain in
terms of key concepts or factors and draw relationships among these
factors. For the ending version of the essay, we provided the same
instructions as for the preliminary essay but supplemented them by giving
participants the freedom to use whatever tools they thought would best
represent their domain. Essay evaluation schemes are described in the
assessment materials section above.
Once the expert system topic areas had been defined, participants
spent thirty minutes of class time in domain-specific groups (regardless of
whether in the semantic networking group or not) discussing factors that
influence decisions in their domains. The purpose of this time was to get
all participants thinking about their domains in terms of interrelated
concepts and factors that ultimately produce decisions or problem
solutions. We informed the participants that by no means did we expect
that they know all that was necessary to completely represent their
domains at this point, but rather to focus on the relationships involved in
83


what they did know and how they could proceed to gather and structure
more domain knowledge.
Expert system and semantic network generation took place over the
first nine weeks of the semester, and the final expert system was due
during the tenth week of the course. Participants kept a record of the
amount of time they spent on the expert system and whether that time
was spent collaboratively with other members of their team in an Activity
Log. We informed them that the Activity Log information would not
affect their grades but was needed for the research and for improving
subsequent iterations of this project. A copy of the Activity Log form is in
Appendix C.
On their activity logs, students indicated whether the time spent on
the project was individual or collaborative. In spite of the fact that
learners generated individual expert systems, they would naturally
collaborate with one another especially if they are generating systems
on the same topic. In order to prevent contamination between the SemNet
and non-SemNet groups during this potential collaboration, the course
professor and I informed the subjects that they could only collaborate or
discuss their expert systems with other members of their "team". We
distributed a handout to inform subjects about which team they were on
either on Team A (SemNet group) or Team B (non-SemNet group). In
order to keep the teams from intermixing during expert system
collaboration, the team concept was repeatedly stressed throughout the
semester in other course exercises.
84


Any level of collaboration may be viewed from a research point of
view as a confounding factor to the study's results. Critics may observe
that students didnt generate expert systems totally on their own. Though
this is true, I recognized that to prohibit any sort of natural collaboration
would also be a confounding factor. Thus, the activity log provides
information on the extent of the collaboration.
Summary
This chapter has described the procedures, participants and
instruments used in this study. The chapter focused on the assessment of
the various data sets produced in the study, and the measures taken to
ensure both a reasonable environment for students to complete their
projects (including collaboration) and ways to prevent this collaboration
from invalidating this research. Chapter four addresses the data analysis
results of the study.
85


CHAPTER 4
RESULTS
Introduction
This chapter describes the data analysis results of the study.
Results are presented in the order in which the analyses were performed.
The purpose of this chapter is only to present the statistical results.
Chapter five then reviews the study's hypotheses and discusses the
significance and reasons for these results.
Data Analysis Overview
There were several types of data analyzed for this study and
several types of data analysis performed on these data. This section
reviews the study's hypotheses and provides an overview of the data
analysis procedures performed to test these hypotheses. The following
section presents the actual results from each of these analyses. The
hypotheses for the study were as follows: 1
1. Learners in the semantic networking group will produce better expert
systems than those in the non-semantic networking group, where
better is defined by an expert system set of assessment criteria
86


(completely explained in chapter 3) based upon number of rides,
factors and decisions in the expert system.
2. Learners in the semantic networking group will perform better on
ending essays than those in the non-semantic networking group.
Ending essays are evaluated on their ability to represent completely
the complexity of the expert system topic domain.
3. Within the semantic networking group, learners who generate more
expansive/complete concept maps (# of nodes, linked-ness) will
produce better expert systems where better is defined by the expert
system evaluation criteria.
Hypothesis one addresses the main question from the study. Do
subjects who produce preliminary semantic networks produce expert
systems that represent deeper knowledge processing and greater synthesis
(as represented by the assessment criteria explained in chapter 3), than
subjects who simply produce expert systems? The following data analysis
procedures were performed to test this hypothesis.
Expert system data MANOVA. A MANOVA was performed to
determine if there were significant differences in expert system
performance between those subjects in the SemNet versus the
non-SemNet group.
Preliminary essay data MANOVA. Subjects completed
preliminary essays on their expert system topic areas. These
essays were used to determine the subjects entry level
knowledge of their topic area. In order to show that the two
groups were random and that any differences that may show up
in the expert systems measures or the ending essay measures, a
87


Full Text

PAGE 1

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 2

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 3

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 4

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 5

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 6

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 7

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 8

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 9

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 10

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 11

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 12

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 13

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 14

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 15

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 16

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 17

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 18

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 19

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 20

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 21

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 22

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 23

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 24

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 25

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 26

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 27

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 28

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 29

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 30

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 31

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 32

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 33

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 34

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 35

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 36

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 37

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 38

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 39

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 40

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 41

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 42

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 43

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 44

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 45

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 46

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 47

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 48

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 49

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 50

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 51

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 52

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 53

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 54

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 55

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 56

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 57

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 58

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 59

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 60

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 61

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 62

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 63

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 64

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 65

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 66

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 67

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 68

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 69

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 70

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 71

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 72

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 73

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 74

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 75

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 76

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 77

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 78

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 79

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 80

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 81

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 82

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 83

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 84

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 85

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 86

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 87

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 88

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 89

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 90

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 91

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 92

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 93

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 94

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 95

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 96

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 97

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 98

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 99

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 100

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 101

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 102

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 103

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 104

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 105

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 106

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 107

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 108

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 109

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 110

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 111

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 112

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 113

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 114

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 115

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 116

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 117

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 118

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 119

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 120

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 121

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 122

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 123

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 124

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 125

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 126

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 127

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 128

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 129

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 130

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 131

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 132

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 133

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 134

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 135

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 136

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 137

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 138

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 139

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 140

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 141

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 142

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 143

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 144

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 145

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 146

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 147

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 148

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 149

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 150

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 151

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 152

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 153

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 154

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 155

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 156

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 157

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 158

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 159

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 160

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 161

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 162

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 163

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 164

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 165

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 166

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 167

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 168

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 169

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 170

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 171

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 172

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 173

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 174

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 175

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 176

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 177

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 178

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 179

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 180

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 181

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 182

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 183

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 184

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 185

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 186

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 187

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 188

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 189

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 190

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 191

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 192

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 193

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 194

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 195

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 196

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 197

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 198

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 199

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 200

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 201

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 202

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 203

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 204

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 205

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 206

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 207

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 208

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 209

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 210

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

PAGE 211

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.