Citation
Nonlinear adaptive control using neural networks and hybrid recursive kalman identification

Material Information

Title:
Nonlinear adaptive control using neural networks and hybrid recursive kalman identification
Creator:
Rogers, Dale A
Publication Date:
Language:
English
Physical Description:
vii, 74 leaves : illustrations ; 29 cm

Subjects

Subjects / Keywords:
Neural networks (Computer science) ( lcsh )
Control theory ( lcsh )
Kalman filtering ( lcsh )
Control theory ( fast )
Kalman filtering ( fast )
Neural networks (Computer science) ( fast )
Genre:
bibliography ( marcgt )
theses ( marcgt )
non-fiction ( marcgt )

Notes

Bibliography:
Includes bibliographical references (leaves 72-74).
General Note:
Submitted in partial fulfillment of the requirements for the degree, Master of Science, Electrical Engineering.
General Note:
Department of Electrical Engineering
Statement of Responsibility:
by Dale A. Rogers.

Record Information

Source Institution:
University of Colorado Denver
Holding Location:
Auraria Library
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
31508811 ( OCLC )
ocm31508811
Classification:
LD1190.E54 1994m .R64 ( lcc )

Full Text
Nonlinear Adaptive Control using
Neural Networks and
Hybrid Recursive Kalman Identification
by
Dale A. Rogers
B.S., Colorado State University
A thesis submitted to the
Faculty of the Graduate School of the
University of Colorado at Denver
in partial fulfillment
of the requirements for the degree of
Master of Science
Electrical Engineering
1994


This thesis for the Master of Science
degree by
Dale A. Rogers
has been approved for the
Department of
Electrical Engineering
by
Edward T. Wall
Miloje Radenkovic
1


Rogers, Dale A. (M.S., Electrical Engineering)
Nonlinear Adaptive Control using Neural Networks and Hybrid Kalman Identification
Thesis directed by Professor Edward T. Wall
Abstract
In recent years the use of neural networks in control dynamics have resulted in the
ability to successfully control a wide range of nonlinear time varying systems. The utiliza-
tion of a neural network in controlling a nonlinear time varying system is demonstrated. A
linear hybrid Kalman based estimator/controller is also implemented to provide a basis for
comparison. The neural and linear methods each have trade-offs which the designer must
be aware of. A neural controller is able to function with more degrees of freedom (plant
order, variable nonlinearities, noise statistics, etc.) than a more conventional controller.
A neural network is also inherently fault tolerant some percentage of the neurons can
be disabled while still maintaining adequate control. The most obviouB disadvantage of a
neural controller is the computational costs when compared to a linear controller. Another
issue that may concern the designer is inability to prove global stability with neural net-
works. Although some progress has been made in recent years, this has remained difficult
hurdle to overcome. Nevertheless, neural networks have proven to be effective methods of
controlling many types of systems.
This abstract accurately represents the content of the candidates thesis. I recommend
its publication.
Signed:
Edward T. Wall


Acknowledgements
I would like to thank Professors Edward T. Wall, Miloje (Mike) Radenkovic, and Shelly
Goggin for their helpful guidance and suggestions. I would also like to thank Tuan and
Hai Ho for their advice, creative discussions, and assistance with software design.
111


Contents
1 Introduction 1
1.1 Contribution............................................................. 1
1.2 Short History of Artificial Neural Networks.............................. 1
1.3 Fundamental Concepts of Artificial Neural Networks....................... 2
1.3.1 Biological Neuron ................................................ 2
1.3.2 Artificial Neuron................................................. 4
2 Optimization and Learning 6
2.1 Back-propagation......................................................... 6
2.1.1 Some notes on Back-Propagation....................................13
2.2 Kalman Based Adaptive Control............................................13
2.2.1 Linear Regressions................................................13
2.2.2 Least Squares Criterion...........................................14
2.2.3 Recursive Least Squares...........................................14
2.2.4 Kalman Filter Implementation......................................16
2.2.5 Adaptive Control................................................. 17
3 Application 19
3.1 Problem Statement........................................................19
3.2 Simulations............................................................. 19
3.2.1 Speed.............................................................20
3.2.2 NN structure and optimal number of neurons........................20
3.2.3 Comparison with the linear controller.............................20
3.2.4 Fault tolerance...................................................28
3.2.5 Training approaches...............................................34
3.2.6 Step Responses....................................................34
3.3 Conclusions..............................................................47
A Source Code Listings 48
A.l Main routine for the two layer NN controller .............................48
A.2 Main routine for the Kalman/RLS linear controller.......................53
A.3 Main routine for the three layer NN controller..........................56
IV


A.4 Forward pass with a sigmoid activation function .............................60
A.5 Forward pass with a linear activation function...............................61
A.6 Back-propagation with a sigmoid activation function..........................61
A.7 Back-propagation with a linear activation function...........................62
A.8 Calculation of the nonlinear plant...........................................63
A.9 Majority vote calculations...................................................63
A.10 Plot Ymse(k) ...............................................................64
A. 11 Plot the neuron outputs....................................................65
A.12 Plot the state vector and parameter estimates, x(ft) and 0{k)...............66
A. 13 Plot the control signal u{k) and error signal e{k).........................68
A.14 Plot the neuron input weights...............................................68
A.15 Plot the output y(k) and desired output y*{k)...............................70
v


List of Figures
1.1 Diagram of a biological neuron................................................... 3
1.2 Diagram of an artificial neuron.................................................. 4
1.3 Sigmoid activation function for several values of A.............................. 5
2.1 Neural network controller........................................................ 7
2.2 Neural network control system block diagram...................................... 7
2.3 Back-propagation implementation block diagram for a single neuron................ 8
3.1 NN controller: output signals y(k) and y*(k).....................................21
3.2 NN controller: output error, e[k)................................................22
3.3 NN controller: output mean squared error, Ymse(k)................................22
3.4 NN controller: state vector, x(fc)...............................................23
3.5 NN controller: input layer (layer 1) weights, wrj(A).............................23
3.6 NN controller: neuron outputs for input layer (layer 1), Oi(k)...................24
3.7 NN controller: output layer (layer 2) weights, Wy(fc)............................24
3.8 NN controller: control signal, u(k)..............................................25
3.9 Linear controller: output signals, y(k) and y*(k)................................25
3.10 Linear controller: control signal, u(k)..........................................26
3.11 Linear controller: state vector, x(fc)...........................................26
3.12 Linear controller: output error, e(k)............................................27
3.13 Linear controller: output mean squared error, Ymse(k)...........................27
3.14 Linear controller: parameter estimates, 0i(k)...................................28
3.15 Faulty NN controller: output signals y(k) and y*[k).............................29
3.16 Faulty NN controller: output error, e{k)........................................30
3.17 Faulty NN controller: output mean squared error, Ymse(k)........................30
3.18 Faulty NN controller: state vector, x(fc).......................................31
3.19 Faulty NN controller: input layer (layer 1) weights, vfTi(k)....................31
3.20 Faulty NN controller: neuron outputs for input layer (layer 1), Oi(k). ... 32
3.21 Faulty NN controller: output layer (layer 2) weights, 'Wij(k).................32
3.22 Faulty NN controller: neuron outputs for output layer (layer 2), Oj(fe). ... 33
3.23 Faulty NN controller: control signal, u(k)......................................33
3.24 NN controller step response: output signals y(k) and y*(k)...................34
3.25 NN controller step response: output error, e(fc)................................35
vi


3.26 NN controller step response: output mean squared error, Ymae(k).............35
3.27 NN controller step response: state vector, x(A:)............................36
3.28 NN controller step response: control signal, u(k)...........................36
3.29 Linear controller step response: output signals, y{k) and y*{k)..............37
3.30 Linear controller step response: control signal, u{k)........................38
3.31 Linear controller step response: state vector, x(fc).........................38
3.32 Linear controller step response: output error, e(fc).........................39
3.33 Linear controller step response: output mean squared error, Ymae(k)..........39
3.34 Linear controller step response: parameter estimates, 0i{k)..................40
3.35 Pretrained NN controller step response: output signals y(k) and y*(k). ... 41
3.36 Pretrained NN controller step response: output error, e(k).............41
3.37 Pretrained NN controller step response: output mean squared error, Ymae(k). 42
3.38 Pretrained NN controller step response: state vector, x(fc)............42
3.39 Pretrained NN controller step response: control signal, u{k)...........43
3.40 Pretrained linear controller step response: output signals, y(k) and y*(k). 44
3.41 Pretrained linear controller step response: control signal, u(k).............44
3.42 Pretrained linear controller step response: state vector, x(fc)..............45
3.43 Pretrained linear controller step response: output error, e(k)...............45
3.44 Pretrained linear controller step response: output mean squared error,
Ymae{k).....................................................................46
3.45 Pretrained linear controller step response: parameter estimates, 0i(k). ... 46
Vll


Chapter 1
Introduction
1.1 Contribution
The goal of this thesis is to demonstrate the ability of a neural network to control a time
varying plant of unknown order, nonlinearities, and noise statistics. A linear adaptive
controller is used as a basis by which to judge the capabilities and compare the charac-
teristics of a neural controller. The Kalman filter/predictor was chosen because it is well
understood and works well with time varying systems. Another reason for choosing a
linear controller is that linear controllers are more widely understood than their nonlinear
counterparts. The inherent ability of neural networks to be fault tolerant will be exam-
ined and some issues concerning the choice of either a neural or more conventional control
system will be discussed.
A plant may be thought of as a physical, electrical, or chemical system which is made
stable or controlled in some manner by a control system. A common example of a plant
is an automobile and a control system being the cruise control for the automobile.
Nonlinear plants can present a difficult design problem for the engineer. Nonlinearities
can be difficult to linearize (by using describing functions) or model mathematically. Of-
ten the nonlinearities may be of a nature that makes modelling them impractical. Other
considerations such as a variable order (and dynamics) of a plant and varying noise statis-
tics can all make designing a control system difficult. Neural network controllers have
demonstrated the ability to function well with these types of characteristics.
1.2 Short History of Artificial Neural Networks
Artificial neural networks have been in existence for several decades but the theoretical
development (primarily of the training algorithms) has only been accomplished in recent
years [28]. The foundational work of decades past is fundamental to the construction of
artificial neural networks today.
In 1943, McCulloch and Pitts [16] outlined the first formal model of the artificial
neuron. Not much development was accomplished due to the lack of advanced computers
1


and solid state electronic circuits.
Donald Hebb in 1949 [3] first proposed a learning algorithm for updating the neuron
weights. Today we call this the Hebbian learning rule.
In the 1950s the first artificial neural networks were built. These systems had weights
that were updated automatically. During this period the neuron-like element called a
perceptron was invented by Frank Rosenblatt [23]. He developed a trainable machine
capable of learning to classify certain patterns by modifying connections to the threshold
elements.
The early 1960s brought more significant developments. A device called ADALINE
(for ADAptive LINEar combiner) was designed by Widrow [26] and a powerful learning rule
called the Widrow-Hoff learning rule was developed [27]. An extension of the ADALINE
was called MAD ALINE (for Many ADALINEs). These systems were quite successful in a
variety of pattern recognition techniques. In 1962 Widrow produced a film demonstrating
the capabilities of this technology. Even with the primitive digital computers of the
day, the results were impressive. His system was able to perform some limited speech
recognition and foreign language translation in near real time1.
Although the accomplishments of the early sixties were significant, active interest
diminished. This was due largely to a book written by Marvin Minsky and Seymour
Papert [17] that shed some doubts with rather limited evidence on the artificial neural
networks potential.
From the 1970s to the present, interest in artificial neural networks increased dramat-
ically. During this period a lot of pioneering work has been accomplished by people such
as Kohonen [9, 10, 11, 12], Hopfield [6], Werbos [25], McClelland and Rumelhart [15], and
others, with new and valuable results.
1.3 Fundamental Concepts of Artificial Neural Networks
Artificial neural networks have evolved from early mathematical simulation of biological
experimental data. Attempts at models followed from the differential equations which were
developed by mathematical biologists. Unfortunately, the actual correspondence between
the real and artificial neurons is weak. Knowledge of actual brain functions is so limited
that, at present, only very simple emulation is possible. Nevertheless, much can and has
been accomplished with artificial neural systems.
1.3.1 Biological Neuron
The elementary nerve cell is called a neuron and is the fundamental building block of the
biological neural network. A diagram of a biological neuron is shown in Figure 1.1. A
1This film was viewed by this author as part of a short course offered by the National Technological
University (NTU) and broadcast live via satellite from Stanford University (circa 1992). The course title
was: Learning Systems: Adaptive Signal Processing and Adaptice Neural Networks. The instructor was
Bernard Widrow from the Dept, of Elect. Eng. at Stanford University.
2


Figure 1.1: Diagram of a biological neuron.
typical neuron has three major regions: the cell body, which is also called the soma, the
axon, and the dendrites. Dendrites form a dendritic tree, which is a very fine bush of thin
fibers around the neurons body. These receive information from neurons through axons
long fibers that serve as transmission lines. An axon is a long cylindrical connection
that carries impulses from the neuron. The end part of an axon splits into a fine arboriza-
tion. Each branch of it terminates in a small end bulb almost touching the dendrites of
neighboring neurons. The axon-dendrite contact organ is called a synapse. The synapse is
where the neuron introduces its signal to the neighboring neuron. The signals reaching a
synapse and received by dendrites are electrical impulses. The interneuronal transmission
is sometimes electrical but is usually effected by the release of chemical transmitters at the
synapse. The terminal boutons generate the chemical that affects the receiving neuron.
The receiving neuron either generates an impulse to its axon, or produces no response.
A neuron is able to respond to inputs only under certain conditions and also exhibits
certain characteristics:
1. A response is generated if the total potential of its membrane reaches a certain level.
The membrane can be considered as a shell, which aggregates the magnitude of the
incoming signals over a short time duration, the period of latent summation.
2. Incoming impulses can be excitatory or inhibitory depending on whether they en-
courage or inhibit a response.
3. Excitation exceeds inhibition by a threshold value (~ 40mV).
4. Incoming impulses that are closely spaced in time and arrive synchronously are more
likely to cause the neuron to fire. 5
5. After firing, a nerve is unable to conduct any signals at all for a short period of time
called the refractory period.
3


6. The output of the neuron does not differ much in amplitude so the signals transmitted
can be considered binary.
Although the above description is extremely simplified from a biological point of view, a
lot of insight can be gained to construct an artificial neuron.
1.3.2 Artificial Neuron
Given the previously mentioned biological neuron characteristics, it is possible to construct
a simplified mathematical model of a neuron. A general neuron symbol is shown in Figure
1.2. Each neuron consists of a number of synaptic input connections with a weighting
Synaptic
connections
Figure 1.2: Diagram of an artificial neuron.
factor applied to each, a processing element, and a single output. The output is given by
the following relationship
o = /{net} (1.1)
where net = w x (1.2)
and fl
wTx = Y^WiXi 1=1 (1.3)
The function /{net} (or /{wTx}) is often referred to as an activation function. Typical
activation functions used are the sigmoid function
finet] = 1 + e2_Awet 1 (1-4)
the hard limiting function
f{net} k sgn (net) = j ^ < 0 (1'5)
4


and the linearly scaled function
f{net} = K net (1.6)
where A > 0 in (1.4) determines the steepness of the activation function f{net} near
net = 0 and (1.6) defines a linear ramp activation function where K is the slope. These
are bipolar activation functions since they can have values greater than and less than zero.
An alternative to (1.4) is to use the hyperbolic tangent function such that
f{net} = tanh(A7ie<) (1*7)
Figure 1.3 shows the sigmoidal activation function (1.4) for several values of A. The
slopes of the curves in Figure 1.3 are proportional to the values of A. There are numerous
other activation functions; however, the only ones of interest here are given by equations
(1.6) and (1.7).
An artificial neuron, when combined with an activation function, is known as a per-
ceptron.
Figure 1.3: Sigmoid activation function for several values of A.
5


Chapter 2
Optimization and Learning
Optimization and learning rules, or algorithms, are ubiquitous in the field of neural net-
works. In recent years numerous algorithms have been developed and implemented. Each
has its own strengths and weaknesses. Some are more suited to a given application than
others. Most learning rules are based on minimizing a cost function through a variety of
gradient techniques. These cost functions may be based on error or energy.
The goal of the learning, or optimization, process is to adjust the weights of the neural
network such that the desired response is obtained. The learning process may be of the
supervised or unsupervised form.
Supervised learning, also called learning with a teacher, is learning based on the direct
comparison of the output of the network with known correct answers. Unsupervised
learning uses no external correct answers to adjust their weights. The learning goal is
not defined in terms of specific correct examples. Instead, information is used about
correlations of the input or signals [2].
A control system is a supervised system since a desired response is always provided.
An example of an unsupervised learning system is one that is used to classify patterns.
For Kalman based system identification, the goal of learning is the same minimize
a cost function. The linear adaptive theory will be discussed later in this chapter. Since
the focus here is on the neural networks, more attention will be paid to it. For further
research on the linear theory, see Sections 7.3 and 11.2 in [13].
2.1 Back-propagation
Back-propagation is a method of supervised learning. The back-propagation algorithm
was developed by Paul Werbos [25] but the achievement remained largely unknown until
1982 when it was rediscovered by Parker [20, 19] and then again in 1986 by Rnmelhart,
Hinton, and Williams [24].
The goal of an adaptive control system is to minimize the error between the actual
output of the system (controller and plant) and the desired output of the system. The
6


(2.1)
cost function to be minimized is given as
m = {[y'(fe +1) y(* + I)f [y*(* +1) y(* +1)]}
where !?{} denotes the mathematical expectation. The weights are updated by propa-
gating the error back through the plant and each layer of the neural network. The neural
identifier/controller is depicted in Figure 2.1 and a block diagram is shown in Figure 2.2.
An approach similar to Figure 2.2 using a model reference neural adaptive controller can
be found in [7]. Figure 2.3 depicts the implementation of the back-propagation algorithm
for a single neuron located in the output layer of a neural network. The activation function
shown is hyperbolic tangent function given by (1.7).
1 st Layer 2nd Layer Output Layer
Figure 2.1: Neural network controller.
ydtt
Figure 2.2: Neural network control system block diagram.
The following is a derivation [5, 4] of the back-propagation equations for a three layer
neural identifier/controller. A similar, though not as comprehensive, derivation can be
7


*1
Figure 2.3: Back-propagation implementation block diagram for a single neuron.
found in [2]. The derivation for a two layer network is very similar to the three layer
network.
Theorem 2.1 The weight values connecting the hidden layer to the output layer at time
t = k are wij(k). These can be updated as follows:
Wij(k + 1) = Wij(k) + A Wij(k) (2.2)
Proof Substituting (2.1) into (2.2) gives
Awij(k) = -r)
= ~V
= n
(*) G y(*++ x)
^- G [y'(* +1) y(t + l)f [y*(* +1) y(* +1)])
far*C* + 1) y(* + 1)1 (2 3)
dwij{k) du(k)
du{k) dy(k + 1)
dwij(k) du(k)
where r) is the step size for the gradient search. Given the plant model
x(fc + 1) = A(fc)x(fc) + B(/s)u(A:) + u>(fc)
y (k) = C(k)jc(k) + u(k)
(2.4)
(2.5)
8


and assuming C varies slowly with respect to the sampling rate such that C(fe + 1) r\j C(k)
[5] then
y(k + 1) = C (fc)x(fc) + u{k + 1)
S C(fc)A(Jfe)x(Jb) + C(fc)B(Jfe)u(ife) + C (k)u(k) + u(k + 1)
and finally
fly (k + 1)
flu (k)
= C(k)B(k)
The derivative du(k)/dw{j(k) can be computed [21] as
flu(fc) flnetj(fc) :flu(fc)
dwij(k) dwij{k) flnetj(fc)
where
netj(A:) = y net? net? net"3
n2
netjik) = ^
nT
t=l
since u(fc) = ij{netj{k)} and with (2.9) and (2.10), (2.8) becomes
flu(fc)
dwij(k)
= Oi(k)ej
0 0 0 Oi(k) 0
flfj {netj(k)}
dij {netj(k)}
flnetj
flnetj
Substituting (2.7) and (2.12) into (2.3) yields
Awij(k) = TjOiikfe9*3 ^B(fc)C(fc) [y*{k + 1) y(fe + 1)]
= Vej6j{k)Oi{k)
(2.6)
(2.7)
(2.8)
(2.9)
(2.10)
(2.11)
(2.12)
(2.13)
(2.14)
where the local error 6j{k) is given as
m = 8fi^y)}B(*)c(<0 |y*(* +1) y (k +1)1 (2.15)
Since the actual output y{k + 1) is not available to the control system it is replaced by
the predicte output y(k + 1). d
Theorem 2.2 The weight values connecting the input layer to the hidden layer at time
t = k are wTi(k). These can be updated as follows:
wTi{k + 1) = Wri(k) + A wri(k) (2.16)
9


Proof Substituting (2.1) into (2.16) yields
a /, \ dJ{k)
A wri(k) -7/------
dwTi{k)
_ do ,-(fc) d
= -v------------
= V-,
Q [yW fWf [y*(*) y(*)l)
[y*(*) m]
dwri(k) dO
dOj(k) dy(k)
dwriik) dOj(k)
The derivative dOj{k)jdwTi{k) can be computed as
dOj{k) _ dij{netj(k)}
dwTi(jk) dwTi(k)
9netj(fc) dij{netj(k)}
dwri(k) dnetj(k)
Substituting (2.18) into (2.17) yields
=va-£§- m w*) m
Using (2.15), (2.19) becomes
d
A wri(k) = 7/
= n
dwri(k)
d
i(k) dnetj(k) dOj(k)
[net ,(&)] Sj(k)
ZZiV>nWOi(k) YZi v>*{k)Oi{k)
dwri{k)
dneti(k)

Since
neti(k) = Y^wri(k)r(k)
r
Oi(k) = fi {neti(k)}
using (2.21) and (2.22), equation (2.20) becomes
AwTi(k) Tj
wn
w

w.
tns
Or
dfj {netj(k)}
dneti
S^k)
(2.17)
(2.18)
(2.19)
dwri(k)
E£i to* (*)<*(*) ]**(*)
" H'(fc)
7/ -u>ii(&) 7i>j(fc) w*n,(*) ]
ao^(fe)
(2.20)
(2.21)
(2.22)
(2.23)
10


Defining
we know have
r /i \ A 9/ Wi(fc)} f /fN
m =w* s*k)
A wTi(k) = 7]6i(k)0T(k)
(2.24)
(2.25)
Theorem 2.3 The weight values connecting the delay line to the input layer at time t = k
are war. These can be updated as follows:
wsr(k + 1) = wsr(k) + Awar(k) (2.26)
Proof Solving for Awar(k) gives
dJ{k)
Aw[k) -T]
dwTi(k)
= -va.dn^ Q y(fe)]r ty*(fc) y(fc)])
= ~v
= V
dw^k)
dls{k) d
dw3T(k) dls(k)
dl a(k) dy (fc)
dw^k) dls(k)
The computation of dl3(k)/dw3T(k) is as follows:
Q[y*W-yWf[y'W-yW])
[y*(*) y(fc)l
9Is(fc) dij{netj(k)}
dw(k) dwBT(k)
_ dnetj(k) dfj {netj(k)}
dwaT(k) d net j(k)
(2.27)
(2.28)
Substituting (2.28) into (2.27) gives
Awar(k) rj
dnetj(k)
dwar(k)
dfj {netj(k)}
dnet j(k)
dy{k)
dh(k)
far(ft) y(*)]
Using (2.15), (2.29) now becomes
dnetj(k) dnetj(k)
dwaT(k) 9neti(A) J
_ 9netj(fc) dfi{neti{k)} dnetj(k) .
^ dwar(k) dnetj(k) dii{neti(k)} J
(2.29)
(2.30)
11


where
dnetj(fc) dnetj(fc)
dfi {net{(k)} dOi(k)
9 F Hill wnOi E£i wiji
dO i
EZlWintOi
With
O i(k) =
Oi(k)
02(k)
0n2{k)
wijOl
w2j02
equation (2.31) now becomes
wnOi
9netj w2i02
dii {neti} j j
. wn2lO n 2 wn2jO n2
Also, dneti(k)/dw(k) in (2.30) can be computed as
dnetj(fc)
dwaT{k)
Wln3Ol
v>2 na02
- wn2n3 On2
= wO


d Erl i wTl(k)0T{k) E ?=i wTj(k)Or(k)
dw{k)
Hr=l^n2{k)0T{k)
wr i(k) dOr(k) dw3r(k) .. w (k) *M*) 1 wrn2 \K)dw(k) \
^rl(A) wri(k) rn2(k) dOT{ k) dwaT(k)
WTi(k) wTi{k) rn2(*) dnetT{k) dOT(k) dwgrik) dnetT{k)
Since
netr(k) = ^waT(k)Ia(k)
a
0T(k) = fT {netr(k)}
equation (2.34) becomes
^^ = [,rl(t) w(k)
Equation (2.30) can now be written as
Awar(k) = 7jSr(k)Ia(k)
dfr{netT(k)}
* dnetT{k)
(2.31)
(2.32)
(2.33)
(2.34)
(2.35)
(2.36)
(2.37)
(2.38)
where
6r{k) = wT
dfT {netr }
dnetr
dfi {neti} dnetj
dneti dfi {neti}
(2.39)
12


2.1.1 Some notes on Back-Propagation
If the sigmoid activation function is chosen to be the hyperbolic tangent function given
by (1.7), then its derivative is given by
dii{neti)
dneti
tanh2 (Ajneti)] = \ (l y2)
(2.40)
If C(fc) and B(k) in (2.7) are not known, then we can approximate the plants partial
derivative [22] as
dP ^ P{u + Su) P(u)
du ~ 6u
This approximate derivative can be determined either by changing each input to the plant
slightly at the operating point and measuring the change at the output or by comparing
changes with previous iterations. The former was chosen due to divide by zero problems.
2.2 Kalman Based Adaptive Control
Since the plant (given in the next chapter) is time varying, a suitable parameter estimator
must be designed which will track the parameters2. The recursive least squares (RLS) is
not suitable as it is designed for linear time-invariant systems. A Kalman based approach
is used since it is designed for time varying systems. Although this method is intended
for linear systems, it has been applied successfully to some nonlinear systems. As will be
shown, the Kalman and RLS equations are very similar to each other.
2.2.1 Linear Regressions
The linear regression model structure is useful and basic in describing linear and non-
linear systems. The plant will be modelled as an ARX model where AR refers to the
autoregressive part and the X refers to the extra input B(q)u{t) (called the exogeneous
variable in econometrics). With the ARX model, we have
Ay(t) = q lBu(t) (2.42)
where g-1 is the delay operator and
A(g) = 1 + dig-1 + + dnaq~na (2.43)
B(q) = M-1 + b2q~2 + + bnbq~nb (2.44)
Letting
9 =
Ol ' On0
(2.45)
2The system identification portion of this section is reproduced in part from Chapters 7 and 11 in [13].
13


and
4>T(t) = -y(t) y{t 1) -y(t na) u(t) u(t 1) u(t nb) (2.46)
Applying (2.45) and (2.46) to (2.42) gives the predicted value y(< + 1)
y(t + 1) = f{t)0 (2.47)
2.2.2 Least Squares Criterion
With (2.47) the prediction error becomes
e(f, 0) y{t) 4>T(t)6 (2.48)
and the criterion function is given as
Rr 2
t=1 ^
(2.49)
where Vn(0, Zn) is the least squares criterion [13, page 176] for the linear regression
(2.47) and ZN is the set of all measurements such that ZN = {ia(0), 2/(0),..., u(N), y{N)}.
The unique feature of this criterion, developed from the linear parameterization and the
quadratic criterion, is that it is a quadratic function in 6. Therefore, it can be minimized
analytically, which gives, provided the indicated inverse exists,
= arg min Vjv (fl, ZN)
77
JV t=l J 1 t=l
(2.50)
the least squares estimate (LSE). We can let
R(N) =
which is symmetric positive definite, and
t=l
N
f(N) =
t=l
(2.51)
(2.52)
2.2.3 Recursive Least Squares
The least squares criterion given in (2.50) is similar to the weighted least squares criterion
[13, page 305]:
0t = arg min ^ P[t, k) [;y{k) T(k)6] (2.53)
k=i
14


where as in (2.51) and (2.52)
(2.54)
(2.55)
Ot = R
R(t) = Y,P{t,k){k)T{k)
Jfc=1
/(O =
k=1
If the weighting sequence has the following property
P[t, k) -- A 1, A:), 1 < k < t 1
0(*,O = 1
then the weighting sequence may be written as
This implies
k+1
12(0 = \{t)R(t 1) + (f>(t)(j>T{k)
f(t) = \(t)f(t-l) + (t)y(t)
Now
et = r
IT-1/
= R (t) [A(t)/(t 1) + {t)y{t)]
- 12_1(0 [A(t)I2(t l)0t_i + 4>{t)y(t)\
= 12_1(0 {[i(0 />(t)/>r(fc)] 0t-x + 0( = §t-1 + 12_1(0 We thus have
Ot = Ot-i + R y(t) T(t)0t-1
!(/) = A(0l(t 1) + (t)T(k)
(2.56)
(2.57)
(2.58)
(2.59)
(2.60)
(2.61)
(2.62)
which is the recursive algorithm. To avoid calculating the inverse of 12(0 fr every itera-
tion, the matrix inversion lemma
[A + BCD]-1 = A-1
A~XB
DA-'B + C
-l
-l
DA
-l
(2.63)
15


is applied to (2.62). Performing the substitutions A = X(t)R(t 1), B
C = 1 gives
P(t) =
1
m
{ } X(t) + Wt)P(t 1
= Dt - and
(2.64)
Moreover, we have

1
m
p(t-i)^t)-
l P(t -1){t)4?{t)P{t i)
\{t) A(t) + {t)
P(t 1M<)
A(t) + ^(t)P(t-l)^(<)
To be consistent with previous notation 0* is changed to 9{t). The recursive least squares
algorithm can now be summarized as
m = P{t-i)4>{t) (2.65)
\(t) + T(t)P(t-l)ct>(t)
p[t) = ^ [p(i -1) m*T(t)p(t ~ i)] (2.66)
m = 0(t 1) + L(t) y(t) T(t)0(t l)j (2.67)
2.2.4 Kalman Filter Implementation
The Kalman filter predictor in state space form is given as
x(t + l,fl) = A(0)x(t, 6) + B(8)u(t) + K(t, Q) [p(t) C(0)x(t, 0)] . .
y(t\6) = C(0)x(t,e)
K(t,0)={A(0)P(t,0)CT(0) + R12(9)\ [c(9)P(t,8)CT(0) + R2(e)]~1
P(t + 1,0) = A(0)P{t, 0)At(6) + Pi(0) K(t, 0) [c(8)P(t, 0)CT{0) + P2(0)] KT(t, 0)
(2.69)
where
£(0,0) = so(0)
P(O,0) = J5{[a:(O)-io(0)][x(O)-xo(0)]r}
and the covariance matricies are
l£i(0) = E |o;(t)a;T(t)|
_ff2(0) = Pjv(t)uT(t)}
Pi2(0) = #{u;(t)uT(t)}
16


With some substitutions (2.65), (2.66), and (2.67) become the Kalman filter implementa-
tion
m = p(t i)m (2.70)
S-2{t) + T(t)P(t -1 wt)
m = P{t 1) L{t)f{t)P{t 1) + Ri{t) (2.71)
0(t) = 0(t 1) + L(t) y(t) 4>T(t)0(t l)j (2.72)
After using the RLS and Kalman algorithms, it was found that neither would perform
an acceptable job of identifying the plant. A heuristic solution which proved to work well
is a hybrid of the two and is given below. Here, a change of variables (ft = t) has been
made to be consistent with previous sections in this chapter.
m
m
m
P(k 1 )<£(*:)
X{k) + {k)
[P(k 1) L(k)T(k)P(k 1)] + *!(*)
0(k 1) + L(k) [y(A:) y*(k 1)]
(2.73)
(2.74)
(2.75)
The only difference between (2.65), (2.66), (2.67) and (2.73), (2.74), (2.75) is the addition
of the term Ri{k) in (2.74).
The value X(k) is also known as a forgetting factor. One method of calculating this is
A(&) = A0A(k 1) + Ae (2.76)
where
A0 = 0.99
Ac = 1 Ao
A(0) = 0.95
The purpose of the forgetting factor is to assign less weight to older measurements that
no longer represent the system.
For the plant described by (3.3) and (3.4), it was determined that setting A(k) = X
0.95 was sufficient.
2.2.5 Adaptive Control
The system identification algorithm in the previous section can be combined with the
control law to form an adaptive control system. The control law is given by
0^(k)(k) = y*(k + l) (2.77)
17


Equation (2.77) in expanded form gives
-ai{k)y(k) a2(k)y(k 1)------ana(k)y(k na + 1)
+ b0(k)u(k) + + bnb(k)u(t nb) = y*(k + 1)
Solving this for the control signal u(k) yields
u(k) = YTT:i^(k)y(k) + &2(k)y(k-l) + + ana(k)y(k-na + l)
b0(k)
- bi(k)u(k ~ 1) -----bnb(k)u(k n&) + y*(k + 1)]
(2.78)
(2.79)
18


Chapter 3
Application
3.1 Problem Statement
The goal is to design a neural network that will perform the task of system identification
and control. In addition to this, a demonstration is performed of the ability of a neural
network to be fault tolerant while maintaining adequate control of the plant. A comparison
of the results of the non faulty neural network controller is made with those of a linear
adaptive controller.
A time varying nonlinear plant is described by the following general stochastic state
space model
x(fc) = A(fc,x)x(fc) + B(fc)u(fc) + tv(k) (3-1)
y(k) = C (k)x(k) + v(k) (3.2)
where x(fc), u(k), and y(k) are the system states, inputs, and outputs respectively. A(k, x)
is the unknown, nonlinear, time varying system matrix, B(fc) and C(k) are known sys-
tem matrices, and u(k) and v(k) are process and measurement noises, respectively, with
unknown statistics.
3.2 Simulations
The nonlinear plant [5] chosen is given by Equations (3.3) and (3.4). Although this plant is
not based on a physical system it is acceptable since the focus here is on the characteristics
of neural network controllers rather than the characteristics of the plant.
xi(k + 1) 0.4i2(fc) 0.2 sin(0.05fc) xi (fc) + 1 u(fc) + ai(fe)
x2(k + 1) 0.1 cos(0.1A:) -0.2 x2{k) 0.7 u>2(k)
(3.3)
19


and
y{k) =
1 0 zi(fc)
1 u x2(k)
+ v(k)
(3.4)
The variables v[k), and u(k) represent measurement and process noises respectively and
have the following statistics
cj(k) ~ AT(0,0.005) (3.5)
v(k) ~ AT(0,0.005) (3.6)
The desired response of the system was chosen to be a sine wave with an increasing
frequency (or chirp). Making the frequency a variable rather than a constant adds
another degree of freedom (and realism) for the controller to deal with.
The neural controller utilizes sigmoid activation functions for all neurons except those
on the output layer which use the linear activation function. Numerous simulations were
done to determine the best choice of parameters and network structure. The parameters
which controlled the slope and magnitude of the activation functions were set to a constant
for each layer. For this plant the approach proved to be adequate. For some other plant
these parameters may need to change with time. To accomplish this the parameters
can be added to the weight matrix of their respective layer. The training algorithm will
automatically adjust these parameters.
All simulations used the identical noise statistics and were run for 750 iterations. All
the neural network controllers utilized a fifth order delay line on the input. After all
simulations were completed the following observations were made:
3.2.1 Speed
The two layer network simulation required 0.643 million floating point operations (mflops).
The linear adaptive controller required 0.341 mflops, or about half that of the neural net-
work. Consequently, the linear simulation ran about twice as fast as the neural simulation.
3.2.2 NN structure and optimal number of neurons
Determining the optimal structure and number of neurons for each layer is largely an exper-
imental process requiring numerous simulations. In addition to finding the best structure,
optimal values for K and A must be found (if they are not determined adaptively).
It was found in this process that the two layer neural network out performed the three
layer one (J2 < J3). The optimal number of neurons for the two layer neural network was
found to be 15 in the first hidden layer (input layer) and one in the output layer (only one
output!). Also, it was found that 3 neurons in the first layer was a minimum.
3.2.3 Comparison with the linear controller
It is clear from the results that the linear controller performed better than the neural
controller (see Figures 3.1 through 3.8 for the neural controller plots and Figures 3.9
20


through 3.14 for the linear controller plots). Notice, however, that the linear controller
had some difficulty tracking the parameter variations.
The better performance of the linear controller is attributed to the fact that more
knowledge of the plant is required for the controller to function (ie: plant order and noise
statistics).
The weights for the neural controller were initialized with random values.
Figure 3.1: NN controller: output signals y(k) and y*(k).
21


Figure 3.2: NN controller: output error, e(k).
Figure 3.3: NN controller: output mean squared error, Ymse(k).
22


Figure 3.4: NN controller: state vector, x(fc).
Figure 3.5: NN controller: input layer (layer 1) weights, wri(k).
23


Figure 3.6: NN controller: neuron outputs for input layer (layer 1), Oi(k).
Figure 3.7: NN controller: output layer (layer 2) weights, wij(k).
24


4
Iteration (k)
Figure 3.8: NN controller: control signal, u(k).
Figure 3.9: Linear controller: output signals, y(k) and y*(k).
25


Figure 3.10: Linear controller: control signal, u(k).
Figure 3.11: Linear controller: state vector, x(fc).
26


Figure 3.12: Linear controller: output error, e(k).
Figure 3.13: Linear controller: output mean squared error, Yme(k).
27


2.5
Figure 3.14: Linear controller: parameter estimates, 0*(fc).
3.2.4 Fault tolerance
The fault tolerance code successively disables neurons (sets their outputs to zero) through
the course of the simulation (see Figures 3.15 through 3.23). Both the first and second
layers start with 15 active neurons each.
The fault tolerant neural network with the majority vote code operational required
1.34 mflops. This is almost four times the number of calculations required by the linear
simulation and about twice as many as without the fault tolerance code active. The
majority vote routine is primarily to blame for this. Since this is only a software simulation,
results from an actual implementation in hardware would no doubt be different.
Although the controller lost 4/5 of its neurons, it continued to function without a
significant increase in the output mean squared error (Figure 3.17). Even though the
controller experienced the loss of several neurons at each step (k = 200, 300,400, and 500),
recovery from the transient caused by these losses was very quick. As time progressed,
more and more neurons are disabled until there are three neurons left in each of the two
layers.
Adding redundant neurons to the output layer had no effect on performance other
than adding more calculations and simulation time. All the neurons on the output layer
had identical weight values.
The linear adaptive controller was not examined for any fault tolerance since it does
not posses any inherent fault tolerant capabilities. A possible method of building a fault
tolerant linear system would be to have redundant controllers feeding their control signals
28


into a majority vote routine like tlie one used for the neural controller.
Figure 3.15: Faulty NN controller: output signals y(k) and y*(k).
29


S 0
-0.5-
Figure 3.16: Faulty NN controller: output error, e(k).
Figure 3.17: Faulty NN controller: output mean squared error, Ymae{k).
30


Figure 3.18: Faulty NN controller: state vector, x(fc).
750
Iteration (k) Weights
Figure 3.19: Faulty NN controller: input layer (layer 1) weights, wTi(k).
31


Figure 3.20: Faulty NN controller: neuron outputs for input layer (layer 1), 0{(k).
Iteration (k) Weights
Figure 3.21: Faulty NN controller: output layer (layer 2) weights, w,j(Ai).
32


750
Neuron
Iteration (k)
Figure 3.22: Faulty NN controller: neuron outputs for output layer (layer 2), Oj(k).
Figure 3.23: Faulty NN controller: control signal, u(k).
33


3.2.5 Training approaches
All the simulations used a constant scale factor K, slope A, and learning rate 77. If it
were necessary to adjust these parameters, there are a number of methods and algorithms
available to adaptively adjust them. Attempts to adjust 77 locally (unique value for each
neuron) or globally (single value for the entire neural network) proved unsuccessful.
Efforts to get the adaptive approaches to work were not exhaustive. It is clear from the
results that adjusting these parameters adaptively may be unnecessary for this particular
plant. In any case, adding these capabilities would increase the number of calculations
neccessary to perform the simulations.
3.2.6 Step Responses
Step responses were evaluated for each controller. In addition, another set of step responses
were done for each controller with no process or measurement noise, and with pretrained
weights and parameters. The pretrained parameters for each simulation were calculated
by saving the parameters half way through the simulation. The simulation was then rerun
using the saved parameters as the initial values.
Figures 3.24 through 3.28 show the step response of the two layer neural network
controller. The slow rise time is due to the random initial values of the weights.
g
*0
s
g
>?
Iteration (k)
Figure 3.24: NN controller step response: output signals y(k) and y*(k).
34


Iteration (k)
Figure 3.25: NN controller step response: output error, e(k).
Figure 3.26: NN controller step response: output mean squared error, Ymse(k).
35


1.4
Figure 3.27: NN controller step response: state vector, x(A).
Figure 3.28: NN controller step response: control signal, u(k).
36


Figures 3.29 through 3.34 show the step response of the linear controller. The large
overshoot at the beginning of the simulation is the result of using non-pretrained values
for the system identification parameters.
Figure 3.29: Linear controller step response: output signals, y(k) and y*(k).
37


12
Iteration (k)
Figure 3.30: Linear controller step response: control signal, u(k).
12
10
* 4
1 1 r 1 1 xl(k) .... -- *2(k)





waffle

J i
100 200 300 400 500 600 700 800
Iteration (k)
Figure 3.31: Linear controller step response: state vector, x(fc).
38


10
1



DWMty

1 1
100 200 300 400 500 600 700 800
Iteration (k)
Figure 3.32: Linear controller step response: output error, e(&).
Figure 3.33: Linear controller step response: output mean squared error, Ymse{k).
39


Figure 3.34: Linear controller step response: parameter estimates, 0i(k).
Figures 3.35 through 3.39 show the step response of the two layer neural network
controller with no noise. The controller used pretrained weight values for initial conditions.
40


Figure 3.35: Pretrained NN controller step response: output signals y(k) and y*(k).
Figure 3.36: Pretrained NN controller step response: output error, e(k).
41


Figure 3.37: Pretrained NN controller step response: output mean squared error, Ymse(k).
Figure 3.38: Pretrained NN controller step response: state vector, x(&).
42


Figure 3.39: Pretrained NN controller step response: control signal, u(k).
Figures 3.40 through 3.45 show the step response of the linear adaptive controller using
the pretrained parameter values.
43


Figure 3.40: Pretrained linear controller step response: output signals, y(k) and y*(k).
Figure 3.41: Pretrained linear controller step response: control signal, u(fc).
44


Figure 3.42: Pretrained linear controller step response: state vector, x(fc).
Figure 3.43: Pretrained linear controller step response: output error, e(k).
45


Figure 3.44: Pretrained linear controller step response: output mean squared error,
Ymae(k).
Figure 3.45: Pretrained linear controller step response: parameter estimates, 46


3.3 Conclusions
Neural network controllers work well for nonlinear systems where little is known about the
plant, its nonlinearities, and noise statistics. The linear controller performed better (lower
mean squared error) but has no inherent fault tolerance and made some assumptions about
the plant (linear, known noise statistics, known order). Although (as alluded to earlier)
the linear controller performed better with this particular plant, this may not always be
the case. The addition of hysteresis, dead zones, or other nonlinearities may make the
linear controller unsuitable.
Possible improvements to the neural controller would be to incorporate the activation
function parameters into the weight matricies rather than leaving them as constants (as
opposed to using the separate algorithms mentioned earlier). Also, a different training
algorithm such as the scaled conjugate gradient method developed by Moller [18] has
been shown to train much faster than the simple back propagation method [8].
The neural and linear methods each have trade-offs which the designer must be aware
of. A neural controller is able to function with more degrees of freedom (plant order,
variable nonlinearities, noise statistics, etc.) than a more conventional controller. The
most obvious disadvantage of a neural controller is the computational costs. The linear
controller required about 1 /4 the number of calculations as did the neural controller with
the fault tolerance active. Another issue that may concern the designer is inability to prove
global stability with neural networks. Although some progress has been made in recent
years to prove or disprove global stability, this has remained difficult hurdle to overcome.
Nevertheless, neural networks have proven to be effective methods of controlling many
types of systems.
47


Appendix A
Source Code Listings
All tlie m files used will work with Matlab Version 4.1 or above. The only routines used
that are not listed are two public domain tool boxes available from The MathWorks via
anonymous ftp (Internet address: ftp .mathworks. com). The first tool box creates legends
on a plot and the second allows the use of Greek symbols, subscripts, and superscripts in
text strings and axes labels. It is possible subsequent versions of Matlab will have these
features built in. Any code changes the reader may wish to make concerning this should
be minor and will not affect the results of any of the simulations.
Although the three layer network controller was not used, the code is included here for
the sake of completeness and to provide the reader with a mapping from the equations
given in Section 2.1 to their implementation in the m files.
A.l Main routine for the two layer NN controller
function proj 2(fflag,print it,pdev.plotid)
'/, PR0J2 A two layer neural controller.
*/.
'/, Usage: V proj 2(fflag,print it,pdev,plotid)
/ /. Inputs: fflag fault flag (1 = run in faulty mode, 0 = faulty mode off)
*/. 7. printit flag for writing plot files (1 = write files, 0 = dont write files).
7. 7. pdev printer device ("ps" for Postscript, "ljet3" for HP LaserJet III, etc.).
7. 7. plotid unique identifier (character string) that will be part of the plot file names.
7.
global PDEV PLQTID PRINTIT
'/.-----Set the default values for the input arguments:
48


if nargin == 0, fflag = 0; printit = 0; pdev = ps; plotid = []; end
if nargin == 1, printit = 0; pdev = ps; plotid = []; end
if nargin == 2, pdev = ps; plotid = []; end
if nargin == 3, plotid = ; end
fflag = abs(fflag) > 0;
PRINTIT = abs(printit) > 0;
PDEV = pdev;
PLOTID = plotid;
'/,-----Close any existing plots and initialize the random number generator:
close(get(0,Children));
randn(seed,0);
'/,-----Initialize some variables:
'/, Note: Some of the following arrays (such as the ones for plotting) do
'/, not need to be initialized. Natlab will automatically resize each
'/, array for each iteration, however this takes time. Building the
arrays beforehand will save computational time.
T = 0;
N = 750; '/. Number of Iterations.
X0 = 2; '/, Dimension of State Vector.
U0 = 1; */, Dimension of Input Vector.
Y0 = 1; '/, Dimension of Output Vector.
y,------Dimensions of each layer, eta is the step size:
nl = 6; '/, Delay line size = nl 1.
n2 = 15; '/, Number of neurons for 1st (input) layer.
n3 =1; '/. Number of neurons for 2nd (output) layer,
nout = 15; '/. Number of neurons for multiple outputs,
eta = 0.80; '/. Step size for training.
'/,-----Initialize the plant variables:
x = 0.1*ones(X0,l); '/, Initial Value for State Vector,
y = 0.1*ones(Y0,l); '/, Initial Value for Output Vector.
U = 0.1*ones(U0,l); */, Initial Value for Input Vector,
yd = 0;
e =0;
delu = 0.01; delta u(k) used for approximating the plant derivative.
'/,-----kv and kvmi are the index vectors used for the input delay line:
kv = nl:-l:2;
49


kvml = kv 1;
'/.-----Initialize the input and layer outputs:
Ur = zeros(nl.i); Oi = zeros(n2,l); Qj = zeros(n3,l);
Alpha_i = 0. l*ones(n2,1); Alpha_i2 = Alpha_i / 2;
Alpha_j = 0.04*ones(n3,l); Alpha_j2 = Alpha_j / 2;
V,------Network biases for each, layer:
Hi = 0*ones(n2,l);
Hj = 0*ones(n3,1);
'/,-----Slopes of the activation functions:
Ki = 2*ones(n2,l);
Kj = 10*ones(n3,l);
7, Initialize the weight matricies. The file weights2.mat contains the
'/, weight values at 350 interations.
cl = ones(l.nout);
'/.if exist(weights2.mat') == 2,
*/, load weights2.mat
'/.else
Wri = randn(nl,n2); Wij = randn(n2,U0);
'/.end
W1 = zeros(N,length(Wri(:))); Wl(l,:) = Wri(:);
W2 = zeros(N,length(Mij(:))); W2(l,:) = Wij(:)';
fil = []; fi2 = [] ; /, Indecies of failed neurons for each layer.
'/,-----Initialize the noise vectors:
Var_w = 0.002; Mean_w = 0; sVar_w = sqrt(Var_w); w = sVar_w*randn(X0,N) + Mean_w
Var_v = 0.002; Mean_v = 0; sVar_v = sqrt(Var_v); v = sVar_v*randn(l,N) + Mean_v
I.------Initialize the plot arrays:
E = zeros(N.l); Y = zeros(N.l); Yd = zeros(N.l);
X = zeros(XO.N); Uc = zeros(N.l); UV = zeros(N.nout);
I = zeros(N,n2); Ymse = zeros(N.l);
/t-------------------
'/.
'/, Begin iterations:


clc,home
disp( k u(k) o(k) yd(k)
flpO = flops;
for k = 2:11,
it fflag.
if k == 200, fil = [1 3 5]; fi2
if k == 300, fil = [1 3:5 7 8]; fi2
if k == 400, fil = [1 3:5 7 8 10 11]; fi2
if k == 500, fil = [1 3:5 7:13 15]; fi2
end
y(k) Ymse(k))
[10 11]; end
[2:5 10 11]; end
[2:7 10:12]; end
[2:7 10:12 14 15]; end
V.
The forward pass through the NN to generate the
control signal:
Ur(l) = yd;
[Oi,neti] = fnet_sig(Ur,Wri,Alpha_i2,Hi,Ki,fil);
if fflag,
[Uv.netj] = fnet_lin(0i,Wij*cl,Alpha_j2,Hj,Kj,fi2);
[U,cl] = majvote(Uv,cl);
else
[U.netj] = fnet_lin(0i,Wij,Alpha_j2,Hj,Kj,[]);
end
'/ 1st layer
'/ Fault flag
7. 2nd (output) layer
7, Majority vote
7. 2nd (output) layer
Back-propagate through the plant and network to update the weights:
[xt.ydu] = plant(k,x,U+delu,v(k),w(:,k),T);
[x,y] = plant(k,x,U,v(k),w(:,k),T);
p = e*(ydu y) / delu; 'l Partial of the plant when Bp and Cp are unknown.
P = Bp*Cp*e; 7, Partial of the plant when Bp and Cp are known.
[Wij,deltaj] = bp_lin(Wij,P,0i,Kj,Alpha.j2,eta,[]); 7 2nd layer
Wri = bp_sig(Wri, Wi jtdeltaj,Oi,Ur,Ki,Alpha_i2, eta,f il); 7. 1st layer
7.-------Update the input vector and calculate yd(k) and e(k):
Ur(kv) = Ur(kvml);
7. yd =1;
yd = 2 sin(5.0e-05*k"2);
e = yd y;
7.-------Put selected values in arrays that will be used for plotting.
Y(k) = y;
Uc(k) = U;
Yd(k,:) = yd;
E(k,l) = e;
X(:,k) = x;
51


if fflag, UV(k,:) = Uv; end
Wl(k,:) = Wri(:) ;
W2(k,:) = Wi j (:) ;
01(k,:) = 0i;
Ymse(k) = Ymse(k-l) + (E(k)2 Ymse(k-l)) / k;
/.----Display progress on screen:
if rem(k,25) == 0,
home, disp( )
disp( [sprintf (y,4. Of,k) sprintf (*%11.4e' ,U) ...
sprintf ('y,11.4e* ,e) sprintf (*%11.4e, yd) ...
sprintf (*y,11.4e,y) sprintf (*%11.4e ,Ymse(k,l))])
end
/,----Stop and plot results if something went wrong:
if any(Cisnan(U) isnan(yd) isnan(y) isnan(Ymse(k,1))]),
k = k 1;
disp(' ')* disp('Something broke!),disp( ')
break
end
end
nflops = flops flpO;
disp( sprintf (Number of floating point operations: '/,7.0f , nflops))
X--------------------------------------------------------------------------------
/.
'/. Do the plotting:
nl = [l:k];
plotyyd(nl,Y(nl),Yd(nl),output_signals2',Output Signal with 2 Layer Net)
plotspv(nl,X(:,nl),state_vector2,x,State Vector with 2 Layer Net)
plotue(nl,Uc(nl),control_signal2,u(k),Control Signal with 2 Layer Net)
plotue(nl,E(nl),output_error2,e(k),Output Error with with 2 Layer Net)
plotmse(nl,Ymse(nl),output_mse2,Output MSE with 2 Layer Net)
plotwt(N,Wl,nl*n2,layerl_weights2*,Wri(k),Layer 1 Weights with 2 Layer Net)
plotwt(N,W2,n2*n3,layer2_weights2,Wij(k),Layer 2 Weights with 2 Layer Net)
plotno(N,01,n2,layerl_output2,Oi(k),Layer 1 Outputs with 2 Layer Net)
if fflag,
plotno(N,UV,n2,layer2_output2mv,Uv(k),...
Majority Vote Input with 2 Layer Net)
end
%
52


A.2 Main routine for the Kalman/RLS linear controller
function projl(printit,pdev,plotid)
'/. PR0JL y An adaptive linear controller.
/ /, Usage: / proj1(printit,pdev,plotid)
ft V, Inputs: printit flag for uriting plot files (1 = write files, 0 = dont
y. write files).
7. pdev printer device ("ps" for Postscript, "ljet3" for HP
7. LaserJet III, etc.).
7. plotid unique identifier (character string) that will be part of
7. the plot file names.
/.
global PDEV PLOTID PRINTIT
if nargin == 0, printit = 0; pdev = ps; plotid = ; end
if nargin == 1, pdev = ps'; plotid = []; end
if nargin == 2, plotid = ; end
PRINTIT = abs(printit) > 0;
PLOTID = plotid;
PDEV =.pdev;
close(get(0,Children)); '/. Close any existing plot windows.
randn(seed,0);
T = 0;
N = 750; !, Number of Iterations.
X0 =2; '/, Order of the plant.
Nnum =2; % Number of numerator parameters to identify.
Nden =2; '/. Number of denominator parameters to identify.
Np = Nnum + Nden;
/.-----Initialize the plant variables:
u = zeros(N.l);
x = 0.i*ones(X0,N); '/, Initial Value for State Vector,
y = zeros(N,l);
y([l 2]) = 0.1*ones(2,l); % Initial Value for Output Vector,
e = zeros(N,l);
V.------Initialize the noise vectors:
Var_w = 0.005; Mean_w = 0; sVar_w = sqrt(Var_w); w = sVar_w*randn(X0,N) + Mean_w;
Var_v = 0.005; Mean_v = 0; sVar_v = sqrt(Var_v); v = sVar_v*randn(l,N) + Mean_v;
53


/.
Initial values for the Kalman/LS Parameter Estimator:
Lam = 0.9S;
'/if exist(paramstep.mat),
'/, load paramstep.mat
'/else
thest = zeros(Hp,1); thest(l) = 0.1; P = O.E*eye(Np);
'/.end
'/------Initialize the plot arrays:
Thest = zeros(Hp.H);
Ymse = zeros(N.l);
'/------Initialize the index vectors:
sti = max([Nnum Nden]); sil = sti 1;
kvn = sil:-l:sti-Nnum;
kvd = sil:-l:sti-Nden;
kvu = sil:-l:sti-Bnum+l;
kvy = sti:-l:sti-Bden+l;
y.-----------------------------------------------------------
/.
'/ Begin iterations:
clc.home
dispC k u(k) e(k) y(k) yd(k)
flpO = flops;
for k = sti+l:N,
kml = k 1; kvn = kvn + 1; kvd = kvd + 1; kvu = kvu +
'/.-----Kalman/RLS Parameter Estimator:
Phi = [u(kvn); -y(kvd)];
L = P*Phi / (Lam + Phi*P*Phi);
P = std(w(:,k))"2 + (P L*Phi*P) / Lam;
% ------ Calculate the desired output, plant state vector,
'/. yd(k) = 1;
yd(k) = 2 sin(5.0e-05*k*2);
[x(:,k),y(k)] = plant(k,x(:,kml),u(kml),v(k),w(:,k));
'/,-----Update the parameter estimates:
e(k) = y(k) thest*Phi;
thest = thest + L*e(k);
Ymse(k))
1; kvy = kvy + 1;
and plant output:
54


'/.-----Controller:
Phiu = [1; thest(2:Np)] / thest(l);
Tim = Cyd(k); -u(kvu); y(kvy)];
u(k) = Tim *Phiu;
'/,-----Put selected values in arrays that will be used for plotting.
Thest(:,k) = thest;
Ymse(k) = Ymse(k-l) + (e(k)*2 Ymse(k-l)) / k;
*/,-----Display progress on screen.
if rem(k,25) == 0,
home, disp( O
disp( [sprintf ('/,4.0f ,k) sprintf (/.ll^e,u(k)) ..
sprintf('/,11.4e,e(k)) sprintf ('/t11.4e ,y(k))
sprintf ('Xll.4e* ,yd(k)) ' sprintf (JCll^e ,Ymse(k))]);
end
7,-----Stop and plot results if something went wrong:
if any([isnan(u(k)) isnan(yd(k)) isnan(y(k)) isnan(Ymse(k,l))]) I e(k) > 50,
k = k 1;
disp( '), disp(Something broke!OidispC ')
break
end
end
nflops = flops flpO;
disp(sprintf(Number of floating point operations: /,7.0f',nflops))
X---------------------------------------------------------------------------
'/. Do the plotting:
nl = [l:k]';
plotyyd(nl,y(nl),yd(nl),'output.signalsl1,Output Signed with Linear Ctrl)
plotspv(nl,x(:,nl),state_vectorl,x,State Vector with Linear Ctrl)
plotue(nl,u(nl),'control_signall,u(k),Control Signal with Linear Ctrl)
plotspv(nl,Thest(:,nl),param_est,p,'Parameter Estimates)
plotue(nl,e(nl),output_errorl,e(k),Output Error with with Linear Ctrl)
plotmse(nl,Ymse(nl),output_msel,'Output MSE with Linear Ctrl')
55


A.3 Main routine for the three layer NN controller
function proj3(print it,pdev,plot id)
'/, PR0J3 A three layer neural controller.
'/, Usage: pro j 3 (print it, pdev, plot id)
'/. Inputs:
/.
7
7
/.
/.
printit
pdev
plotid
flag for writing plot files (1 = write files, 0 = don't
write files).
printer device ("ps" for Postscript, "ljet3" for HP
LaserJet III, etc.).
unique identifier (character string) that will be part of
the plot file names.
global PDEV PLOTID PRINTIT
if nargin == 0, printit = 0; pdev = 'ps'; plotid = ; end
if nargin == i, pdev = 'ps'; plotid = []; end
if nargin == 2, plotid = []; end
PRINTIT = abs(printit) > 0;
PDEV = pdev;
PLOTID =.plotid;
close(get(0, 'Children')); /. Close any existing plot windows.
randn(seed,0);
T = 0;
N = 750; */, Number of Iterations.
*/,T = 3/749; N = length(0:T:3); 7, Sampling period for the missile
fil = ; fi2 = ; fi3 = ; 7% Indecies of failed neurons for each layer.
7.------Initialize some variables.
7. Note: Some of the following arrays (such as the ones for plotting) do
'/, not need to be initialized. Matlab will automatically resize each
7% array for each iteration, however, this takes time. Building the
'/. arrays beforehand will save computational time.
X0 = 2; /, Dimension of State Vector.
U0 = 1; '/, Dimension of Control Vector.
Y0 = 1; /, Dimension of Output Vector.
7%------Dimensions of each layer, eta is the step size:
/nl = 11; /. Delay line size = nl 1.
/ji2 = 50; '/, Number of neurons for 1st (input) layer.
56


%n3 =40; % Number of neurons for 2nd (hidden) layer.
*/,eta = 0.10; '/, Step size for training.
nl =6; '/, Delay line size = nl i.
n2 =5; '/, Number of neurons for 1st (input) layer.
n3 =5; '/ Number of neurons for 2nd (hidden) layer.
n4 = U0; % Number of neurons for 3rd (output) layer,
nout = 5; '/, Number of neurons for multiple outputs,
eta = 0.80; '/, Step size for training.
/.-----Initialize the plant variables:
z = 0.1*ones(X0,l); V, Initial Value for State Vector,
y = 0.1*ones(Y0,l); '/. Initial Value for Output Vector.
U = 0.1*ones(U0,l); '/ Initial Value for Input Vector,
yd =0;
e =0;
delu = 0.01;
'/.-----kv and kvml are the index vectors used for the input delay line
kv = nl:-l:2;
kvml = kv 1;
'/,-----Initialize the input vector and output vectors for each layer:
Dr = zeros(nl,l);
Or = zeros(n2,l);
Oi = zeros(n3,l);
Oj = zeros(n4,l);
'/,-----Steepness values for the sigmoid function:
Alpha_r = 0.10*ones(n2,l); Alpha_r2 = Alpha_r / 2;
Alpha.i = 0.10*ones(n3,l); Alpha.i2 = Alpha.i / 2;
Alpha.j = 0.04*ones(n4,l); Alpha.j2 = Alpha.j / 2;
'/,-----Network biases for each layer:
Hr = 0*ones(n2,1);
Hi = 0*ones(n3,1);
Hj = 0*ones(n4,l);
'/,-----Scale factors for the output functions:
Kr = 2*ones(n2,l);
Ki = 2*ones(n3,l);
Kj = 10*ones(n4,l);
57


'/.-----Initialize the weight matricies. The file weights3.mat contains the
I, weight values at 350 interations.
cl = ones(l.nout);
'/.if exist('weights3.mat) == 2,
'/. load weights3.mat
'/.else
Wsr = randn(nl,n2); Wri = randn(n2,n3); Wij = randn(n3,n4);
'/.end
W1 = zeros(N,length(Wsr(:))); Wl(l,:) = Msr(:)j
W2 = zeros(N,length(Wri(:))); W2(i,:) = Wri(:);
W3 = zeros(N,length(Wij(:))); W3(i,:) = Wij(:);
!,------Initialize the noise vectors:
Var_w = 0.005; Mean_w = 0; sVar_w = sqrt(Var_w); w = sVar_w*randn(X0,N) + Mean_w
Var_v = 0.005; Mean_v = 0; sVar_v = sqrt(Tar_v); v = sVar_v*randn(l,N) + Mean_v
%-------Initialize the plot arrays:
E = zeros(N,1);
Y = zeros(N,1);
Yd = zeros(N,l);
X = zeros(X0,N);
Uc = zeros(N,l);
UV = zeros(H.nout);
R = zeros(F,n2);
I = zeros(R,n3);
Ymse = zeros(N,l);
%------------------------------------------------------------------------------
'/.
'/. Begin iterations:
clc,home
disp( k u(k) e(k) y(k) yd(k) Ymse(k))
flpO = flops;
for k = 2:N
'/. if k == 300, fil = [2 4]; end
'/, if k == 500, fi2 = [13]; end
'/,----The forward pass through the network to generate the control signal:
Ur(l) = yd;
[Or.netr] = fnet_sig(Ur,Msr,Alpha_r2,Hr,Kr,fil); '/, 1st layer (input layer)
[0i,neti] = fnet_sig(0r,Wri,Alpha_i2,Hi,Ki,fi2); '/. 2nd layer (hidden layer)
[U.netj] = fnet_lin(0i,Wij,Alpha_j2,Hj,Kj,[]); % 3rd layer (output layer)
58


7. [Uv.netj] = fnet_lin(0i,Wij*cl,Alpha_j2,Hj,Kj,fi3); '/, 3rd layer (output layer)
'/, [U,cl] = majvote(Uv,cl); 7, Majority vote
7,------Back propagate through the plant and network to update the weights:
7. [xt.ydu] = plant(k,x,U+delu,v(k) ,w(: ,k) ,T);
[x,y,Bp,Cp] = plant(k,x,U,v(k),w(:,k),T);
7. P = e*(ydu y) / delu; 7. Partial of the plant when Bp and Cp are unknown.
P = Bp*CpJ*e; 7. Partial of the plant when Bp and Cp are known.
[Wij.deltaj] = bp_lin(Wij ,P,0i,Kj,Alpha_j2,eta, []); 7. 3rd layer
[Wri.deltai] = bp_sig(Wri,Wij*deltaj,0i,0r,Ki,Alpha_i2,eta,fi2); 7. 2nd layer
Wsr = bp_sig(Wsr,Mri*deltai,0r,Ur,Kr,Alpha_r2,eta,fil); 7. 1st layer
7.-------Update the input vector and calculate yd(k) and e(k):
Ur(kv) = Ur(kvml);
7. yd = 1;
7. yd = sin(5.0e-05*k2);
7. yd = sin(T*k"2/100); 7. Used for missile
yd = 2 sin(5.0e-05*k2);
e = yd y;
7.-------Put selected values in arrays that will be used for plotting.
E(k,1) = e; Y(k,l) = y; Yd(k,l) = yd;
X(:,k) = x; Uc(k.l) = U; 7.UV(k,:) = Uv;
Wl(k,:) = Wsr(:); W2(k,:) = Wri(:)*; W3(k,:) = Wij(:);
OR(k,:) = Or*; DI(k,:) = Oi*;
Ymse(k) = Ymse(k-l) + (6*2 Ymse(k-l)) / k;
'/,----Display progress on the screen:
if rem(k,25) == 0,
home, disp( *)
disp([sprintf(74.Of,k) sprintf(711.4e,U) ...
sprintf ('/Jll^e,e) 1 1 sprintf ('711.4e, yd) ...
sprintf ('7.11.4eJ ,y) sprintf ('7.11.4e' ,Ymse(k,l))])
end
7.-------Stop and plot results if something went wrong:
if any([isnan(U) isnan(yd) isnan(y) isnan(Ymse(k,l))]),
k = k 1;
59


disp( ), disp(Something broke!),disp( )
break
end
end
nllops = flops flpO;
disp (sprint! (Humber of floating point operations: '/,7.0f .nflops))
y.--------------------------------------------------------------------------------
7.
7, Do the plotting:
nl = [l:k];
plotyyd(nl,Y(nl),Yd(nl),output_signals3,Output Signal with 3 Layer Net)
plotspv(nl,X(:,nl),state_vector3,x,State Vector with 3 Layer Net)
plotue(nl,Uc(nl),control_signal3,u(k),Control Signal with 3 Layer Net)
plotue(nl,E(nl),output_error3,e(k),Output Error with with 3 Layer Net)
plotmse(nl,Ymse(nl),output_mse3,Output MSE with 3 Layer Net)
plotwt(N,Wl,ni*n2,layeri_weights3,Wsr(k),Layer i Weights with 3 Layer Net)
plotwt(N,W2,n2*n3,Iayer2_weights3,Wri(k),Layer 2 Weights with 3 Layer Net)
plotwt(N,W3,n3,Iayer3_weights3,Wij(k),Layer 3 Weights with 3 Layer Net)
plotno(N,0R,n2,layerl_output3,Oi(k),Layer 1 Outputs with 3 Layer Net)
plotno(N,0I,n2,layer2_output3,Oi(k),Layer 2 Outputs with 3 Layer Net)
7,plotno(NtUV,n2,layer3_output3,Uv(k),Majority Vote Input with 3 Layer Net)
V.
A.4 Forward pass with a sigmoid activation function
function [o,net] = fnet_sig(x,W,lamlH,Klfi)
7, FNET.SIG performs a forward pass through one layer of a neural network with
% a sigmoid activation function on the output.
7.
7. Usage: [o.net] = fnet_sig(x,W,lam,H,K,fi);
/ 7. Inputs: X vector of inputs.
7. W the weight matrix.
7. lam steepness coefficient.
7. H neuron bias.
7. 7. K scale factor to adjust activation function upper bounds. and lower
7. 7. y fi is the index of the neuron in this layer that is (ie: output is set to 0). faulty,
7. Outputs: 0 output vector.
60


/.
net neuron output.
net = lam .* (W *x + H);
o = K .* tanli(net);
o(fi) = zeros(length(fi) ,1); '/, Set faulty neuron outputs to 0.
A.5 Forward pass with a linear activation function
function [o.net] = fnet_lin(x,W,lam,H,K,fi)
/, FNET.LIH performs a forward pass through one layer of a neural network with
'/. a linear activation function on the output.
/.
'/. Usage: / [o.net] = fnet_lin(x,U,lam,H,K,fi);
/ '/. Inputs: X vector of inputs
*/. W the weight matrix
*/. lam steepness coefficient.
7 H neuron bias
7. 7. K scale factor to adjust activation function upper bounds. and lower
*/. 7. / fi is the index of the neuron in this layer that is (ie: output is set to 0). faulty,
/ '/, Outputs: 0 output vector
net neuron output
net = lam .* (W*x + H);
o = K net;
o(fi) = zeros(length(fi),1); '/, Set faulty neuron outputs to 0.
7.
A.6 Back-propagation with a sigmoid activation function
function [Wnew.del.eta] = bp_sig(Wold,dW,Oi,Ii,K,lam.eta.fi)
'/. BP_NET performs the back propagation through one layer of a neural network
'/, with a sigmoidal activation function to update the weights for that layer.
/.
'/ Usage: [Wnew.del] = bp_sig(Wold,dW,Oi,Ii,K,lam,eta.fi);
/.
Inputs: Void
'/. dW
/.
/. 0i
/. Ii
last weight matrix.
delta from previous layer times the weight matrix from the
previous layer,
output of this layer,
input to this layer.
61


7. K
'/, lam
V. eta
*/. fi
7.
7.
/, Outputs: Wnew
*/. del
amplification factor of the sigmoid activation function,
slope of the sigmoid activation function,
training step size.
is the index of the neuron in this layer that is faulty,
(ie: derivative is set to 0).
updated weight matrix,
local error.
fp = K .* lam .* (1 0i.2);
fp(fi) = zeros(length(fi),l);
del = fp .* dW;
delW = eta Ii del;
Wnew = Wold + delW;
'/, Derivative of the sigmoid function.
7. Set faulty neurons to 0.
V, Local error.
'/, delta weight.
'/, Weight update.
7.=
A.7 Back-propagation with a linear activation function
function [Wnew,del] = bp_lin(Wold,dW,On,K,lam,eta.fi)
7> BP.LItf performs the bach propagation through one layer of a neural network
'/, with a linear activation function to update the weights for that layer.
y
'/. Usage: / [Wnew, del] 1 = bp_lin(Wold,dW,On,K,lam,eta.fi);
7. Inputs: : Wold last weight matrix
7. dW delta from previous layer times the weight matrix from the
7. previous layer.
7. On inputs to this layer.
7. K scale factor.
7. lam "slope" of the activation function.
7. eta training step size.
7. fi is the index of the neuron in this layer that is faulty,
'/, (ie: derivative is set to 0).
7.
'/, Hote: K*lam determines the slope of the linear activation function.
7.
y. Outputs: Wnew updated weight matrix
7. del local error.
fp = K lam;
fp(fi) = zeros(length(fi),1);
del = fp .* dW;
delW = eta On del;
Wnew = Wold + delW;
7. Derivative of the linear activation function.
7. Set faulty neurons to 0.
y. Local error.
7. Weight delta.
7. Weight update.
y.
62


A.8 Calculation of the nonlinear plant
function [xn,yn,B,C] = plant(k,x,u,v,w,T)
'/, PLANT calculates the state vector and output of the nonlinear plant.
'/. Usage: [xn,yn,B,C] = plant(k,x,u,v,w,T);
7.
'/, Inputs:
7.
7.
7.
7.
7.
7. Outputs:
7.
7.
'/.-----Model of the nonlinear plant:
A = [-0.4*x(2) 0.2*sin(0.05*k); 0.1*cos(0.1*k) -0.2]; B = [1; 0.7]; C = [10];
7[A,B,C,D] = sysmodel(T,k*T,le+6); */, Model of the missile
xn = A*x + B*u + w;
yn = C*xn + v; '/. xn is used since the current iteration is k + 1, not k
k iteration.
x state vector from previous iteration.
u control signal.
v,w measurement and system noises.
T sampling Period
xn.yn updated state vector and plant output.
B, C system matricies (optional).
7.
A.9 Majority vote calculations
function [xm.vlm] = majvote(x,vl)
7 MAJVOTE performs a majority vote scheme on the vector x.

7, Usage: [xm.vlm] = majvote(x,vl);
7
7* Inputs: x
7. vl
7.
7.
7. Outputs: xm
7. vim
7.
input vector.
vector containing ones where the "good" neuron are located from
the last iteration.
output scalar.
vector containing ones where the "good" neuron are located.
vli = find(vl);
lv = length(vli);
if lv > 2,
xt = x(vli); mien = [];
for k = l:lv,
63


mlen(k) = length(find(xt([l:k-l k+l:lv]) == xt(k)));
end
if 'any(mlen), x, error(All input values are different!), end
[mx,mi] = max(mien);
zi = find(mien "= mx);
oi = find(mlen == mx);
xm = xt(mi);
vim = vl;
vlm(vli(zi)) = zeros(length(zi),1);
vlm(vli(oi)) = ones(length(oi),1);
else
error(Not enough functioning neurons left on the output layer!)
end
*/.=
A.10 Plot Ymse(k)
function plotmse(xdata,ydata.plot.file,pt)
'/, PLOTMSE plots the output mean squared error signal, Ymse(k).
*/.
'/, Usage: plotmse(xdata,ydata,plot_file,pt)
7. Inputs: xdata, ydata x and y axis data to be plotted.
'/, plotfile name of the plot file
7, pt plot title
7.
global PDEV PLOTID PRINTIT
figure
set(gel,Name,Output Mean Squared Error)
plot(xdata,ydata),grid on,xlabel(Iteration (k))
ylabtex([ntex(Y),stex( mse),ntex( (k))])
if PRINTIT,
orient portrait
eval([print -deps /usr/jpeople/rogers/dass/thesis/doc/ plot.file
PLOTID .eps])
orient landscape
title(pt)
eval([print -d PDEV plot.file _ PLOTID . PDEV])
end
64


A. 11 Plot the neuron outputs
function plotno(np,0,n,plot_file,zlab.pt)
'/ PLOTNO plot the neuron outputs in 3 dimensions.
*/.
Usage: y plotno(np.0, n,plot_file.zlab.pt)
n '/. Inputs: : np number of data points (time axis)
/. 0 array of neuron outputs
*/. n number of outputs
plot_file name of the plot file
*/. zlab Z-Axis label
*/. pt plot title
global PDEV PLOTID PRINTIT
ss = 10;
'/.------The X-Axis labels:
kk = 0;
for k = 0:np/5:np,
kk = kk.+ 1;
xlab(kk.r) = sprintf ('/.3.0f ,k);
end
'/,-----Plot the outputs in three dimensions:
figure
set(gcf,Name,Neuron Output)
colormap(white)
mesh(0(l:ss:np,:))
setCgca,Box,on,...
XGrid.on,...
YGrid,on....
ZGrid,on....
XLim,[0 np/ss)....
XTickMode,manual,...
XTick,0:150/ss:np/ss,...
XTickLabels,xlab,...
YLim,[1 n],...
XLabel.text(String,Iteration (k)),...
YLabel,text(String,Neuron),...
ZLabel.text(String,zlab)....
View,[50 40]....
LineWidth,0.1)
------Write the plot to the output files:
65


if PRINTIT,
orient portrait
eval([print -deps /usr/people/rogers/class/thesis/doc/ plot_file
PLDTID .eps])
orient landscape
title(pt)
eval([print -d PDEV plot_file _ PLOTID . PDEV])
end
V.
A.12 Plot the state vector and parameter estimates, x(/e)
and §(k)
function plot spv(xdata,ydata,plot_file,ylab.pt)
V. PLOTTH plot the state vector or parameter estimates from the least squares
V, estimator.
7.
7, Usage: plotspv(xdata, ydat a, plot _f ile, ylab, pt)
7.
*/, Inputs: xdata, ydata x and y axis data to be plotted.
plot_file name of the plot file.
7. ylab y-axis label (see legend(.) below)
7. pt plot and figure title
7.
global PDEV PLDTID PRINTIT
figure
set(gcf,Name,pt)
ca = gca;
lxd = length(xdata);
xlm = [xdata(l) xdata(lxd)];
7.------Plot the data and generate the text for the legend:
linsty = str2mat(-,
lincol = ymcrgb;
k4 = 0;
k6 = 0;
hlegtxt = ;
[yr.yc] = size(ydata);
for k = l:yr,
k4 = k4 + i; k6 = k6 + 1;
if k4 > 4, k4 = 1; end
66


if k6 > 6, k6 = 1; end
hl(k) = line('XData,xdata,...
'YData',ydata(k,:),...
'LineStyle',deblank(linsty(k4,
Color,lincol(k6));
if strcmp(ylab,'p'),
hlegtxt = [hlegtxt 'Thest' int2str(k) (k)
else
hlegtxt = [hlegtxt 'x' int2str(k) (k)
end
end
11 = length(hlegtxt) 1;
'/,-----Put the X-axis and Y-axis labels and grid lines on the plot
set(gca,XLabel,text('String,'Iteration (k)'),'XLim',xlm,...
'Box','on','XGrid','on','YGrid','on');
if strcmp(ylab,'p),
ylabtex([gtex('q) stex( i) ntex( (k)')]);
else
ylabtex([ntex(x) stex(' i) ntex(' (k))]);
end
X ------ Resize the plot, put the legend on it, and draw the plot:
orient portrait
posn = getCgcf,'Position'); Wysiwyg
if yr > 4,
hleg = eval([legend( hlegtxt(l:ll) ,-1)]);
else
hleg = eval([legend( hlegtxt(l:11) )]);
end
axes(hleg).refresh
drawnow
X ------ Write the plot to the output files:
if PRINTIT,
orient portrait
axes(hleg).refresh
eval([print -deps /usr/people/rogers/class/thesis/doc/' plot_file
PLOTID .eps])
orient landscape
axes(ca); title(pt)
axes(hleg),refresh
eval([print -d PDEV plot_file _ PLOTID '. PDEV])
end
67


set(gcf,Position1,posn)
x=======================
A.13 Plot the control signal u(k) and error signal e(k)
function plotue(xdat a,ydat a,plot _f ile,ylab,pt)
7, PLOTUE plots either the control signal, u(k) or the output error signal, e(k).
7, Usage: plotue ( xdata, ydat a, plot_f ile.ylab.pt)
*/.
'/. Inputs: xdata, ydata x and y axis data to be plotted.
plot_file plot file name
7% ylab Y-Axis label
'/, pt plot title
7
global PDEV PLOTID PRINTIT
figure
7%------Give the figure the correct name and plot the data:
if strcmpCylab(l),u),
setCgcf,Name,Control Signal)
elseif strcmp(ylab(l),e),
set(gcf,Name,Output Error)
end
plot(xdata,ydata).grid on,xlabel(Iteration (k)).ylabel(ylab)
'/,-----Write the plot to the output files:
if PRINTIT,
orient portrait
eval([print -deps /usr/people/rogers/class/thesis/doc/ plot_file
PLOTID .eps])
orient landscape
title(pt)
eval([print -d PDEV plot_file _ PLOTID . PDEV])
end
7
A. 14 Plot the neuron input weights
funct ion plotnt(np,W,n,plot.f ile,zlab,pt)
'/, PLOTWT plot the weights in 3 dimensions.
68


/.
'/. Usage: plotwt(np,W,n,plot_file,zlab,pt)
X
V, Inputs: np number of data points (time axis)
X w array of weights
7. n number of weights
X plot_file name of the plot file
/. zlab Z-Axis label
/. pt plot title
global PDEV PLOTID PRINTIT
ss = 10;
'/.-----The X-Axis labels:
kk = 0;
for k = 0:np/5:np,
kk = kk + 1;
xlabCkk,:) = sprintf (/.3.0f ,k);
end
X-------Plot the weights in three dimensions:
figure
set(gcf,'Name1,Weights')
colormap(white)
mesh(W(l:ss:np,:))
setCgca,Box,on....
XGrid.on,...
YGrid.on,...
ZGrid,'on'....
XLim',[0 np/ss],...
XTickMode,manual,...
XTick,0:150/ss:np/ss,...
XTickLabels,xlab,...
YLim,[1 n],___
XLabel.text(String,Iteration (k)),...
YLabel,text(String,Weights),...
ZLabel,text('String,zlab),...
View,[60 40]....
LineWidth,0.1)
/,-----Write the plot to the output files:
if PRINTIT,
orient portrait
eval([print -deps /nsr/people/rogers/class/thesis/doc/ plot_file
69


PLOTID .eps])
orient landscape
title(pt)
eval([print -d' PDEV ' plot_file PLOTID PDEV])
end
V.
A.15 Plot the output y(k) and desired output y*(k)
function plotyyd(xdata,ydatal,ydata2,plot_file.pt)
7. PLOTYYD plots the output, y. and the desired output, yd.
7.
/. Usage: plotyyd(xdata,ydatal,ydata2,plot_file,pt)
7.
7. Inputs: xdata, ydatal, ydata2 x and y axis data to be plotted.
7. plot_file name of the plot file
7. pt plot title
7.
global PDEV PLOTID PRINTIT
'/.-----Plot the data:
figure
setCgcf,'Name',Output and Desired Output')
plot(xdata,ydatal,,xdata,ydata2,),grid on
ca = gca;
'/,-----Write the axes labels and the legend:
xlabel(Iteration (k))
ylabtex([ntex(y').stex(d),ntex( (k) andy(k))])
posn = get(gcf,'Position'); Wysiwyg
hleg = legend(y(k),yd(k));
drawnow
axes(hleg).refresh
V,------Write the plot to the output files:
if PRINTIT,
orient portrait
axes(hleg).refresh
eval(['print -deps /usr/people/rogers/class/thesis/doc/ plot_file
PLOTID .eps])
orient landscape
axes(ca); title(pt)
70


axes(hleg) .refresh
eval([print -d PDEV plot.file PLOTID PDEV])
end
set(gcf,Position,posn)
/.=
71


Bibliography
[1] J. T. Bialasiewicz, J. C. Proano, and E. T. Wall. Implementation of Intelligent
Controller Using Neural Network State Estimator. In Proceedings of the IEEE Inter-
national Symposium on Intelligent Control, 1989.
[2] A. Cichocki and R. Unbehauen. Neural Networks for Optimization and Signal Pro-
cessing. John Wiley and Sons, 1993.
[3] D. 0. Hebb. The Organization of Behavior, A Neuropsychological Theory. John
Wiley, 1949.
[4] T. T. Ho and H. T. Ho. Stochastic State Space Neural Adaptive Control. In Pro-
ceedings of the International Conference On Systems Engineering in Control and
Communication Systems, 1991.
[5] T. T. Ho, H. T. Ho, J. T. Bialasiewicz, and E. T. Wall. Stochastic Neural Direct
Adaptive Control. In Proceedings of the IEEE International Conference on Intelligent
Systems, 1991.
[6] J. J. Hopfield. Neural networks, and physical system with emergent collective com-
putational abilities. In Proceedings of the National Academy of Sciences, pages 79:
2554-58, 1982.
[7] D. L. Ince, J. T. Bialasiewicz, and E. T. Wall. Neural Network-Based Model Reference
Adaptive Control System. Technical report, University of Colorado at Denver, 1991.
[8] T. T. Jervis and W. J. Fitzgerald. Otimization Schemes for Neural Networks. Techni-
cal Report CUED/F-INFENG/TR 144, Cambridge University Engineering Depart-
ment, Trumpington St., Cambridge, CB2 1PZ, England, August 24, 1993.
[9] T. Kohonen. Associative Memory: A System-Theoretical Approach. Springer Verlag,
Berlin, 1977.
[10] T. Kohonen. A simple paradigm for the self-organized formation of structured feature
maps. In M Arbib S. Amari, editor, Competition and Cooperation on Neural Nets,
page Vol. 45. Springer Verlag, Berlin, 1982.
72


[11] T. Kohonen. Self-Organizing and Associative Memory. Springer Verlag, Berlin, 1984.
[12] T. Kohonen. The neural phonetic typewriter. IEEE Computer, 27(3):ll-22, 1988.
[13] L. Ljung. System Identification Theory for the User. Prentice-Hall, Inc., 1987.
[14] L. Ljung and J. Sjoberg. A System Identification Perspective on Neural Nets. Tech-
nical report, Department of Electrical Engineering, Linkoping University, Linkoping,
Sweden, May 27, 1992.
[15] T. L. McClelland and D. E. Rumelhart. Parellel Distributed Processing. MIT Press
and the PDP Research Group, 1986.
[16] W. S. McCulloch and W. H. Pitts. A logical calculus of the ideas immenent in nervous
activity. Bull. Math. Biophy., 1943.
[17] M. Minsky and S. Papert. Perceptrons. MIT Press, Cambridge, Mass., 1969.
[18] M. F. Moller. A Scaled Conjugate Gradient Algorithm for Fast Supervised Learning.
Neural Networks, 6:525-533, 1993.
[19] D. B. Parker. Optimal algorithms for adaptive networks: second order back propa-
gation, second order direct propagation and second order hebbian learning. In Pro-
ceedings of the International Conference on Neural Networks, pages 593-600. IEEE
Press, 1987.
[20] D. B. Parker. Learning-logic. Technical Report Invention Report S81-64, File 1, Office
of Technology Licensing, Stanford University, Stanford, CA, Oct. 1982.
[21] J. C. Proano. Neurodynamic Adaptive Control Systems. PhD thesis, University of
Colorado, 1989.
[22] D. Psaltis, A. Sideris, and A. Yamamura. A Multilayered Neural Network Controller.
In Proceedings of the IEEE International Conference on Neural Networks, June 21-24,
1987.
[23] F. Rosenblatt. The perceptron: A probabilistic model for information storage and
organization in the brain. Psych. Rev, 1958.
[24] D. E. Rumelhart, G. E. Hinton, and R. J. Williams. Learning internal representations
by error propagation. In D. E. Rumelhart and J. L. McClelland, editors, Parallel
Distributed Processing: Exploration in the Microstructure of Cognition. MIT Press,
Cambridge, Mass., 1986.
[25] P. J. Werbos. Beyond Regression: New Tools for Prediction and Analysis in the
Behavior Sciences. PhD thesis, Harvard University, Mass., 1974.
73


[26] B. Widrow. Generalization and information storage in networks of adaline neurons.
In G. Goldstein M. C. Jovitz, G. T. Jacobi, editor, Self-organizing Systems, pages
435-461. Spartan Books, 1962.
[27] B. Widrow and M. E. Hoff. Adaptive switching circuits. 1960 IRE Western Electric
Show and Convention Record, part 4:96-104, August 23 1960.
[28] J. M. Zurada. Introduction to Artificial Neural Systems. West Publishing Company,
1992.
74