Citation
Crowd counting using aerial thermography

Material Information

Title:
Crowd counting using aerial thermography
Creator:
Mauser, Lisa Anne
Publication Date:
Language:
English
Physical Description:
x, 56 leaves : illustrations ; 28 cm

Subjects

Subjects / Keywords:
Crowds -- Measurement ( lcsh )
Thermography ( lcsh )
Remote sensing ( lcsh )
Genre:
bibliography ( marcgt )
theses ( marcgt )
non-fiction ( marcgt )

Notes

Bibliography:
Includes bibliographical references (leaves 55-56).
General Note:
Department of Electrical Engineering
Statement of Responsibility:
by Lisa Anne Mauser.

Record Information

Source Institution:
|University of Colorado Denver
Holding Location:
|Auraria Library
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
757823680 ( OCLC )
ocn757823680
Classification:
LD1193.E54 2011m M38 ( lcc )

Full Text
CROWD COUNTING USING AERIAL THERMOGRAPHY
by
Lisa Anne Mauser
B.S., University of Colorado Boulder, 2007
A thesis submitted to the
University of Colorado Denver
in partial fulfillment
of the requirements for the degree of
Master of Science
Electrical Engineering
2011


This thesis for the Master of Science
degree by
Lisa Anne Mauser
has been approved
by
Tixsa Papantoni
Jan Bialasiewicz
Yiming Deng
Date


Mauser, Lisa Anne (M.S., Electrical Engineering)
Crowd Counting Using Aerial Thermography
Thesis directed by Professor Titsa Papantoni
ABSTRACT
This thesis explores the use of infrared photography in an aerial application, to count
the size of a crowd as seen from above. The use of thermographic images is
beneficial, since all members of a crowd will have similar infrared thermal signatures,
eliminating the need for the use of other distinguishing features that identify human
shapes.
Once the aerial infrared images have been obtained, the sequential detection of
change algorithm is used to detect the presence of individual human shapes in the
image. The algorithm utilizes Bernoulli models to distinguish between human shapes
and background and detects changes sequentially, from such shapes to background
and vice versa. The output of the detection of change algorithm is an image that
solely contains accentuated human shapes on a white background.
A novel counting algorithm was developed to identify and count the detected human
shapes in the image.


This abstract accurately represents the content of the candidates thesis. I recommend
its publication.
Signed
Titsa Papantoni


ACKNOWLEDGMENT
I wish to give my thanks to my advisor, Dr. Titsa Papantoni, for her unwavering
guidance and support in the development of this thesis. I would also like to thank Dr.
Yiming Deng for his insight and guidance on this topic.


TABLE OF CONTENTS
Figures...................................................................viii
Tables.......................................................................x
Chapter
1. Introduction.............................................................1
1.1. Motivation...........................................................1
1.2. Objectives...........................................................1
1.3. Outline..............................................................2
2. Thermographic Imagery....................................................3
2.1. Use of Thermographic Images in this Study............................5
3. Sequential Detection of Change Algorithm................................13
3.1. Use of the Sequential Detection of Change Algorithm in this Study...18
3.2. Calculating Threshold...............................................24
4. Counting Algorithm......................................................29
5. Results Using Sequential Detection of Change Algorithm..................33
6. Other Applications......................................................37
7. Conclusion..............................................................40
runcrowd.m..................................................................41
run cell.m..................................................................44
vi


imagetraining.m.......................................................47
imagetrainingcell.m...................................................48
falsealarmcurve.m.....................................................49
powercurve.m..........................................................51
runcurves.m...........................................................53
colorgray.m...........................................................54
Bibliography..........................................................55
Vll


LIST OF FIGURES
Figure 1. Infrared Spectrum ("Spectroscopy", n.d.)..................................3
Figure 2. FLIR i60 Camera (FLIR 160, n.d.)..........................................6
Figure 3. Overhead Infrared Image...................................................7
Figure 4. Image With Different Temperature Scale....................................8
Figure 5. Image With Normalized Intensities.........................................9
Figure 6. Aerial Crowd Image 1 (trp80).............................................10
Figure 7. Aerial Crowd Image 2 ([Purple Crowd Photograph]).........................11
Figure 8. Representative Aerial Thermographic Image.................................12
Figure 9. Cropped Section Representing Background...................................19
Figure 10. Cropped Section Representing a Person....................................20
Figure 11. Thresholded Section of Background........................................21
Figure 12. Thresholded Section of Person............................................21
Figure 13. Power Curve..............................................................25
Figure 14. False Alarm Curve........................................................26
Figure 15. Algorithm Detection of Background Image..................................27
Figure 16. Algorithm Detection of Person Crop.......................................28
Figure 17. Overhead Detection of Three People.......................................30
Figure 18. First Pixel Encountered..................................................31
Figure 19. Counted People...........................................................32
Figure 20. Representative Aerial Thermographic Image................................33
vm


Figure 21. Image Processed With Simple Threshold......................................34
Figure 22. Image Processed With Sequential Detection of Change Algorithm..............35
Figure 23. Cell Culture...............................................................37
Figure 24. Cell Culture Processed With Sequential Detection of Change Algorithm.......38
IX


LIST OF TABLES
Table 1. Emissivity of common materials
4
x


1. Introduction
1.1. Motivation
We are concerned with the accurate assessing of the size of large crowds. Current
available methods are susceptive to miscounting, misrepresentation and manipulation
of crowd sizes. The use of aerial thermography is a novel technology, well-suited to
the application at hand. The use of infrared imagery allows for the accurate detection
of people in a crowd. All members of the crowd have similar thermal signatures, and
can be identified as such. This removes the need for the use of additional features,
such as hair and skin color, attire, etc., that would be required in the use of traditional
imagery. Additionally, other objects that would be difficult to differentiate using
traditional imagery are easy to discard when viewed in the infrared spectrum.
This study explores the use of infrared imagery, combined with signal processing
techniques, to accurately detect and count the humans present in a crowd.
1.2. Objectives
The goal of this study is to explore the use of image processing techniques in the
analysis of infrared images, enabling the counting of the number of humans present in
the image. The steps necessary to achieve this are the following:
1


1. Obtain representative thermographic images, depicting an aerial view of a
crowd.
2. Use the sequential detection of change algorithm to detect human shapes in
the image.
3. Use a counting algorithm developed for this study to count the detected
human shapes in the image.
The overall objective is to identify and count the number of humans in the image of a
crowd.
1.3. Outline
This thesis is organized as follows. Chapter 1 contains the introduction, including
motivation and objectives. Chapter 2 details the use of thermographic imagery in the
thesis. Chapter 3 discusses the use of the sequential detection of change algorithm.
Chapter 4 outlines the counting algorithm used. Chapter 5 contains results. Chapter
6 details other applications, and Chapter 7 concludes the thesis.
2


2. Thermographic Imagery
Thermography is the detection of emitted energy in the IR spectrum. The exact
definition of infrared energy varies slightly from source to source, but in general
infrared energy is from approximately 0.7 to 300 pm in wavelength, depicted in
Figure 1.
Tn??o35In^v
10 HZ

io'Vu
Cosmic and Y-rays X-rays (UV) Vacuum Ultraviolet (UV) Near Ultraviolet Visible (NIR) Near Infrared (IR) Infrared Microwave Radio
0.1 nm 200nm 400 im 75C am 2 nm 50 tun

Figure 1. Infrared Spectrum ("Spectroscopy", n.d.).
The energy emitted, and subsequently detected, is a combination of the temperature
of the object and its emissivity. An object's emissivity at a given temperature is
defined as the ratio of the energy it radiates, to the energy emitted by a perfect
radiator, or blackbody, both at the same temperature." (Introduction to Infrared
Thermometers, n.d.). Thus, while infrared energy is often considered as a measure of
3


heat for an object, it also incorporates the amount of energy radiated. An object's
emissivity is a constant representing the material of its construction. A black body, or
a perfectly emissive body, has an emissivity of 1. A body that does not emit any
energy, but is, instead, a "perfect thermal mirror," has an emissivity of 0. (Emissivity
Table, n.d.). The emissivity of some common objects are listed in Table 1 below
(Emissivity Table, n.d.).
Table 1. Emissivity of common materials.
Aluminium: anodised 0.77
Aluminium: polished 0.05
Brass: highly polished 0.03
Brass: oxidized 0.61
Brick: common .81-.86
Brick: common, red 0.93
I Carbon: graphite, filed surface 0.98
Carbon: purified 0.80
Cement 0.54
Concrete 0.92
Glass 0.92
Rubber 0.95
Water 0.95
Thermography produces a visual representation of this measure of energy. To
produce a thermographic image, light is focused onto an array of infrared-detector
4


elements, which produce a representation of the detected temperatures called a
thermogram. This thermogram is subsequently represented by a set of electrical
impulses, which are then processed into a visual image. (How Thermal Imaging
Works, n.d.). In general, thermography images are natively produced in grayscale,
but are often artificially colorized for the ease of quick comprehension by the viewer.
For the purposes of this study, native grayscale images will be used.
Thermography is frequently utilized as a form of nondestructive analysis.
Information and observations of an environment can be gathered without disrupting
the subject under 'test.' Thermography is currently used in a variety of fields,
including medical diagnostics, material and building inspections, and defense
applications. This study utilizes reasonably accessible thermography technology to
provide a proof of concept regarding feasibility and the benefit of using the available
technology in an aerial application.
2.1. Use of Thermographic Images in this Study
The thermographic images used in this study are derived from images taken with a
FLIR i60 infrared camera, seen in Figure 2. The i60 has a Focal Plane Array (FPA)
detector, utilizing an uncooled microbolometer. It detects energy with wavelengths
from 7.5pm to 13pm, and produces images 180 by 180 pixels. It has a temperature
range of -20 degrees to 120 C, with +/- 2 degrees C of accuracy.
5


Figure 2. FLIR i60 Camera (FLIR i60, n.d.).
Images used in this study were taken with the i60, from a third floor roof of an on-
campus building. The intent was to gather a wide variety of representative
thermographic images, to understand the distinguishing image features between
human shapes and background, to analyze properties of human clusters and to
identify the effects of the time period within a day. The images were taken over a
period of late afternoon to early evening.
6


Due to the constraints of the available location with roof access, the ability to take
images truly representative of an aerial application was limited. However, we were
still able to gather valuable information from these images.
Figure 3. Overhead Infrared Image.
Figure 3 is representative of the type of image gathered. In the top left comer, the
temperature of the sensor target (seen in the center of the image) and the emissivity of
the target image are shown. As directed by the camera manufacturer, a value e = 0.96
is appropriate for most applications, and is the value used for the collection of the
shown images. The bottom of the image shows the temperature scale, in degrees
Celsius, of the temperatures detected in the image field of view.
7


Naturally, for use in this project, the extraneous elements displayed on the screen will
need to be removed prior to processing. Additionally, due to the nature of the device
and the various temperatures in which the images were collected, there is some
variance in the temperature scale (and, accordingly, intensity scale). Figure 4
illustrates the variance in background and person intensities. In addition to the
noticeable difference in background and person intensities, the scale at the bottom
shows that the temperature range corresponding to black and white is different than
that of the previous image. The variation in intensities can be corrected by
normalizing the input intensities in MATLAB prior to processing.
3.6 c $FUR
e=0.96
Figure 4. Image With Different Temperature Scale.
8


The intensity ranges for both images are 1 to 256. In Figure 3, the 'background'
portions of the image have a nominal range of 75-115, and the 'people' portions of the
image range from 115-235. In Figure 4, the 'background' portions of the image have
a nominal range of 49-80, and the 'people' portions of the image have a nominal range
of 80-235. To bring the intensity ranges closer to a nominal range, defined for this
project based on the range seen in Figure 3, 25 was added to the intensity of each
pixel on Figure 4. Figure 5 shows the resulting image.
3.6 c OFLIR
e=0.96
Figure 5. Image With Normalized Intensities.
It can be seen that the intensity composure of this image is closer to that seen in
Figure 3, allowing for use of the same processing thresholds on both images.
9


Difficulty arises when considering these images' usability as representative of a truly
aerial application. Aerial crowd images would not be collected from a rooftop three
stories high, but from a craft travelling above the crowd. The images collected from
such an application would feature little to no profile data of a person obtained using a
side angle, but would feature the heads and shoulders of each member of the crowd.
10


Figure 7. Aerial Crowd Image 2 ([Purple Crowd Photograph]).
While we were not able to obtain images of these characteristics with the infrared
camera, we can use the images we were able to collect and alter them to create head
and shoulder representative images. From the collected images, there are adequate
overhead representations of'head and shoulder' sections for selection. Located quite
closely together, in a fashion representative of an aerial overhead crowd shot, we can
approximate aerial overhead crowd images from collected thermal images.
11


>
t *
*t> *',
r,$ > * i
A *
Figure 8. Representative Aerial Thermographic Image.
This image uses collected thermal image representations, plus observed
characteristics of aerial crowd images, to create thermal crowd images acceptable for
processing. We know that the thermal properties are those that would be collected
from a target system. We may also observe that the crowd composition is similar to
that expected crowd. The images were normalized to equivalent thermal ranges, and
they contain sufficient background noise to necessitate utilization of the sequential
detection of change algorithm (vs. a simple thresholding method).
12


3. Sequential Detection of Change Algorithm
Let xn denote an n-dimensional vector of data values and let fo(xn) and fi(xn) denote
the n- dimensional density functions of two "well known, distinct, discrete-time, and
mutually independent stochastic processes. The initial algorithm assumes that the
data sequence is initially generated by the process whose density function is fo,
where there is the possibility that the active data-generating process may shift at some
random point to that represented by the density function ft, instead" (Kazakos, 1990).
The algorithm can be generalized to the case where the data-generating process may
shift among M distinct and mutually independent processes, instead. For the
purposes of this study, however, we will assume the possible presence of just two
processes. We will also assume that both of these processes are Bernoulli. For our
specific application, the following algorithm is used, adapted from (Papantoni-
Kazakos, 2010 and Kazakos, 1990).
For each of the two processes, fo and ft, we assign the Bernoulli parameters p and q,
respectively. We then select a threshold t > 0 and implement the algorithm, as
follows:
We observe the data and update the algorithmic value T sequentially, where the
adaptations of the value T are performed as shown by the expressions below.
13


T(0) = 0
(3.1)
T(xn) = max(0,r(xn_1) + 5n(x"))
where,
(3.2)
9i{x') = log
/b(^l4_1)
(3.3)
The algorithm stops the first time that the value T crosses the threshold t, at which
point it is declared that the shift from the process represented by the density function
fo to that represented by the density function ft has occurred.
Per Kazakos (1990), we can see that this sequential test has two thresholds, 0 and t,
with 0 being a "reflecting barrier" and with t being an "absorbing or decision barrier".
For a Bernoulli processes, after some modifications and scaling, the algorithmic
values T^x") may be sequentially updated as shown below:
T(0) = 0
r(x") = max(0,T(xn~1') + zn)
zn= xn + K<7-P)
(3.4)
(3.5)
(3.6)
14


where,
r(p-q) =
log
i -q
i-p
q(i ~ p)
log
p(i q)
(3.7)
For the purpose of convenient numerical evaluations, y may be approximated by a
rational number, as follows:
r(p-q) =
S
(3.8)
Consequently, the algorithmic adaptations r(xn) are then transformed to the
following:
T'(0) = 0
(3.9)
r'(x") = max(0,T'(x"-1) + yn)
(3.10)
where,
yn = sxn l
(3.11)
The performance of the algorithm, and the viability of the selected threshold, can be
evaluated via the induced power and false alarm curves, as discussed by Kazakos
15


(1990). Specifically, let us define Prit(n), as "the probability that the threshold t is
first crossed at the time instant n, under the condition that the p to q change actually
occurred just after the datum xr." We then define /^(n) and Pat(n), the false alarm
and power curves, respectively. The false alarm curve indicates the probability of the
algorithm crossing the threshold t at each evaluated time, when the transition from p
to q processes never occurred. The power curve, on the other hand, indicates the
probability of crossing the threshold t at each evaluated time, when the p to q
transition has already occurred (represented as time = 0).
From Kazakos (1990), the following recursive expressions are used to numerically
calculate the false alarm and power curves:
i. If t-1 >= s > 1+1
pTlt(n) = C1 v)2|=oP(n ~ x'0
(3.12)
Prf(n,;) = (1 v)Prt(n- l,j + 1); 1 (3.13)
PrjK;') = (1 v)PrAn W + 1) + vPrAn l,j S + Q;s ~ l l;l (3.14)
16


Pr>t(nJ) = (1 v)Prf(n l,j- s + l),t l-l ii. If t-1 >= s = 1+1
= (1 ^lUo^rjC71 ~ I/O
(3.16)
Pr,f (^y) = (1 v)PrJt(n W + 1) + t?Pr (3.17)
Prt(n,j) = (1 v)Prt(n l,j + l);s l (3.18)
PrAn>D = (1 v)PrAn 1 ,j l);t- 1- l where,
(3.20)
Pr.t(n) = vZj^t-s-H PrAn ~ - 1 > 5 > l + 1
(3.21)
17


From this definition of Pr>f (n), we can calculate the false alarm and power sets.
From Kazakos (1990), we specifically define,
a(p, q,n,t) = XT=o^t(0 (3.22)
P(p, q, n, t) = Xr=o^o,(0 (3.23)
The a( )and p( ) functions above represent the probabilities that the threshold t is
crossed by the algorithm before or at time n, given the Bernoulli parameters p and q,
under false alarm versus power conditions, respectively. These values, the false alarm
and power curves, can be use to evaluate possible thresholds for given values of the
Bernoulli parameters p and q.
3.1. Use of the Sequential Detection of Change Algorithm in this Study
The first step in implementing the Sequential Detection of Change algorithm for the
study in this thesis consists of determining the light/dark thresholds for the target
images. For the application under study, the algorithmic objective is to detect the
change from background pixels to person pixels, and from person pixels to
background pixels. A determination needs to be made as to what brightness of pixels
nominally represents a section of background, and what brightness of pixels
nominally represents a section of person.
18


It is observed that, due to the nature of the thermographic images, the image sections
that contain a person are much brighter than those of background. Two equally sized
image sections are considered, one containing a person, and one containing a
representation of the background. For the purposes of this study, the background is
considered not just to be the ground, but also include extraneous portions of people
that won't be used for detection or counting of human subjects. For this threshold
determination, the image portion considered as representing background is actually a
portion containing a person's back carrying a backpack. Each image portion used is a
grayscale image with pixel values from 1 to 256, and 14x18 pixels in size.

Figure 9. Cropped Section Representing Background.
19


Figure 10. Cropped Section Representing a Person.
It is noted that, as compared to the pixel brightness that appears in an image section
representing a person, a background section containing light pixels was selected.
This accentuates the performance of the algorithm, allowing for the interpretation of a
nominal amount of bright pixels in the background as noise. The anticipation of
image background noise, in the form of light pixels, is part of the algorithm's strength.
For the two image sections shown in Figure 9 and Figure 10, a determination must be
made as to the brightness level of a pixel that is deemed to indicate the presence of a
person. This is a manual operation to some degree, and is part of the algorithmic
training discussed in Section 3. Out of a scale of 1 to 256, it was determined that a
value greater than 160 indicates a person, and a value of 160 or lower is deemed to
belong to the background. The following images illustrate this selected threshold.
Pixels above the threshold, or greater than 160, are kept at the original value. Pixels
20


below the threshold are colored black. This is for illustrative purposes only, allowing
the reader to understand what components of each image are above the threshold.
Figure 11. Thresholded Section of Background.
Figure 12. Thresholded Section of Person.
Figure 11 shows that there is a fair number of pixels above the light threshold present
in the background. This is reflective of the background environment present in the
21


images, and allows the algorithm to compensate for noise. Figure 12 shows that, with
the exception of a few pixels in the center of the head, most of the pixels of the
person's head are above the light threshold.
As discussed in Section 3, background and person are assumed to be processes of a
Bernoulli distribution. To find the Bernoulli variable for each process, we consider
the number of pixels above the light threshold, divided by the number of total pixels,
p represents the Bernoulli variable for the background process, and q represents the
Bernoulli variable for the person process.
p = bg light count/(m*n); (3.1.1)
q = person_light_count/(m*n); (3.1.2)
The values p and q are found to be the following:
p = 0.1230
q = 0.3770
Since the above two image two sections are used to train the algorithm, these p and q
values will be used in the execution of the detection of change algorithm, when used
22


on full images containing the approximate light/dark characteristics of the training
images.
As discussed in Section 3, the computed p and q values are used to calculate y, where
y(p,q) =
log
i- q
l-p
q(i p)
log
p(i q)
(3.1.3)
Substituting the p and q values found above, we find y = -0.2399. We subsequently
approximate y as a rational number,
r(p,q)
1
s
(3.1.4)
For the y = -0.2399 value, we approximate 1=11 and s = 50. Thus, T(xn) becomes
T'(xn), where
T'(0) = 0
T\xn) = max(0,T'(x ) + yn)
yn = sxn i
(3.1.5)
(3.1.6)
(3.1.7)
23


3.2. Calculating Threshold
The algorithm decides that a change has occurred when 7'(x") > t. We begin by
assuming that we start in the background. Then, the algorithm is monitoring for a
change from background into person. When a change from background to person is
detected, the algorithm resets, and this time performs as if a person is present,
monitoring for a change to background. Deciding an appropriate threshold t is
important for the best performance of the algorithm. From Kazakos (1990) we know
that the following must hold:
t 1 > s > 1+ 1
(3.2.1)
y > IT 1 holds for the selected values s and 1, thus the threshold t must be greater
than or equal to 51 for this application.
Using the methods described in Section 3, we look at the power and false alarm
curves for a set of thresholds meeting these criteria.
24


Probability
Power Probabilities (Beta)
Figure 13. Power Curve.
25


8
False Alarm Probabilities (Alpha)
Figure 14. False Alarm Curve.
Note that the threshold values shown are normalized by the value of s. Since the
values p and q are reasonably far apart, the values for various thresholds are 'drawn
further apart' than in the example discussed in Section 3 (Kazakos 1990). It is seen
that, for the values of p and q, any of the demonstrated thresholds will perform
exceptionally well for the number of samples (pixel width of an image) that we could
reasonably expect to encounter.
26


With these in mind, a threshold of 250 (when normalized by s, 5) was selected for
use.
To illustrate the performance of the algorithm, the training images discussed in
Section 3.1 are run through the algorithm. The background and person images,
exhibited in Figure 9 and Figure 10 are used as inputs to the algorithm. Below, we
present the results.
Figure 15. Algorithm Detection of Background Image.
27


Figure 16. Algorithm Detection of Person Crop.
As can be seeing from Figure 15 and Figure 16, the algorithmic performance is quite
satisfactory for both the background and person image sections. For the background,
we see that the threshold was never crossed. A transition from background to person
was never found to have occurred, despite the presence of a fair number of light
pixels in the image. For the person image, a concise blob indicates the region of
detection of the person, with no interference from the background. Additionally,
despite the presence of some darker pixels in the center of the person, a transition
from person to background is never detected.
28


4. Counting Algorithm
The resulting image from the detection of change algorithm is now ready to be
processed by the counting algorithm. The output of the detection of change algorithm
is an image of the same dimensions as those of the original image, with black pixels
(value = 1); where a person has been detected by the algorithm and white pixels
(value = 256); where the image has been detected as background.
The present counting problem falls into the category of blob detection. There has
been some research in recent years concentrating around blob detection. Danker
(1981) presents a specific method that utilizes probabilistic analysis of an image's
interior and edge characteristics, to detect and ultimately count objects in infrared
images. Kothari presents an end-to-end process of edge detection and ellipse fitting
to identify and present for counting individual cells in an image. Yang uses polygon
analysis to detect and count people in an image with horizontal perspective.
For the images used in this thesis, the sequential detection of change algorithm
accentuates the contained blobs sufficiently, to facilitate counting. Thus, a novel
method of counting detected blobs is presented. Let the following image be a
representative outcome from the sequential detection of change algorithm.
29


Figure 17. Overhead Detection of Three People.
Figure 17 represents the detection of three overhead persons. It can be seen that
neither of the three blobs detected are completely circular in nature, ruling out the use
of any simplistic methods of detection based on the pixel locations and properties of a
circular blob. In fact, while the approximate size of an overhead person can be
assumed consistent within a certain mean and variance, the individual shapes may
vary significantly.
The algorithm developed for this paper evaluates the image row by row, examining
each pixel to determine whether it is black. When a black pixel is detected, the color
of the pixel is changed to gray.
30


Figure 18. First Pixel Encountered.
It can be seen that the first black pixel encountered in Figure 17 is the leftmost pixel
in the first row of the middle person. The middle person is oriented highest, and thus
the indicated pixel is the first encountered by the algorithm. The encountered pixel
has been turned gray. A recursive algorithm is now entered, evaluating the pixel
above, below, left, and right of the targeted pixel, in that order, for color. If the pixel
is black, it is changed to gray, and the pixels above, below, left, and right of that pixel
are then evaluated. This ensures that all the pixels that comprise a single person are
only counted once, regardless of the geometry of the persons shape.
Upon exiting the recursive algorithm, the number of recursions encountered is
evaluated. This number is indicative of the number of pixels encountered in the blob.
31


If the number of pixels encountered is greater than 25, the person count is
incremented by one. This ensures that small regions, if encountered, are discarded,
deemed not significant enough to indicate a person.
For this example, upon exit of the algorithm, the image appears as shown below, and
the person count is equal to 3.
Figure 19. Counted People.
32


5. Results Using Sequential Detection of Change Algorithm
At this point, the sequential detection of change algorithm has been properly 'trained',
and is calibrated for use on images of similar quality as those used to tune the
algorithm. The following image, discussed in Section 2.1, is the target image for
verification of the algorithm.
f l> t
Figure 20. Representative Aerial Thermographic Image.
It can be observed and manually counted that there are 35 people present in this
image.
The image is transformed via the binary algorithm, with 160 set as the intensity
threshold, both y values -0.2399, and the threshold for T at 250. For comparison,
4}^ *


33


Figure 21 shows the binary image after passing through a simple thresholding filter.
All pixels valued at 160 or below are shown black; all pixels valued above 160 retain
their pixel value.
Figure 21. Image Processed With Simple Threshold.
It can be seen that, in addition to the subjects' heads, there is a significant amount of
noise above the intensity threshold in the image. This illustrates the need for the
sequential detection of change algorithm to accentuate the human shapes, as well as
to filter noise.
Figure 22 is the resulting image after processing using the sequential detection of
change algorithm. It can be seen that the portions of heat from the heads and
34


shoulders are detected concisely, while the noise observed in Figure 21 has been
removed.
Figure 22. Image Processed With Sequential Detection of Change Algorithm.
The latter image is now passed through the counting algorithm. As discussed in
Section 3, the algorithm discards the small sections seen in the image that are a result
of darker portions in the person detection, since they are not significant enough to trip
the threshold. This is easily remedied by using a pixel threshold for each blob, when
35


the counting algorithm is implemented. The counting algorithm is executed as
described in Section 4, and results in the detection of all 35 people.
36


6. Other Applications
One other possible application is as a method for counting cells present in an image.
The following image, source unknown, depicts a culture of cells.
Figure 23. Cell Culture.
The detection of cells in the image is similar to the crowd counting application, with a
couple of notable differences. In the overhead crowd application, dark intensity is an
indicator of background and light intensity is an indicator of a person. In the cells
image, light intensity indicates background, and dark intensity indicates the presence
of a cell, instead. Additionally, the overlap of cells is much more significant than the
overlap of people. In numerous instances, there is complete overlap of cells, a
phenomenon encountered much less frequently in images of crowds.
37


For the image of cells, we take a similar approach as that taken for the images of
crowds.. First, a light/dark threshold is selected, in which pixels above a certain
intensity are deemed cells, and pixels below a certain intensity are deemed
background. Next, the algorithm is trained, determining the Bernoulli coefficients for
background and cell processes, and selecting a value y based on these coefficients.
Finally, after selecting an algorithmic threshold for the algorithmic function T, the
image is processed. Figure 24 illustrates the result.
I
i
Figure 24. Cell Culture Processed With Sequential Detection of Change Algorithm.
The algorithm does a remarkable job of filtering out the background which makes the
cells difficult to count. Due to the significant overlap of cells, there is not as clean a
38


separation between individual cells as there is with the separation of individuals in
aerial crowd images. For use in this application, a more specific counting algorithm
would be beneficial. A measured understanding of the nominal cell size will enable
an estimate of the number of cells present in a larger blob.
39


7. Conclusion
This thesis demonstrates the viability of image processing techniques in the analysis
of aerial infrared images, for the counting of humans present in a crowd. We have
first demonstrated the practicality of producing infrared aerial images that accurately
capture images of a crowd from above. We have then explored the use of the
sequential detection of change algorithm, and its powerful characteristics, in
processing such images. Finally, we presented a novel blob counting algorithm that
may accurately and efficiently count the members of a crowd from a so processed
image.
The techniques demonstrated in this thesis may be easily applied to aerial images
taken with an IR camera, providing the ability to accurately detect the number of
humans present in the crowd.
40


APPENDIX A
run crowd.m
close all
clear all
[ im__rgb, map2 ] = imread('crowd vl.jpg');
im = rgb2gray(im_rgb);
[m, n] = size (im) ;
person_light_count = 0;
for i = l:m
for j = l:n
if (im(i,j) > 160)
person_light_count = person_light_count + 1;
else
im(i,j) = 0;
end
end
end
figure
imshow(im)
p = 0.1230;
q = 0.3770;
gamma_bg = (log2((1-q)/(1-p)))/(log2((q*(1-p))/(p*(1-q))));
gamma_person = (log2((1-p)/(1-q)))/(log2((p*(1-q))/(q*(1-p))))
1 = 11;
s = 50;
[a, b] = size (im) ;
T(1) = 0;
person_check =0;
threshold_bg = 250;
threshold_person = 250;
for j = l:a
T(j,l) = 0;
T_image(j,l) = 255;
for i = 2:b
if person_check == 0
if (im(j,i) > 160)
T(j,i) = T(j,i-1) + s 1;
41


if (T{j,i) < 0)
T(j,i) = 0;
end
else
T(j,i) = T(j,i-1) 1;
if (T(j,i) < 0)
T(j,i) = 0;
end
end
if T(j,i) >= threshold_bg
T_image(j i) = 0;
person check = 1;
T(j,i) = 0;
else
T image(j,i) = 255;
end
elseif person_check == 1
if (im(j,i) <= 160)
T(j,i) = T (j,i-1) + s 1
if (T(j,i) < 0)
T(j,i) = 0;
end
else
T(j,i) = T (j i-1) 1;
if (T(j,i) < 0)
T{j,i) = 0;
end
end
if T(j,i) >= threshold_person
T_image(j,i) = 255;
person_check = 0;
T (j,i) = 0;
else
T_image(j,i) = 0;
end
end
end
end
figure
imshow(T_image)
person_count = 0;
rec_size = 0;
for j = l:a
for i = l:b
42


0
if T_image(j,i) ==
rec_size = 0;
[T_image,rec_size] =
color_gray(j,i,a,b,T_image,rec_size);
if rec_size > 25
person_count = person_count + 1
end
end
end
end
person_count
43


APPENDIX B
run cell.m
close all
clear all
[im_rgb,map2] = imread('cells.jpg');
im = rgb2gray(im_rgb);
[m, n] = size (im) ;
person_light_count = 0;
for i = l:m
for j = l:n
if (im(i,j) < 110)
person_light_count = person_light_count + 1;
else
im(i,j) = 255;
end
end
end
figure
imshow(im)
p = 0.9359;
q = 0.1762;
gamma_bg = (log2((1-q)/(1-p)))/(log2((q*(1-p))/(p*(1-q))));
gamma_person = (log2((1-p)/(1-q)))/(log2((p*(1-q))/(q*(1-p))))
1 = 60;
s = 100;
[a,b] = size(im);
T (1) = 0;
person_check = 0;
threshold_bg = 200;
threshold_person = 200;
for j = l:a
T(j,1) = 0;
T_image(j,l) = 255;
for i = 2:b
if person_check == 0
if (im(j,i) < 110)
T (j,i) = T(j,i-1) +s-l;
44


if (T(j, i) < 0)
T (j i) = 0;
end
else
T(j,i) = T (j,i-1) 1;
if (T(j,i) < 0)
T (j i) = 0;
end
end
if T(j,i) >= threshold_bg
T_image(j,i) = 0;
person_check = 1;
T(j,i) = 0;
else
T_image(j,i) = 255;
end
elseif person_check == 1
if (im(j,i) >= 110)
T(j i) = T(j,i-1) + s 1
if (T (j,i) < 0)
T (j i) = 0;
end
else
T(j,i) = T(j,i-1) 1;
if (T(j,i) < 0)
T(j,i) = 0;
end
end
if T(j,i) >= threshold_person
T_image(j,i) = 255;
person_check = 0;
T(j,i) = 0;
else
T_image(j,i) = 0;
end
end
end
end
figure
imshow(T_image)
person_count = 0;
rec_size = 0;
for j = 1:a
for i = l:b
45


0
i f T_image(j i) ==
rec_size = 0;
[T_image,rec_size] =
color_gray(j,i,a,b,T_image,rec_size);
if rec_size > 25
person_count = person_count + 1
end
end
end
end
person_count
46


APPENDIX C
imagetraining.m
close all
clear all
[b_crop_rgb,mapl] = imread('b_crop.jpg');
b_crop = rgb2gray(b_crop_rgb);
imshow(b_crop)
% imtool(b_crop)
[p_crop_rgb,map2] = imread('p_crop.jpg');
p_crop = rgb2gray(p_crop_rgb);
figure
imshow(p_crop)
[m,n] = size(p_crop);
person_light_count = 0;
for i = l:m
for j = l:n
if (p_crop(i,j) > 160)
person_light_count = person_light_count + 1
else
p_crop(i,j) = 0;
end
end
end
figure
imshow(p_crop)
bg_light_count = 0;
for i = l:m
for j = l:n
if (b_crop(i,j) > 160)
bg_light_count = bg_light_count + 1;
else
b_crop(i,j) = 0;
end
end
end
figure
imshow(b_crop)
p = bg_light_count/(m*n);
q = person_light_count/(m*n);
47


APPENDIX D
image training cell.m
close all
clear all
[b_crop_rgb,mapl] = imread('cells_bg.jpg');
b_crop = rgb2gray(b_crop_rgb);
imshow(b_crop)
[p_crop_rgb,map2] = imread('cells_cell.jpg');
p_crop = rgb2gray(p_crop_rgb);
figure
imshow(p_crop)
[m,n] = size(p_crop);
person_light_count = 0;
for i = l:m
for j = l:n
if (p_crop(i,j) < 110)
person_light_count = person_light_count + 1
else
p_crop(i,j) = 0;
end
end
end
figure
imshow(p_crop)
bg_light_count = 0;
for i = l:m
for j = l:n
if (b_crop(i,j) > 110)
bg_light_count = bg_light_count + 1;
else
b_crop(i,j) = 0;
end
end
end
figure
imshow(b_crop)
p = bg_light_count/(m*n);
q = person_light_count/(m*n);
48


APPENDIX E
false_alarm_curve.m
function beta = false_alarm_curve(n, thresh)
P(l,l) = 1;
P(1,2:n+3) = 0;
p = 0.1230;
q = 0.3770;
s = 4 0;
1 = 1;
v = p;
t = round(thresh*s);
if (t-l>=s && s==l+l)
for y = l:n+l
p_sum = 0;
for b = 1:1+1
p_sum = p_sum + P(y,b);
end
P(y+l,l) = (1-v)*p_sum;
for z = l:n+l
if t-l-l P(y+1,z+1) = v*P(y,z);
elseif l P(y+l,z+l) = (1-v)*P(y,z+1+1) + v*P(y-1,z-1);
end
end
end
elseif (t-l>=s && s>l+l)
for y = l:n+l
p_sum = 0;
for b = 1:1+1
p_sum = p_sum + P(y,b);
end
P(y+l,l) = (1-v)*p_sum;
for z = l:n+l
if l<=z && z P(y+l,z+l) = (1-v)*P(y,z+1+1);
elseif s-l<=z && z<=t-l-l
P(y+l,z+l) = (1-v)*P(y,z+1+1) + v*P(y,z-s+1+1)
elseif t1l P(y+l,z+l) = v*P(y,z-s+1+1);
end
end
49


end
end
Prob(599:n) = 0;
for a = 600:n
for j = t-s+l:t-l
Prob(a) = Prob(a)
end
Prob(a) = v*Prob(a);
end
beta(599) = 0;
for b = 600:n
beta(b) = beta(b-l) +
end
+ P(a,j+1);
Prob(b);
50


APPENDIX F
powercurvc.ni
function alpha = power_curve(n, thresh)
P(l,l) = 1;
P(1,2:n+3) = 0;
q = 0.3770;
s = 50;
1 = 11;
v = q;
t = round(thresh*s);
if (t-l>=s && s==l+l)
for y = l:n+l
p_sum = 0;
for b = 1:1+1
p_sum = p_sum + P(y,b);
end
P(y+l,l) = (1-v)*p_sum;
for z = l:n+l
if t-l-Kz && z<=t-l
P(y+1,z+1) = v*P(y,z);
elseif l P(y+l,z+l) = (1-v)*P(y,z+1+1) + v*P(y-1,z-1);
end
end
end
elseif (t-l>=s && s>l+l)
for y = l:n+l
p_sum = 0;
for b = 1:1+1
p_sum = p_sum + P(y,b);
end
P(y+l,l) = (l-v)*p_sum;
for z = l:n+l
if l<=z && z P(y+l,z+l) = (1-v)*P(y,z+1 + 1) ;
elseif s-l<=z && z<=t-l-l
P(y+l,z+l) = (1-v)*P(y,z+1+1) + v*P(y,z-s+1+1)
elseif t-l-Kz && z<=t-l
P(y+l,z+l) = v*P(y,z-s+1+1);
end
51


end
end
end
Prob(599:n) = 0;
for a = 600:n
for j = t-s+l:t-l
Prob(a) = Prob(a)
end
Prob(a) = v*Prob(a);
end
alpha(599) = 0;
for b = 600:n
alpha(b) = alpha(b-1)
end
alpha;
P (a,j+1)
Prob(b);
52


APPENDIX G
run curves.m
close all
clear all
n = 1:2000;
alphal(600:2000) = 0;
alpha2(600:2000) = 0;
alpha3(600:2000) = 0;
betal(600:2000) = 0;
beta2(600:2000) = 0;
beta3(600:2000) = 0;
alphal = power_curve(2000, 5);
alpha2 = power_curve(2000, 3.92);
alpha3 = power_curve(2000, 3.5);
betal = false_alarm_curve(2000, 5);
beta2 = false_alarm_curve(2000, 3.92);
beta3 = false_alarm_curve(2000, 3.5);
plot(n,betal,n,beta2,n,beta3)
title('Power Probabilities (Beta)')
xlabel('Number of samples')
ylabel('Probability')
legend('5','3.92', '3.5 ')
figure
plot(n,alphal,n,alpha2,n,alpha3)
title('False Alarm Probabilities (Alpha)')
xlabel('Number of samples')
ylabel('Probability')
legend('5','3.92','3.5')
53


APPENDIX H
color gray.m
function [T_iraage,rec_size] = color_gray(j,i,a,b,T_image,rec_size)
T_image(j,i) = 125;
if (j > 2)
if T_image(j-1, i) == 0
[T_image,rec_size] = color_gray(j-
1,i,a,b,T_image,rec_size);
rec_size = rec_size + 1;
end
end
if (j < a 1)
if T_image(j+1, i) == 0
[T_image,rec_size] =
color_gray(j+1,i,a,b,T_image,rec_size);
rec_size = rec_size + 1;
end
end
if (i > 2)
if T_image(j,i-1) == 0
[T_image,rec_size] = color_gray(j i-
1,a,b,T_image,rec_size);
rec_size = rec_size + 1;
end
end
if (i < b 1)
if T_image(j,i+1) == 0
[T_image,rec_size] =
color_gray(j,i+1,a,b,T_image,rec_size);
rec_size = rec_size + 1;
end
end
54


BIBLIOGRAPHY
Burrell, A. & Papantoni-Kazakos, P. (September 1998). Extended Sequential
Algorithms for Detecting Changes in Acting Stochastic Processes. IEEE
Transactions on Systems, Man, and CyberneticsPart A: Systems and
Humans Vol. 28 (5), 703-710.
Burrell, A. & Papantoni, P. (in press). Robust Sequential Algorithms for the
Detection of Changes in Data Generating Processes.
Danker, A., & Rosenfeld, A. (1981). Blob Detection by Relaxation. IEEE
Transactions on Pattern Analysis and Machine Intelligence Vol. PAMI-3 (1),
79-92.
Emissivity Table, (n.d.). Retrieved from
http://www.thermoworks.com/emissivity_table.html
FL1R i60. (n.d.). Retrieved from
http://support.flir.com/DsDownload/Assets/39901-0101_en_41.pdf
How Thermal Imaging Works, (n.d.). Retrieved from
http://www.morovision.com/how_thermal_imaging_works.htm
Introduction to Infrared Thermometers, (n.d.). Retrieved from
http://www.omega.com/prodinfo/infraredthermometer.html
Kazakos, D., & Papantoni-Kazakos, P. (1990). Detection and Estimation. New
York: Computer Science Press.
Kothari, S., Chaundry, Q., & Wang, M.D. (2009). Automated Cell Counting And
Cluster Segmentation Using Concavity Detection And Ellipse Fitting
Techniques. 1SBI2009, 795-798.
Papantoni-Kazakos, P., & Burrell, A. (2010). Traffic and Network Performance
Monitoring for Effective Quality of Service and Network Management.
P. Bhattarakosol (Ed.), Intelligent Quality of Service Technologies and
Network Management: Models for Enhancing Communication. Hershey, PA:
IGI Global.
55


Spectroscopy, (n.d.). Retrieved from
http://www.baylor.edu/bucas/index.php?id=37025
trp80 (Photographer). (2007). Overhead Crowd [Photograph], retrieved December
14, 2010, from: http://www.flickr.com/photos/trp0/522543823/
Yang, D.B., Gonzales-Banos, H.H, & Guibas, L.J. (in press). Counting People in
Crowds with a Real-Time Network of Simple Image Sensors.
[Purple crowd photograph]. Retrieved April 21, 2011, from:
http://ycorpblog.com/2008/09/15/start-wearing-purple/
56