Citation
Enhancing usability of web sites providing airline flight and fare information

Material Information

Title:
Enhancing usability of web sites providing airline flight and fare information
Creator:
Chariton, Craig Albert
Publication Date:
Language:
English
Physical Description:
ix, 88 leaves : ; 28 cm

Subjects

Subjects / Keywords:
Web sites -- Design ( lcsh )
Airlines -- Rates -- Computer network resources ( lcsh )
Genre:
bibliography ( marcgt )
theses ( marcgt )
non-fiction ( marcgt )

Notes

Bibliography:
Includes bibliographical references (leaves 87-88).
General Note:
Department of Computer Science and Engineering
Statement of Responsibility:
by Craig Albert Chariton.

Record Information

Source Institution:
University of Colorado Denver
Holding Location:
Auraria Library
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
57494910 ( OCLC )
ocm57494910
Classification:
LD1190.E52 2004m C41 ( lcc )

Full Text
ENHANCING USABILITY OF WEB SITES PROVIDING AIRLINE FLIGHT
AND FARE INFORMATION
by
Craig Albert Chariton
B.S., Metropolitan State College, 1987
A thesis submitted to the
University of Colorado at Denver
in partial fulfillment
of the requirements for the degree of
Master of Science
Computer Science
2004
i


2004 by Craig Albert Chariton
All rights reserved.


This thesis for the Master of Science
degree by
Craig Albert Chariton
has been approved
by
Min-Hyung Cnp'\
.nr* r^~ t
i j John Clark
30iM
Date
Ellen Gethner


Chariton, Craig Albert (M.S., Computer Science)
Enhancing Usability of Web Sites Providing Airline Flight and Fare Information
Thesis directed by Associate Professor Min-Hyung Choi
Airline and travel Web sites have demonstrated usability problems with the flight and
fare search mechanisms. These problems appear in the query formulation, result
display and locating optimal fares with fewest queries. This paper describes an
approach to improve the usability of these airline reservation Web sites by employing
proposed design guidelines and advanced search algorithm that exploits the user's
scheduling and budget constraints to determine the optimal search space. The design
guidelines and search process were tested with a prototype against two production
Web sites. Results show that the design guidelines and search algorithm increase user
ability to successfully complete a search and cover the search space to find the
optimal fare. Users demonstrated better comprehension of the search results and
reduced number of queries when searching for a low fare.
This abstract accurately represents the content of the candidate's thesis. I recommend
its publication.
ABSTRACT
Min-Hyung Choi
IV


ACKNOWLEDGEMENT
My thanks to my advisor, Min-Hyung Choi, for his assistance and insight during the
work on my thesis the past several years.


CONTENTS
Figures......................................................................viii
Tables.......................................................................ix
Chapter
1. Introduction............................................................1
2. Usability Problems Related to Design Issues.............................3
2.1 Query Formulation Problems..............................................3
2.1.1 Location Codes..........................................................3
2.1.2 Departure and Return Dates..............................................7
2.1.3 One-Way and Round Trip Searches.........................................7
2.2 Process Flow Issues.....................................................8
2.2.1 Requiring Users to Log On to a Web Site.................................9
2.2.2 Extraneous Location Code Matches........................................9
2.3 Information Display Issues.............................................11
2.3.1 Use of Location Codes in Display Results...............................11
2.3.2 Industry Specific Terms................................................11
2.3.3 Information Overload on Initial Results Page...........................12
2.3.4 Code Share.............................................................12
2.3.5 Information Availability...............................................12
2.3.6 Fare Information.......................................................13
2.3.7 Aircraft Type..........................................................13
3. Usability Problems Associated With Searches............................14
3.1 Differences Between Product Search and Flight Search...................14
3.2 Yield Management.......................................................16
3.3 Business Traveler Characteristics......................................17
3.4 Leisure Traveler Characteristics.......................................17
3.5 Examples of Problems with Search.......................................18
4. Related Research.......................................................20
5. Proposed Design Guidelines.............................................22
6. Proposed Low Fare Search Method........................................25
6.1 New Search Algorithm...................................................25
6.2 Ensuring User's Intent.................................................27
6.3 Cost Savings Calculation...............................................28
VI


7. Prototype Description.................................................30
8. Experiment Description................................................41
8.1 Criteria..............................................................42
8.2 First Experiment Description..........................................43
8.3 Second Experiment Description.........................................44
9. Experiment Results....................................................46
9.1 First Experiment Results..............................................46
9.2 Second Experiment Results.............................................52
10. Conclusions...........................................................56
Appendix
A. First Round Test Questionnaire .........................................58
B. Second Round Test Questionnaire.........................................79
References
87


FIGURES
Figure 1 Query Screen......................
Figure 2 Multiple Airport Selection Page...
Figure 3 Results Display...................
Figure 4 Fare Details Display..............
Figure 5 Passenger Facility Charge Definition
Figure 6 Aircraft Information..............
Figure 7 Other Information Display.........
Figure 8 Query Screen for Advanced Search ..
Figure 9 Savings Parameter Input Box.......
Figure 10 Suggestion Box...................
Figure 11 Display of Suggested Flights.....
30
31
32
33
34
35
36
37
38
39
40


TABLES
Table 1 Successful Completion of a Query..................................46
Table 2 Successful Answering of Basic Information.........................47
Table 3 Ranking of Clarity of Information Display.........................48
Table 4 Ranking of Difficulty of Performing a Task........................49
Table 5 Ranking of Perceived Ease of Use..................................49
Table 6 Ranking of Perceived Ease of Use for Low Fare Searches............50
Table 7 Ranking of Preference When Shopping...............................50
Table 8 Performance for Scenario 1........................................51
Table 9 Performance for Scenario 4........................................51
Table 10 Performance for Scenario 5.......................................52
Table 11 Overall Task Completion Rates....................................53
Table 12 Ranking of Ease of Use When Shopping for Lowest Fare.............54
Table 13 Rankings for Shopping for Lowest Fare with Threshold.............54
Table 14 Ranking of Preference When Shopping..............................55
IX


1. Introduction
The travel and lodging industries are currently undergoing a major transition in their
distribution channels, much like the rest of the world economy. The Internet has
provided increased ability for customers to access information concerning
transportation and lodging directly from the service providers, opening a new outlet
for services never before available. The opportunities are immense due to the cost
savings available to the service providers and the flexibility afforded potential
customers.
Like many other industries, the travel industry, particularly airlines, are investing
millions of dollars and planning to make major portions of their sales via the Internet.
However, airlines and travel-related Web sites are making major mistakes in the Web
sites that they use to promote and sell their services. These mistakes take the form of
usability problems with their flight and fare search functions. Without adequate
usability for these functions, the Web sites that airlines and other travel distribution
services are developing to sell airline seats to customers will not provide the success
hoped for.
Usability has many different definitions. A good summary of the characteristics of
usability is provided by Jakob Nielsen [11]:
Ease of learning
Efficiency of use
Memorability
Error frequency and severity
Subjective satisfaction
Usability is a critical factor in the success of a Web site. Users of an e-commerce site
are there by choice in search of a product or service. They can leave an e-commerce
site at any time and find another e-commerce site that will sell them the same product
or a similar product at about the same or a lower cost. Therefore, a business should
not put any obstacles in the way of the customer to drive them away from the Web
site.
This paper will address the issue of airline and travel-related Web site usability
enhancement. First we will examine usability problems related to design issues. The
areas of query formulation, process flow and information display will be examined in
1


detail as to how each contributes to overall usability problems.
The next section will address usability problems associated with searches. An
introduction to yield management will be presented followed by examination of
business vs. leisure travelers and their characteristics. The search needs of each type
of traveler will be explored.
Related research will be presented regarding both design guidelines and low fare
searches. An investigation of the current approaches to low fare searches by several
production Web sites will be made along with examination of the strengths and
weaknesses.
A set of proposed design guidelines will be presented in depth. The specific usability
problems will be presented along with the proposed design guideline aimed at
reducing or eliminating the problem.
A new low fare search method will be developed. A search algorithm and user
interface will be explored.
The description of a prototype will be presented in Section 7. Description of two
experiments will be given in Section 8, followed by the results of each experiment in
Section 9. Finally we will develop conclusions based upon the research and discuss
potential directions for future research and applications of the findings.
2


2. Usability Problems Related to Design Issues
The usability problems associated with the design of Web sites that provide airline
flight and fare information manifest themselves in several different ways. First the
user may not be able to properly formulate the query to obtain the proper listing of
flights that the user wishes to see. Second the user may not be able to obtain the
desired information due to the design of the process flow the user expects when
shopping via the Internet. Third the user may not be able to locate or interpret the
information in the resulting display, thus denying the information to the user and
interfering with the purchase decision.
2.1 Query Formulation Problems
Query formulation is the user's initial contact point with the e-commerce site. The
user must be able to interpret how to use the search interface, input the proper query
parameters and successfully submit the query to the e-commerce system for
processing. If a user is unable to accomplish these tasks, they will stop working with
the Web site and move to another Web site or source that can provide the information
they desire.
Web sites that provide flight and fare searches face unique usability challenges in the
area of query formulation. This is because there is not a limited number of products
to sell. Instead the product consists of numerous combinations of flights with many
characteristics assigned to each flight. This makes the query much more complex.
2.1.1 Location Codes
The first query formulation issue involves location codes. Location codes are at the
very heart of any airline computerized reservation system (CRS). Location codes are
three letter designators for airports and cities around the world and represent these
airports and cities in the CRS. There are over 10,000 location codes designated by
the International Air Transport Association (LATA). It is via location codes that
searches for flights are conducted and bookings are made. To the people that work in
the travel industry location codes are a way of life. To the rest of the world they are
just three characters that they see on their tickets and baggage tags. However, in
order for the rest of the world to communicate properly with a CRS and to allow them
3


to use a Web site to obtain airline schedules and fare information, the world must use
location codes. The question is then, How do we effectively make the translation
between the everyday customer searching for a flight from City A to City B when
they dont understand location codes? At first glance this seems like an easy
question to solve, but underneath it is a very complex issue and one that is not dealt
with effectively by current airline and travel Web sites.
To demonstrate the issue, examine the following list of location codes and the
corresponding airport and cities that they refer to:
DEN Denver, Colorado
ORD OHare International Airport, Chicago, Illinois
LAX Los Angeles International Airport, Los Angeles, California
ABJ Abidjan, Cote D Ivoire
CDG Charles De Gaulle International Airport, Paris, France
FSD Sioux Falls, South Dakota
FAT Fresno Air Terminal, Fresno, California
MCI Kansas City, Missouri
GIG Rio De Janeiro, Brazil
GRU Sao Paulo, Brazil
Some of these the general public can decipher. Others they could not begin to
decipher. A designer cannot expect the customer to know the applicable location
code to enter in a search for flight availability. Therefore it must be made as simple
as possible for the customer to type in the locations in their proposed itinerary and
receive the desired information without laying a roadblock in their way with the
location code issue.
This should be a simple matter of translation of the city entered by the user into an
applicable location code. For most cases this will work. However, the following
usability problems from Web sites show that this is not always understood by Web
site designers.
Some locations have a large population base and can support multiple airports.
Examples of such locations are New York City, Chicago, Dallas Fort Worth,
Miami, Los Angeles, London and Houston. Because each of these cities has multiple
airports, they have multiple location codes associated with them. For example, flights
for New York City can be located using the following location codes: NYC, JFK
(John F. Kennedy International Airport), LGA (LaGuardia), and EWR (Newark).
If an entry is made in a CRS for flights associated with NYC, the CRS will display
4


flights associated with all of the airports (JFK, LGA and EWR). If an entry is made
using one specific airport code, such as EWR, then flights associated with EWR are
displayed first, normally followed by flights to other area airports.
A CRS does not experience confusion with these entries. It is programmed to
respond in a certain way when certain location codes are entered. However, this is
not true for travel Web sites. An example is the American Airlines Web site. On
September 6, 2000 flight availability from Dallas to Denver was requested via the
American Airlines Web site. The response was a page with two columns. The first
column was for the departure city, and the second for the arrival city. The first
column contained a column of five radio buttons with the following departure cities
beside them:
Dallas Ft. Worth, TX
Dallas Love Field, TX
Dallas, TX
Dallas/Fort Worth, TX
Dallas/Ft. Worth, TX
The second column consisted of a column of one radio button with the city of Denver
listed beside it. The page asked the user to click the radio button by the appropriate
city they wished to fly to and from. This can cause confusion on the part of the user.
First the user is wondering why the Web site is displaying a page of cities when they
already informed the Web site the cities they wished to travel between. The second
area of confusion concerns the different offerings for Dallas. Does this display mean
that Dallas has five airports from which to choose? In actuality Dallas is served with
scheduled air service by two airports Dallas / Ft. Worth International Airport and
Dallas Love Field. In this case which one should the customer choose? Will it make
a difference in the flights displayed to the user? These are all valid questions.
It should be noted that American Airlines operates from both airports, which is
unusual. Dallas / Ft. Worth is a major hub for American, and they also serve Dallas
Love Field to compete with Southwest Airlines. While this means that American can
offer more flight options to the traveler, the Web site should not be confusing the
customer with an offering of what appears to be five different airports. The Web site
should narrow down the choice to only the two available airports.
One aspect to consider when dealing with this problem is what information does the
customer want from the Web site and what is their shopping goal. Suppose the
customer is a father planning a trip to Disneyland in Anaheim, California. His goal is
to find the cheapest fares. The closest airport to Disneyland is John Wayne (Orange
5


County) Airport. However, the traveler also has the options of flying into Los
Angeles International, Long Beach, Ontario or Burbank. While these are farther
away from Disneyland, they may have flights that are cheaper than those flying into
Orange County. The father may not mind flying to one of the other airports because
he can rent a car and drive to Disneyland, possibly stopping at other attractions along
the way. Since the family will be on vacation, they have time to spend driving and
visiting other places.
Now consider a business man who wishes to specifically fly into Orange County
because it is closer to his sales client. He doesnt wish to spend the extra money on a
rental car to drive from Los Angeles International nor does he have time to spare. It
is worth the extra money to fly into Orange County.
When either of these two customers enters Orange County in the Destination box on
the flight search, two entirely different displays are desired. The father wishes to see
the cheapest fares with the option of other airports in the vicinity that may have lower
fares. The business man does not wish to be bothered with a display of fares into the
other airports. His only concern is the available flights for Orange County.
Therefore, the Web site must obtain enough information from the client to determine
what the customers goal is and tailor the display accordingly. A travel agent can do
this for the customer, and the Web site must duplicate this function in order to be
successful and provide the desired amount of usability for the customer.
A problem that airline and travel-related Web sites encounter in searches that most
other Web sites do not encounter is that of identical product names. In the case of
flight searches, the problem occurs when cities have identical names. This does not
occur very often, or at least it should not. While there are many cities around the
world with identical names, few have scheduled air service. This simplifies the
problem greatly as far as the number of choices to present to the client is concerned.
Examples of cities with identical names and scheduled air service are as follows:
San Jose, California and San Jose, Costa Rica
Athens, Greece and Athens, Georgia
Portland, Maine and Portland, Oregon
When a passenger does not know a location code but knows a city name that may be
identical to another city name, the Web site must present the customer with a choice
menu. This choice menu needs to display the available choices from which a
6


customer can choose with enough clarity so that the customer can identify the city
that he is searching for. While this does create another screen that the customer must
read and process, it is a necessary evil to provide the customer with the information
he desires.
2.1.2 Departure and Return Dates
Another important piece of data that must be input into the search is that of dates.
Everything in a CRS is dependent upon dates, from the flight schedule to the fares.
An airlines flight schedule can vary daily, with a different schedule available for
different days of the week. In addition, the schedules vary throughout the year
depending upon the travel season. All of the fares, as stated previously, are based
upon restrictions of advance purchase and overnight stays. Travelers have learned
that they can receive lower fares by booking flights 21 days in advance and staying
overnight on a Saturday. Therefore the flight dates are critical to the type of fare that
a traveler may be charged.
There are various ways of obtaining this information from a customer. One popular
method in airline and travel Web sites is the use of pull-down menus. This forces the
customer to select a specific month and day from the list available. This is not a poor
way to solicit this type of input. There are only twelve months to select from and
only 31 days. Many of these menus will not allow a customer to enter an invalid date
(such as February 31), so error rates should be low with this type of mechanism.
However, there is a flaw in that the user is forced to either have a known date or a
calendar available for their use in order to enter this date or dates. Another solution
would be to display a calendar for the user to work with. Customers are familiar with
the analogy of a calendar to select dates. Therefore, using this frame of reference
only seems natural. They can click on the date on the calendar in order to select the
dates for the search. One or two months can be displayed, with arrows permitting the
user to scroll through the months of the year to locate the date they wish.
This is what Travelocity has placed on their page for their Best Fare Finder tool.
Customers in usability tests and focus groups appeared to like an interface that
included a calendar for selecting dates. One problem was that of using different
screens to select departure and return dates. This should be eliminated by allowing
the customer to specify the dates on one screen using the calendar display.
2.1.3 One-Way and Round Trip Searches
One of the complaints that surfaced in the Wichita State study [15] was the inability
7


to search for a fare either by one-way or round trip. In particular Vanguard Airlines
only displayed one-way fares, so the users had to work harder to obtain round-trip
information. We can take this complaint one step further by examining the different
types of scenarios that travelers may be searching for.
Most travelers are searching for a round-trip fare because they need to travel from
City A to City B on a specific date and then return to City A on another date. This
scenario fits most leisure and business travelers. Most of the sites provide this type
of search functionality and provide a round trip fare to the user without forcing the
user to calculate a round trip fare using the one-way fares.
Another scenario is where the traveler only wants to fly one-way from City A to City
B. This may not be as unusual as one may first think. If a traveler knows when the
want to depart from City A but not when they want to return, this is the type of search
that they want to perform. If a person is moving from City A to City B, then they are
not interested in a round-trip fare search. One-way flights fares are provided in the
CRSs, so the information is available to the Web site to provide to the traveler.
A third scenario is what is called a round robin. Suppose that a traveler wishes to
fly from City A to City B on a certain date, from City B to City C on another date,
and finally from City C to City A on yet another date. This itinerary is a round-robin
and is made up of more than two segments. This can be desired by both leisure and
business travelers. This information can be located using three separate searches
using a one-way search, but it should also be possible to collect this information on
one page and allow the search function to do the work for the traveler. A good
example of such a search function is found on the site of ITA Software, which is
developing the interface for the Orbitz travel site. This will allow the traveler to
input the itinerary as desired and then search all of the CRSs to locate the lowest
fares.
2.2 Process Flow Issues
Once a user has successfully interpreted the query interface and submitted the query,
a shopper expects to have the desired information returned containing airline flight
schedules and fares. However, this is not necessarily how the Web sites interact with
the user. When the user encounters a work flow other than what is expected, it can
cause a user to become frustrated and leave the Web site. Thus process flow issues
are very important in the overall Web site design and usability concerns.
8


2.2.1 Requiring Users to Log On to a Web Site
A feature that frustrates many customers of airline Web sites is the requirement to
log on to the Web site in order to obtain certain information. The Wichita State
University study revealed that the American, United, US Airways and TWA Web
sites required the user to logon to the Web site in order to obtain fare information.
This procedure required the customer to enter personal information including their
name, address, telephone number and e-mail address. TWA offered a guest option
that did not require this information, but instead allowed the customer to act as a
guest and log on without actually entering this information.
The responses of the test subjects are typical to any Web site that requires a logon
procedure. Many of the users objected to logging on, and they didnt understand why
they had to logon just to view fares, a most basic request. Even the appearance of the
logon screen frustrated some users in the WSU study. The most interesting response
came from one user that hit the Back button whenever they encountered a logon page
at a Web site. These responses should not be surprising. The users wish to keep their
anonymity while shopping. There is no reason for requiring a user to logon just to
view flight schedules and fares. There may be requests from the marketing
department to do so because they wish to know who is using the Web site, but to
many users it is an invasion of privacy.
2.2.2 Extraneous Location Code Matches
On September 5, 2000, the United Airlines Web site was used to determine the flights
available between Denver, Colorado and Dallas, Texas. After entering the name
Denver in the From box and the name Dallas in the To box and selecting the
dates of travel, the Check button was clicked. The Web site responded with page
titled Uncertain City/Airport Name that read:
Uncertain Airport/City Name: More than one city was found
matching your departure entry of Denver in our database. Please
select an airport in or nearby the city of your choice from the
following list. Otherwise go back and specify a different entry.
This page then displayed 45 airport locations broken down by their location in or
proximity to certain states. The first location was for Denver International Airport for
Colorado. The next location in the list was Lancaster in Lancaster, Pennsylvania.
This was followed by locations for Iowa, Indiana, Missouri, Arkansas, Alabama,
Florida, Georgia, Idaho, Illinois, North Carolina, New York, Ohio, Oklahoma,
9


Kentucky, South Carolina, Tennessee, and West Virginia. It is unexplained why the
name was uncertain to the CRS, nor why 45 locations were returned. This can only
add to the users confusion and frustration when bombarded with so much irrelevant
information when they are expecting a display of available flights.
This experiment was repeated, this time using Dallas in the From box and Denver in
the To box. The same type of response was received. This time Dallas-Ft. Worth
International and Dallas Love Field were listed at the top of the list followed by a
myriad of states and airport locations. When the appropriate city codes of DEN and
DFW were entered in the To and From boxes, the CRS had no difficulty in returning
a flight display.
For another example, during the summer of 1998 the American Airlines Web site was
recognized to return airports that do not have scheduled air service. Flights were
requested with an origin of Denver, Colorado. The search function returned a screen
asking for the desired airport in Denver. Two airports were listed: Denver
International Airport and Buckley Air National Guard Base. Buckley Air National
Guard Base has a location code but no scheduled air service. Therefore, this would
be confusing to a customer with no knowledge of the airports in the Denver area that
had scheduled air service. To some Buckley would be closer to their homes.
However, one can imagine their dismay if they requested flights out of Buckley only
to find that there are no scheduled flights out of that location.
This is not an isolated incident. When searching for flights from Denver to
Cambridge on the Northwest Airlines Web site on September 7, 2000, three relevant
choices for the city of Cambridge were presented. The choices were:
Cambridge Bay, Victoria Island, NWT, Canada (YCB)
Cambridge, England (CBG-Cambridge)
Cambridge, Massachusetts (JHY-Hyatt Regency-Heliport)
If a customer entered JHY for Cambridge, Massachusetts, they would soon learn that
there are no scheduled services to this location. When this experiment was duplicated
on United Airlines site, it did not show the Cambridge, Massachusetts location but it
did show Cambridge, Minnesota, another location without scheduled air service. A
search on Yahoo!s site showed Cambridge, Maryland, which also does not have
scheduled air service. While these locations do have location codes issued by either
the Federal Aviation Administration or IATA, these Web sites should not be
displaying them to the public because there is not scheduled air service for them to
10


shop for. In essence, why should the Web site offer the public destinations for which
there are no scheduled flights? This is frustrating to the customer and interrupts the
process flow.
2.3 Information Display Issues
Information display issues affect the shopper's ability to locate or interpret
information on the display of results from a query. Since the flight information must
be described entirely by text, and it is a complex product, clear and concise
communication of the information desired by the shopper is essential to usability.
2.3.1 Use of Location Codes in Display Results
Some sites encode the airports in the results display using their location codes. Many
sites do not translate this data on the display but rather display it directly to the user
or perhaps displaying the airport name along with the location code. Even with the
airport name displayed, the location code can still cause confusion [15].
2.3.2 Industry Specific Terms
Many sites use industry specific terms and abbreviations in their query and results
display interfaces. Few shoppers are familiar with these travel industry terms. When
the terms are used, there is seldom a definition for the user to refer to for an
understanding of what the term means.
For instance, many users confuse the terms "cabin" and "class". These are industry
specific terms and have different meanings in the industry whereas they may be
synonymous to the shopper. Shoppers believe that First, Business and Economy or
Coach are classes, while in the travel industry these refer to cabins of an aircraft. A
class to the industry refers to the booking class and stands for the fare and restrictions
that a ticket was issued with. Often the booking class will appear on a Web page, and
the user has no understanding of what it means. The cabin code may also appear on a
page with F or J meaning First, C meaning business and Y meaning coach or
economy. The use of these codes adds confusion to the display.
11


2.3.3 Information Overload on Initial Results Page
Many results displays attempt to place too much information on the initial page.
They try to provide additional information regarding who operates the flight, fares,
etc., usually with small fonts that may be difficult for vision impaired shoppers to see.
Other times they provide extraneous information that is of secondary importance to a
shopper. This hinders comprehension by the user and can result in information
overload.
2.3.4 Code Share
A new area of confusion concerns code share flights. In a code share relationship one
airline, called the marketing carrier, sells seats on the flights of another airline, called
the operating carrier. These relationships provide more traffic and revenues to both
airlines. These relationships are very important in today's economic environment,
and they will only become more numerous as airlines consolidate and form alliances.
When these flights are displayed they will appear twice in a display once for the
marketing carrier and once for the operating carrier. Usually the marketing flights
will have a note on the initial page that it is operated by a certain operating airline.
However, this may cause confusion to the traveling public as they are not readily
informed of the code share practice. In addition, they usually do not have access to
information concerning the operating carrier in case they are unfamiliar with it.
2.3.5 Information Availability
One of the most frustrating aspects for users of the search results display is that not all
of the information that a shopper may wish to have is available through the Web site
in an understandable format. For instance, a travel agent can translate the fare data
for a client, or they can tell a client how the restrictions will affect them if they
choose to change their itinerary after purchase. Many Web sites take a stock fare
display from the reservation system, written in travel industry terms and legalese, and
then display this information to the user. Few users are able to decipher this
information and come to an understanding of the restrictions. Many times the sites
omit essential information from a display that a travel agent would be able to provide
because it is provided via other means by the reservation system or through other
sources available to the travel agent.
12


2.3.6 Fare Information
On many Web sites it is difficult to determine the breakdown of fare information for a
specific itinerary. Some sites provide the base fare without taxes, while others may
display the total fare with taxes and other charges. Usually a user must find a link
and proceed to another Web page to find a breakdown of the charges, if one is even
provided. Some sites will have this information scattered over two or more pages
making it difficult for the shopper to locate the information.
2.3.7 Aircraft Type
A final display issue concerns the aircraft type. Many different types of aircraft are in
use today in scheduled air service. Regional airlines that operate as code share
carriers for the major airlines use turboprop aircraft that seat less than 50 persons.
While many people do not talk about it, they are concerned about flying on smaller
aircraft, especially propeller-driven aircraft. In our research 41% of the persons
tested in the first round of tests indicated that they were concerned about flying on a
propeller-driven aircraft. Yet many Web sites do not provide any information
concerning the aircraft other than the aircraft type. Unless one knows a lot about
aircraft types there is no way that a user, who has a concern about flying on a
propeller-driven aircraft, can learn any information about the aircraft that is scheduled
for the flight.
13


3. Usability Problems Associated With Searches
Today's user is concerned with finding the lowest fare available. Many shoppers are
now turning to the Web sites to obtain this information. In fact some fares are only
available via the Web. While a travel agent is familiar with how to find the lowest
fares, the shopper is not. This lack of knowledge on the part of the user creates a
need for the Web sites to provide better search interfaces and tools for the user to find
the desired information. However, the task is complex and not easily accomplished.
3.1 Differences Between Product Search
and Flight Search
In the studies of e-commerce Web site usability, the current literature categorizes all
Web sites that sell products and services into one overall category. No distinction is
made between the types of sales that each Web site engages in. One set of standards
is therefore promulgated to apply to each type of Web site. However, this is
incorrect. There are actually distinct types of e-commerce Web sites and this
therefore affects the design of each e-commerce site and its usability. According to
Palmer and Griffith an organizations Web-based marketing activities and Web site
design are determined by the interaction among the companys market offering,
marketing activities and technical characteristics [12]. Airline and travel related Web
sites fall into a different category of Web site design as compared to a seller of
products, such as Amazon.com. We will examine the differences in each type of e-
commerce site in turn.
It has been suggested by Palmer and Griffith that market scenarios differ according to
market environment, and that these scenarios vary in term of user profiles, site
interactivity, and information content. The market situations provide different
opportunities for product and service branding, interactivity and information
manipulation in the site design. Thus an organizations product offering will have an
influence on its ability to use the Internet for e-commerce. This in turn will affect the
site design and usability.
Palmer and Griffith point out that information intensity affects an organizations
ability to utilize technology. Information intensity describes the amount of
information that a consumer requires to utilize a product or service. They use as their
examples the differences between Aetna Insurance and Allstate Insurance as opposed
14


to hard good manufacturers Goodyear and Exxon. The insurance companies have a
core product that is mostly information based and not tangible. They can therefore be
placed in direct contact with the customer and bypass the middleperson. Goodyear
and Exxon have tangible products for which the Internet is not conducive to direct
sales.
Palmer and Griffith argue that the dimensions of the site design include both
marketing and technical concerns and are driven by the core offering of an
organization. Therefore a site that sells books, such as Amazon, will have a different
Web site design when compared to American Airlines. Why would they be different?
ie the characteristics of each business:
Amazon American Airlines
Product: Books Transportation
Product Type: Goods Services
Information Intensity: Low High
Product Searches: Simple Complex
Product Description: Simple Complex
Pricing: Simple Complex
Consider a transaction for Amazon. A customer proceeds to Amazons Web site in
search of a book about dolphins. The customer is required to enter one or more
keywords for a search through the database. These keywords are natural for the
customer in that they are not special codes. The search proceeds for the customer and
a list of books is returned for a keyword dolphin. The title, author, description and
price of each book is displayed for the user. There may also be a display showing
other topics that may be of interest to the customer or the comments others have made
about the book that would influence the customers decision whether or not to
purchase the book. A lot of information is returned, but the customer has had to
process this information before when purchasing a book from a book store, so the
experience is not much different.
Now consider a transaction for American Airlines. A customer wants to check flights
and fares between Denver and Dallas Ft. Worth. The customer in the past has had a
travel agent or a reservationist perform this function for him. Now he is relying upon
a completely unfamiliar search and display process that he must interpret himself. To
begin the search is much different. The passenger must know the origin city,
destination, cabin of service, departure date and desired departure time. Because
Dallas Ft. Worth is served by two airports, the customer must know the airports
which he desires to use. If he is searching for a round trip flight, he will also need to
know the return date. The computer system he will be searching does not understand
15


the cities Denver and Dallas Ft. Worth, so a keyword search is impossible. Rather
the site must interpret the desired cities and translate these into location codes. This
sounds like a simple task, but more often than not this selection of location codes
leaves users baffled. When a customer has successfully entered the appropriate
search, he must then decipher the vast amount of information provided. The user is
provided with multiple flights, possibly 80 to 100 in various combinations, that will
transport him from Denver to Dallas Ft. Worth on the day desired. The flight
number, departure time, departure and arrival cities and, hopefully, fare is displayed.
There may be non-stop flights or flights involving connections. There may be total
elapsed travel time information displayed. The aircraft type may also be shown.
There may be extraneous information, such as a fare basis code, and the use of terms
such as leg, segment, open-jaw and round-robin. The display may also use the
location code rather than the actual names of the cities to display the departure and
arrival airports. Add to this the fact that a page may display the exact same flight
with two different fares caused by the variable pricing scheme used by airlines. A
customer is not familiar with this type of shopping in their normal life, unless they
have training as a reservationist or as a travel agent. Therefore the user experiences
information overload. The result is that a user goes to another site to search for this
information or, in their opinion, the better solution of contacting the airline directly
by telephone or using a travel agent.
The airline scenario does not occur with the Amazon scenario. This is the result of
the differences between their product types and other marketing environment factors.
The sites are different, both in their design, searches and usability. It is this
difference that causes the usability problems for airline flight and fare searches via
the Internet.
3.2 Yield Management
Unfortunately for the shopper, finding the lowest fare is complicated due to the use
by the airlines of yield management [4]. Yield management is the use of variable
pricing in order to extract the maximum amount of revenue from a seat on a flight.
The goal is to charge a higher price for a seat to those who can afford it and need the
seat on that specific date and time, specifically business travelers. By varying the
rules and restrictions for a specific fare, the airlines are able to segment the market
and thereby charge business travelers more for a seat than a vacation traveler. The
rules and restrictions that may apply to a fare include advance purchase requirements,
refund charges, charges to change a flight, and Saturday night stay requirements. In
addition, the number of seats available at a specific fare are controlled for each flight
and flight date. In this manner airlines attempt to maximize the amount of revenue
16


that they generate for each flight.
For example, by requiring a 21 day advance purchase and a Saturday overnight stay,
an airline will be able to sell services to a family of four traveling on vacation at a low
fare. This is because they know in advance when they will travel, and they will
already plan on staying over a Saturday night. On the other hand, a business person
may not know that they are traveling on business until several days prior to the trip.
Since they will be traveling during the week and can probably only do business
during weekdays, they will not want to stay over a Saturday night in order to obtain a
cheaper fare. Therefore, the airline can charge the business person more than the
vacationing family. This is why a passenger may be traveling on an airplane for
$970.00 round trip and the person sitting next to them paid $249.00 round trip.
However, to make this system function it requires training on the part of the sales
personnel to decipher the fares and restrictions and to communicate this information
to the customer.
3.3 Business Traveler Characteristics
When a business traveler searches for a fare, they generally have a specific date and
time when they need to travel for business purposes. Therefore, the dates and times
of travel will be the overriding consideration in the purchase decision, with cost being
a lower weighted factor. Time spent traveling also affects productivity for these
shoppers, so the number of stops and aircraft changes in route also affects their
choices of flights.
This also applies to customers that wish to fly in First or Business cabins. Some of
these passengers are business travelers who require additional room or comfort to
work or rest in. Others are those who are more wealthy who desire a higher standard
of care and comfort while traveling.
3.4 Leisure Traveler Characteristics
When a leisure traveler searches for a fare, they generally have a general date or dates
of travel in mind, but the prime search criteria is price. It is known that 96% of all
Priceline.com shoppers are leisure travelers, and 80% will change their travel dates in
order to obtain a lower fare [6], Since price is a higher priority for the search than
travel dates and times, the leisure traveler should be more flexible to take advantage
of the complex fare structures in order to find the lower fares. Travel time will not be
as important a factor as this traveler does not necessarily have to be at the destination
17


by a certain time, nor is productivity an issue.
3.5 Examples of Problems with Search
The main search problem confronting shoppers looking for the lowest fare is how to
use the Web site to find the lowest fare. Most Web sites only permit the user to
search a specific departure and return date with one search query. Therefore, if a
shopper wishes to search for the lowest fare within a one week period, the shopper
must make repeated queries using different date combinations. This is time
consuming for the shopper, and a lot of work that many shoppers do not wish to
perform. There is also the possibility of error due to omitting a date search
combination. The shopper must also track the lowest fares found and compare them
across all of the searches. This is also complicated by the fact that the shopper does
not know the fare structure and where to look for the lowest fares. A shopper may
not consider departing on a Tuesday or Wednesday and returning on the following
Tuesday or Wednesday, but this may yield the best results. Therefore the shopper's
ignorance of the industry works to their disadvantage.
The search problem is complicated when budgetary constraints are combined with
scheduling flexibility. The U.S. economic downturn that began in 2001 and
accelerated after September 11, 2001 have changed the ways in which business
people travel. More businesses are not as willing to pay high prices for air fares.
Instead they are seeking to use the leisure fares that are significantly lower than the
normal fares paid by business travelers [2], In order to use the leisure fares,
businesses are having their employees stay through the weekend at a destination in
order to meet the travel restrictions for the lower fares. Even though the company
may pay more for additional hotel and food expenses, the savings in air fare over a
regular business fare may more than justify the effort. However, current Web
products do not allow this type of search to be done via automation. Rather the user
has to manually search for different fares for different dates and then compute the
cost of staying an extra night or nights in a hotel to determine if another flight
combination is more economical than a regular business fare.
Another problem occurs when analyzing a display of flights for a particular day. The
lowest fare may be displayed at the top of the screen. However, many times the
lower fares involve changing aircraft at an intermediate airport or involve stops along
the route. To some business travelers a non-stop flight is more important due to the
time lost traveling. These user's need to know if taking a flight that is not non-stop is
truly worth the savings in air fare. Once again, however, no Web sites currently
18


allow a user to perform such a search. Any calculations must be performed manually
by the user to determine if a lower fare routing is truly more economical.
If a shopper attempts to use current Web sites with their available search tools to
perform these types of searches the task becomes labor intensive. If a shopper has
scheduling flexibility, some sites offer a low fare search tool that takes the user to a
calendar of dates when the lowest fares are available. This is good for leisure
shoppers, but does not meet the needs of shoppers who must travel around specific
dates. These shoppers must use the regular search methods and search each possible
combination of dates in the search space to determine the lowest fares. If budget
constraints are involved, the process becomes even more labor intensive for the
shopper as calculations must be manually performed to determine if a certain flight
meets the budget constraints. Thus there is no guarantee that the shopper covers the
entire possible combinations or determines the best flight combination to meet the
budget requirements.
19


4. Related Research
Numerous guidelines and related articles have been published to increase e-commerce
Web site usability. IBM has produced the IBM Ease of Use: Web Design Guidelines
[9] with a special section dedicated solely to e-commerce. Other resources for
general Web site usability guidelines include Ameritech guidelines [5] and Rohn [14].
These guidelines focus on the areas of customer relations, searching for simple
products, displaying products and other items associated with the sale of tangible
goods. Many of these principles are directly applicable to airline and travel agency
Web sites. However, they do not address the specific problems shown to be prevalent
in Web sites that provide search functions for airline flight and fare information.
Several approaches to Web design have been taken in order to address the problem of
low fare searches. Travelocity and Expedia use the same type of approach to the
problem [3]. After an initial query is made, a search page on the site shows the user
the prices for several airlines based upon an examination of the fare rules. A user
may select an airline and a fare, thus displaying a calendar with shaded dates
indicating when the fare is available. The user can then click on a departure date on
the calendar, then a new page is displayed showing a calendar with the return dates
that are available to provide the quoted fare. These dates are usually twenty-one days
out from the date the user enters the initial search as the lowest fares are usually at
least twenty-one days into the future. This approach uses four screens through which
the user must input their information, requiring additional exchanges between a
browser and the server. If the user wants to check other dates or airlines, a separate
search is required.
Another approach is that of SmartClient [13]. SmartClient views the search process
as a conversation where criteria and proposals are exchanged between the shopper
and seller. The user formulates needs and optimization criteria to an applet, which in
turn queries a database and returns available product offerings. The user can then
view a representation of the products in the search space and adjust criteria and
preferences depending upon the attributes of the products displayed. This approach
uses partial constraint satisfaction techniques instead of weights to allow the user to
reduce the search space. This approach does allow a user to see the initial search
space and then interact with the results to reduce the product options to one that the
user may purchase. The drawback of this approach is that it requires the user to
interact significantly with the display, through the use of graphs, to input the user's
20


requirements. Users may find the interface too complicated to use or too
intimidating.
A new approach that has been made possible through new search methodologies is
that of ITA Software. The demonstrator interface for ITA Software's new search
method allows a user to search around the intended date(s) of travel for the lowest
fare. However, it forces a user to enter whether the user can travel one day before or
after the intended dates, thus severely limiting the search. The positive aspect of this
approach is that it does perform a search across the flight dates specified and provides
the ability to locate the lowest fare among several airlines and travel dates. However,
budget constraints are not addressed.
This idea was expanded and appeared on the Web site for Orbitz in the spring of
2003. This allows a user to choose a broader search up to three days prior to or after
the specified dates. This gives the user greater control over the search and increases
the search span. However, the user could still miss the best flight combination by
limiting the search span due to a lack of knowledge of the fare structure. The Orbitz
site also does not address budget constraints.
The amount of research performed into general Web site usability has been extensive,
including much company confidential research into travel Web site usability. There
have been numerous approaches to solve the various usability problems previously
mentioned. However, the problems still persist with Web sites throughout the travel
industry.
21


5. Proposed Design Guidelines
To resolve some of these usability problems, we developed new design guidelines
targeted at Web sites providing airline flight and fare search capabilities. These
guidelines are built upon the foundation of how a typical consumer views the flight
search process, how they view the information display, and the information they need
to make an informed purchase decision.
A. Do not interrupt the shopper's normal process flow.
Users do not want to have their normal shopping flow and thought process
interrupted. For example, some sites have required users to logon to the system in
order to obtain flight and fare information. This may be desired by the marketing
department to determine the shopping habits of the site's users, but it is annoying to
the users as reported in previous studies [15]. Provide an uninterrupted process flow
much like that provided by a travel agent or reservation agent.
B. Do not use location codes in the query or display processes.
Users have difficulty understanding the use of location codes. Insulate them from
location codes by translating location codes into airport names in displays, and
provide the capability to translate city and airport names into location codes.
C. For cities served by multiple airports, provide only valid airports and only one
selection per airport.
Do not provide airports that do not have scheduled air service in selection lists
displayed to the user. Only provide one selection choice for each available airport.
D. Assume the user knows which airport they wish to depart from except in cases of
multiple airports.
If a user provides an exact match for a city name and only one airport serves the city,
then assume the user intends to use that airport for their search. Do not provide
extraneous airports or other information that may confuse or frustrate the user. Only
22


provide additional airport choices if multiple airports serve a city, such as in the case
of Los Angeles.
E. Do not use industry specific terms unless specifically defined.
Many users do not understand the terms used by the travel industry. Terms such as
airport code, city code, open-jaw, leg, segment, fare basis code, etc., should not be
used in the search and display process unless properly defined. Such definitions can
be provided via hyperlinks to short definition pages or other processes.
F. Provide convenient mechanisms for the user to input date information on the
query interface.
The manner of entering travel dates in the query should be smooth and intuitive to the
user. Options include, but are not limited to, calendars, text fields and selection lists.
An area of consideration should be how the user is able to determine which days of
the week a travel date falls on and the relation of the dates to days of the week and the
month.
G. Provide the same information available through a travel agent.
Compare the information that is available through a travel agent with that available
through the Web site. Provide the same level of detailed information through the site
that a travel agent can provide. This includes basic and detailed information and
definitions of terms. Translate the information into terms that a user can understand.
Do not overwhelm the user with data by transmitting the screens provided to travel
agents.
H. Display only the basic information on the initial results display interface.
The basic information required for a user to decide upon the suitability of a flight
combination is as follows: origin airport, destination airport, dates of departure and
arrival, times of departure and arrival, operating airline, flight number(s), fare, aircraft
type and cabin. From this basic level of information a user can decide if further
information regarding one of the areas is desired. If so, hyperlinks or other means
can be used to provide more in-depth information on a subject, such as fare details
and aircraft information.
23


I. Provide access to information for the user in the near vicinity of a topic.
A user may require information regarding a specific topic. The information provided
should be available in the vicinity of the topic in question. For example, a user may
have a question about a particular airline. Access to information could be provided
via a link through an airline logo or text link to another window or via a pop-up
window.
J. Provide detailed fare information as secondary information.
The total fare should be displayed on the primary flight and fare information display.
From this page a link or other device should be provided to allow the user to "drill
down" into the more detailed fare information. This information should include base
fare, a detailed listing of taxes, and the total air fare. A detailed yet simple
explanation of the restrictions for the fare should be included at this level. A further,
more technical explanation can be provided, but it should be at a level below this
simple description.
K. Fully identify code share flights and their operators to the user.
To avoid user confusion, fully identify the operating carrier to the user as well as the
marketing carrier. Include a description of what a code share is.
L. Provide a description of the aircraft as secondary information.
Provide the user with the ability to identify the type of aircraft they are scheduled to
fly on. This description could include a photograph of the aircraft type, the original
manufacturer of the aircraft, number of engines and engine type (jet, turboprop,
piston), and cabin types available. Additional information can be provided, but it
should not be emphasized. The goal should be to adequately identify an aircraft type
for those shoppers who have concerns about the aircraft they fly on.
24


6. Proposed Low Fare Search Method
The other category of usability problems concerns the ability to search for low fares.
As shown previously, these users must make many searches to properly search
various "what if" scenarios in order to perform a comprehensive search for the lowest
air fares and to determine how the fare level meets their flexibility constraints. To
meet the needs of these shoppers, we propose a new search method that significantly
enhances Web site usability for low fare searches. The new search method asks users
two key questions to estimate the user's time and budget constraints. The system
searches the search space for the user with one entry, thus eliminating the need for the
user to make multiple searches to cover all of the search space.
6.1 New Search Algorithm
The new search algorithm is coupled to a simple interface that is designed to fit on
one screen. The single screen allows the user to input origin, destination, departure
and return dates, class of service, number in party, flexibility of travel dates, and cost
savings parameter. The interface allows the user to set their constraints for travel date
flexibility and desired cost savings at the initial point of contact with the system via
the use of a slider mechanism and a box containing a pull-down selection menu.
These devices are used to answer the question, "How flexible are your travel dates
within a week?" and complete the sentence, "If the savings is less than $(answer) then
I would prefer a better schedule." The flexibility slider setting is converted into an
integer from 1 to 7 to indicate an approximate range for the user's flexibility. The
user then sends the query to the server. The server performs the search for the
possible flight combinations and then sorts them for the user based upon the new
algorithm.
While we used a slider mechanism to input flexibility, other types of input devices
could also be used. Other options are a pull-down selection menu or radio buttons to
indicate the number of days of flexibility. Each approach has its strengths and
weaknesses.
The algorithm performs seven different types of searches which are determined by the
input provided via the slider mechanism. The seven different types of searches are
based upon an analysis of the industry pricing structure, known characteristics of
travelers, and arbitrary assumptions correlating a users travel date flexibility and the
25


breadth of search to conduct. This information is deliberately hidden from the user as
the user should not be concerned with how the system is performing the search.
Type 1 Search: This is the search performed when the user leaves the slider in the
left-most position. It indicates no flexibility as to the dates of travel or travel times.
This user is not as concerned with price as they are with convenience. This is a
specific query search in that the search will be performed using only the user's dates
and times. The flights will be displayed to the user ordered by time of departure with
non-stop flights shown first followed by those with one or more stops and / or
connections.
Type 2 Search: If the user moves the slider slightly more to the right it indicates a
small amount of flexibility as to the dates and times of travel. This is interpreted as
indicating that the user has flexibility regarding the times of departure, but not the
date of departure. A search will be performed of all flights for the day(s) specified by
the shopper without respect to time. The display of the flights will be in increasing
price order with the lowest priced itinerary first. If more than one flight has the same
fare, then they are displayed ordered by time of departure.
Type 3 Search: If a user indicates more flexibility, a search will be performed from
6:00 p.m. the day prior to the specified departure date and until 12:00 p.m. the day
after the specified return date or, if a one way search, the departure date. This
accounts for users who may have the ability to leave the evening prior to their
indicated travel date and those that may have the ability to travel the morning after
their indicated travel date or dates. The flights are displayed as described for the
Type 2 search. If more than one itinerary has the same fare, they are displayed with
the flight closest in date to the user specified date or dates displayed first.
Type 4 Search: This search type will search the entire day before and the entire day
after the specified date or dates. Display of flights is the same as for the Type 3
search.
Type 5 Search: This search type will search two days prior to and after the specified
date or dates. Display of flights is the same as for the Type 3 search.
Type 6 Search: This search type will search three days prior to and after the specified
date or dates. Display of flights is the same as for the Type 3 search.
Type 7 Search: This is the search performed when the user moves the slider to the
right-most position. It indicates absolute flexibility as to the dates of travel and
travel times. This user is very concerned with price, so they are willing to change
26


their dates of travel to obtain a lower fare. This search type will search four days
prior to and after the specified date or dates. This was chosen as the maximum search
because it will move an itinerary from a Wednesday departure and return date to the
previous Saturday and to the following Sunday. Most of the lowest fares (though not
all) have a restriction requiring a Saturday overnight stay. By providing the ability to
move an itinerary so that a Saturday overnight stay is possible we can maximize the
probability of finding the lowest available fare for the user. Display of flights is the
same as for the Type 3 search.
In the spring of 2003 Orbitz began utilizing a similar type of search in its Web site.
However, it can only search up to three days prior and three days after the specified
dates. This has the possibility of missing the critical fourth day where fares may be
lower. The Orbitz site also allows the user to specify the number of days prior or
after the dates. While this provides more control for the user it places more
responsibility for knowledge of the fare structure upon the user. Since many users
don't understand the fare structure this appears to be unnecessary.
6.2 Ensuring User's Intent
After a search has been completed, a concern arises as to whether or not the system
has correctly ascertained the user's intended flexibility. The user has no feedback as
to the extent of the search they have requested using the slider, only that they have
indicated their flexibility to the interface. This situation can also occur using any
other type of input device. If the user believes that the system failed to correctly
interpret their intent, the user may try another search. This will inconvenience the
user, drain system resources that may be used to serve other customers, and defeat the
purpose behind the design to provide the desired information with one search attempt
by the user.
To counter this problem, an extra search is added to the end of the Type 2 through
Type 7 searches. This expands the search space by holding the return date (if any) at
the date input by the user, then searches one and two days prior to the earliest
departure date searched. Then the departure date is set at the date input by the user,
and a search is performed one and two days after the latest return date searched (if
any). The entire search space is not searched again. This provides an extra search to
determine if there are any lower fares available outside the search space as
determined by the interface and algorithm. If the interface incorrectly interpreted the
user's intent and the user actually wished a larger search space, this approach should
provide the ability to adequately search the desired space for the user. The result is
provided in a pop-up dialogue area called a "suggestion box". This appears as a
27


yellow box at the top of the first results display page. The box tells the shopper that a
fare of a certain price was found if the shopper can depart on a specific date. A
button is provided to allow the shopper to proceed to see those flight(s) on a separate
display page. If the shopper is not interested because the flight is outside of the
shopper's possible travel dates, the shopper can ignore the box and continue viewing
the display.
6.3 Cost Savings Calculation
The user's input for the cost saving's criteria is only considered for non-Type 1
searches. If a user has entered an amount for the cost savings variable, the prototype
will then process the results to determine if a lower fare itinerary matches the price
criteria set by the user. It does this during the sorting process to find the lowest fare.
First, a flight with the best schedule for the specific travel date is located. A flight is
considered best if it is a non-stop flight on the specified date or dates of travel. To the
fare of this flight is added the flight time t multiplied by a constant c to account for a
travelers time. This becomes the target fare F against which all other fares are
compared. A candidate itinerary is examined to determine if it is less than the target
fare F. It does this by examining the dates of travel, number of stops and connection
times. If a flight itinerary does not meet the original dates of travel, then a rate h is
added to the fare/for the flight itinerary for each day of difference d between the
dates of travel for the flight itinerary and the original dates of travel. This would
account for the cost of a hotel room. If a flight has a stop s, then an hourly rate c is
added to the fare/to account for the user's time. This would account for the extra
time required to take a flight that is not a non-stop flight. For example, a flight from
Denver to Los Angeles with a stop in Las Vegas will require more travel time since
the flight makes a stop in Las Vegas. If there is a flight connection at another point,
then c is added to the fare/for each hour t between the arrival time of the first flight
and the departure time of the second flight. The equation for flighted becomes:
freighted =f+(h*d) + (p*s) + (c*t)
(6.1)
with the variables
/- fare for the flight;
h rate for one night at a hotel;
d days of difference between the preferred travel date and the dates for the
flight(s);
28


p penalty rate to account for inconvenience due to a stop or possible
connections;
s number of stops a flight makes;
c hourly rate for a user's time during a stop or connection;
t hours of difference between arrival time of a flight and departure time of a
connection.
The fare /weighted for the flight itinerary is then compared to the target fare F. If /weighted
+ T < F, it is moved to the top of the display according to the other variables
mentioned previously. If it does not meet those criteria, then it remains in its current
place in the list of flights. For the prototype we used $90.00 for h, $50.00 for p and
$40.00 for c[l].
29


7. Prototype Description
In order to test the proposed guidelines and search algorithm, a prototype Web site
was developed that simulates an online travel agency. The prototype consists of
multiple Web pages constructed using HTML, JavaScript and CSS. The query page,
shown in Figure 1, provides the user with the means to enter the query parameters.
Figure 1 Query Screen
Once a query is submitted the system may display search results or respond with a
multiple airport selection page, shown in Figure 2. The multiple airport selection
page is displayed when there are multiple airports serving a city and the user must
make a decision regarding which airport or airports to search.
30


H Choose An Airport Microsoft Internet Explorer f- if&iffcl
Fife Edit View Favorites Torts He£ d9** i*] J) tl #*>* #***> 0 £><*> s I an i;D# ; :
Mdrmz ntto;//carbon.atdenver.edi/~ahito/cgi*rtn/search3,pl viElGo tirfa !
The Los Angeles, California area is served by multiple airports. Which airport would you like as your destination?
Los Angeies Internationa! O John Wayne Airport (Orange County) O Ontario International O Burbank 1 GJendate / Pasadena Airport O Long Beach International O Search All Airports
| FIND FLIGHTS |j CANCEL |[ NEW SEARCH |

Done i Id Enterrsel.
Figure 2 Multiple Airport Selection Page
The results display page is shown in Figure 3. This is the initial level of information
regarding the search results that is displayed to the user. From this page a shopper
should be able to select flights for further investigation regarding the desirability of
the flight and whether it meets the shoppers desires.
The results display consists of one or more pages of available flight combinations. At
the top of the first page is a header that informs the user of the date range that the
system actually searched. Not only does this inform the user of the breadth of the
search, but it is also intended to increase the credibility of the Web site. By showing
that the system has searched the database and that these are the best fares that could
be found with the data available, the user will perceive the expertise of the site, which
is a key element of credibility [7]. The flight combinations are displayed in a table
providing the necessary top level information for the user. Key elements of
information, such as fare and aircraft type, are actually links that the user can click to
access separate pages of information providing more detail for that specific
information area.
31


UIE Prototype Search Results Microsoft internet Explorer
Rfe Bit ttew Favorites Tools Help
0* %J [*I J!) y^Sear* note |fe<*a
&ddr**$ http;//carbc^,cjden'^ef.ekj/'>
op
a
Go Is-ts
RESULTS OF YOUR SEARCH
Based upon your flexibility, we have searched flights departing August 13 and returning
August 20.
£ Internet
Figure 3 Results Display
The fare details page is shown in Figure 4. This is a new window opened by clicking
on the fare link on the results display. From this page the shopper can review various
detailed information concerning the fare. The information includes details regarding
the base fare, taxes, and travel and refund restrictions.
32


H Air Fare Details - Microsoft Internet Explorer mam
Air Fare Details Ill
! Total Fare: 1 $424.00 US i .j j \
t | Base Fare: | j $381.40 US
j Mi Transportation Tax: 5 | $28.60 US i \ \ \ ? |
i US Flight j Segment Tax: 1 j $5.50 US i i i \
i US Passenger Fadfitv Charge ! $8.50 US 5 ! I i
| Cabin Type: i ! i Economy
j Fare Basts r ) tr H.TT* i i !?:
Figure 4 Fare Details Display
Figure 5 provides an example of a definition page for the U.S. Passenger Facility
Charge. Such pages are provided whenever a shopper may have a question regarding
the definition of a term or how it affects them. Such information would be available
to the shopper if they were using a travel agent. Pages such as this are designed to
provide this information in a meaningful format to the shopper to keep the shopper
informed.
33


It What is the Passenger Facility Charge? Micros.,.
What is the Passenger Facility
Charge?
The PASSENGER FACILITY CHARGE is a $1.00,
$2.00. S3.00, $4.00 or S4.50 charge assessed by as airport
for airport improvements. Seme airports are permitted to
charge die maximum amount. Others do not assess this
charge at aft. Up to four passenger facility charges can apply
to each ticketed journey.
Close
Figure 5 Passenger Facility Charge Definition
Aircraft information is provided via a link to another window as shown in Figure 6.
This page provides a photograph of the aircraft and a detailed description of the
aircraft scheduled to be used for the flight. This page is intended to provide detailed
information about the aircraft type for those shoppers who have a fear of flying on
smaller aircraft. While airlines do change the equipment types that fly specific flights
making the information provided out of date, such information provided to a user
does aid in providing more information to the shopper so that a more informed
purchase decision may be made.
34


Boeing 757 Microsoft Internet Explore!
Boeing 757
mi
Manufacturer:
I Aircraft Type:
Boemg Aircraft Company j
Jet
Body Type:
Seating Capacity:
I Narrow Bodv
24 First Class calm
j 158 Economy Class cabin
Figure 6 Aircraft Information
To provide other information to the shopper that is not necessary on the initial display
page another page of information is provided through a link on the initial results
display. An example of this is shown in Figure 7. This provides other details of the
flight that a shopper may be seeking but does not necessarily come under one of the
other categories on the initial results page. Use of such a page prevents clutter on the
initial results display and reduces the amount of information overload a shopper may
encounter.
35


Flight Details Microsoft Internet Explorer
Flight Details
| Stops: | Non-stop
j i Layover Time: 1 r I No layover required \
Meals: t | Snack
i EatertaiameBt: I | Audio and video entertainment | provided.
Close
Figure 7 Other Information Display
The backend consists of two Perl scripts. One script provides the query and search
function, including building of the various dynamic Web pages, and the second
provides display functions after the initial results are displayed. A sample database
was constructed for a two week period in August 2001 for the Denver to Los Angeles
and Denver to Santa Barbara markets using actual data captured in January 2001 from
the Apollo reservation system. The Apollo reservation system is operated by Galileo
International and provides reservation functions for travel agents in North America
and United Airlines. Not all of the flights in the markets were incorporated into the
database, but a sufficient sampling of direct and connecting flights were included to
provide the illusion of interacting with a CRS.
The prototype interface for the first round of tests had a slider to answer the question,
"How flexible are your travel dates?" The prototype interface for the second round of
tests revised the slider question to limit it to a one-week period. The first versions of
the second prototype interface incorporated a second slider to answer cost savings
36


criteria. However, initial user tests showed that this was confusing. After several
new designs were tested, the final version of the second interface was devised using a
pop-up box accessed through a button. The box further defined the question to
answer questions that users had concerning how to set the cost savings parameter. An
example of the query screen as used for an advanced search is shown in Figure 8.
The cost savings parameter box is shown in Figure 9.
UUi Prototype Microsoft Interm1! Explorer
Fte Edit Vie#? Favcrftes Tods Heb
O6** vJ tsi t) P5""*1 ^ ^ [3 Lj
http7/ca^bcn.cudenvpr.l(/M:cn3r:tD/aJeryv6,hbn) |gj| Go Lirils w
FROM: !Denver
TO:
Los Angeles
DEPART: lAugust 14.2031 | | CALENDAR j DEPARTING AT | Morning v| AM ~
RETURN: iAugust 17.2001 i [ CALENDAR J RETURNING AT | Morning V j AM s
(Leave Blank If You Are Searching for One-Way Only)
NUMBER OF SEATS DESIRED T §| CLASS ...........||
HOW FLEXIBLE ARE YOUR TRAVEL DATES WITHIN A WEEK?
NOT FLEXIBLE <------------------------------------> VERY FLEXIBLE
j LESS FLEXIBLE MORE FLEXIBLE
WOULD YOU LIKE TO FURTHER OPTIMIZE YOUR SEARCH? I YES 1
1 FIND FLIGHTS 1 [ CANCEL |
.... vOmtenet
Figure 8 Query Screen for Advanced Search
37


3 Ui£ Prototype Microsoft Internet Explorer Hfolfifi
Fife Edit View Favorites Tools Help §r
O8** .*1 is!l A >',Se*rfl !0 w u 3'.., #
Address httDi//carbon.cudeTvr.dj/'-,citartD/c^ietYv6.hhn3 V 0Go *
i i
FROM: !Denver
TO: j Los Angeles
DEPART: [August 14.2001..) | CALENDAR j DEPARTING AT [Mom'mg'
RETURN i Augustl 7.2001
RETURNING AT | Morning v
(Leffre Blank If You Are Searching for One-Way 0:
NUMBER OF SEATS DESIRED; 1 Hi CLASS
HOW FLEXIBLE ARE YOUR TRAVEL DATES WT
NOT FLEXIBLE <----------------------
LESS FLEXIBLE [
WOULD YOU LIKE TO FURTHER OPTIMIZE YOU Close This Window
CANCEL
We may provide a cheaper fere at the
cost of an unfavorable schedule How
much cheaper must the fare be for you
to consider an unfavorable schedule?
(Leave at $0 if you are looking for the
absolute cheapest fere.)
If the savings is less than [S20f> v
then I would prefer s hatter schedule
AM
AM
Â¥
Internet
Figure 9 Savings Parameter Input Box
When an advanced search is performed the system may find a better fare for the
shopper that is outside the flexibility range indicated by the shopper. In this scenario
a suggestion box as shown in Figure 10 is displayed on the initial results page. The
suggestion box is yellow and informs the user of the fare located and the dates of
travel. It also includes a button that will take the user to a display of the flights
available at the lower fare. If the shopper is not interested in the fare or dates of
travel as displayed in the suggestion box they can ignore the suggestion box and
proceed with their review of the other flights.
38


Prototype Search Results Microsoft internet Explorer
Ffe Edit View Favonces Tools Hdp
O8** sJ J*) 1x3 T#l !3 V
Address jti htto://carbon. cndenver .eduf ^ochanto/cgj-twr/searcho.pl
S3 Go
RESULTS OF YOUR SEARCH
Based upon your flexibility, we have searched flights departing August 17 and returning
August 19,
We found a fare of $423.00 if you can leave on Friday August 17 and j-----show methe "flights'"
return on Tues day August 21. '-------------------------------
Select o Fere Airfare! ^Hj Departs Oste Time Arrives Date Time Elapsed Time Stops! Csbm Aircraft 767 On- Time 70% Otrier Into YES
£424.00 wHE; e 17631 Denver United jj f Ftid-ay August 17 7:00 AM Los Angelas International Friday August 17 8:2S AM 2:26 0 Economy
PiP| 720 United | Los Angeles interoetonaj Sunday August i& 7-00 AM Denver Sunday August 19 10-18 AM 2:08 Economy 767 80% YES
o |lMP||l763 Unted fl Denver Friday August 17 700 AM Los Angeles International Friday August 17 | 2:26 | 0 ^conwny 757 70% YES
Unted. | Los Angeles international Sunday August 13 9:1 £ AM Denver Sunday August 13 2:19 H 0 Kconofw PM 1 P 8 SI f&A YES
.TWWVNBM*
m
S In tenet
Figure 10 Suggestion Box
When the shopper clicks on the button in the suggestion box they are taken to another
display screen as shown in Figure 11. The shopper may use the display screen to
investigate the flights further or proceed back to the original display of flights to
further review their details.
39



USE Prototype Search Results Microsoft internet Explorer
Ffe Edc View Favorites Tools Heto
O'** KJ :*) i*i ,w s~th '>*"** #* <*> 0.
Address ijjn htfe3://carbon.ajdenvr.du/" [Hs,
BETTER FARES WE FOUND FOR YOU
ill Done
^^tarnei
Figure 11 Display of Suggested Flights
Searching a database takes time, and more time is required to perform a broader
search such as the one we were investigating. Rudimentary comparisons were made
between the prototype response time and ITA Software's demonstrator site which
incorporates the ability to expand the search by one day as specified by the user. The
response times of the prototype did not appear realistic, so a delay page was
incorporated into the prototype to delay the search depending upon the search type.
This delay gives the user the illusion of a full flight search from a live system, and
also helps to gauge how patient users will be for an expanded search.
40


8. Experiment Description
Two different experiments were conducted. The first experiment involved tasks
aimed at testing the validity of the guidelines and searches using the flexibility slider.
The second experiment was performed after the revision of the first slider and
addition of the second parameter to indicate the user's desire for monetary savings.
The second experiment allowed more users to test the advance search capabilities of
the prototype.
The experiments were conducted using the Web sites of Expedia, Travelocity and the
prototype. This would provide a comparison between two production Web sites and
the prototype to determine if the new design was an improvement over the current
production sites. This would also aid in examining current usability problems
demonstrated on the production Web sites and if they were corrected with the new
prototype design.
The aim of the experiments was to test basic validity of the proposed design
guidelines and the new search method. This is difficult to accomplish since it is not
possible to test the design guidelines directly but rather must be done through an
interpretation of the guidelines via the prototype. In the same manner it is difficult to
test the new search method in a real world setting since the backend systems of
production e-commerce sites today are not designed to provide information for these
types of searches, nor does the prototype accurately replicate such a process of
searching through a large number of flight combinations. The prototype merely
mimics how a backend system may perform as far as time to perform the searches
and calculations. However, the prototype can provide the user with an imaginary
environment in which to perform such searches and to determine if the new search
method does in fact make shopping for the given scenarios easier for the user.
The guidelines and new search method are intended to improve the usability of sites
that provide airline flight and fare information. Determination of the improvement of
usability was based upon two separate criteria [10]. The first is that shoppers are able
to obtain specific pieces of information more often using the prototype as compared
to the production sites. The second criteria is user preference for use among the
three test sites. The experiments measured the aspects of efficiency, effectiveness
and satisfaction to attempt to capture a better understanding of the usability of each
site[8]. It was believed that the design guidelines and new search methodology could
increase user preference and accuracy with the prototype compared with the two
41


production sites with the same comparable amount of mouse clicks and time per
scenario. If possible a desired goal was to decrease the number of mouse clicks and
time per scenario.
The users were not permitted to become familiar with the Web sites prior to the tests.
The goal was to test users as they would react when coming to a Web site to shop.
This would be a disadvantage to the prototype as some test subjects had previous
experience with Expedia and Travelocity. However, this was considered acceptable
as it would replicate the situation when a new Web site was placed in production and
users experimented with it.
An issue that was not considered in the results was the issue of banner advertisements
and other query page items. Such items were present on the Web sites of both
Expedia and Travelocity, while they were not present on the prototype. The ability of
these banner advertisements and other items to distract users during the tests is
unknown. This would require a separate research study.
8.1 Criteria
The experiments were used to determine the validity of the guidelines and search
method using the following criteria:
A. Can the user successfully formulate a query based upon the task and receive the
appropriate display from the site? This includes the proper airports, dates, one-way
trip vs. round trip, and cabin.
B. Can the user successfully understand the basic flight information? This includes
airline, flight number, departure and arrival airports, dates and times, base fare, total
fare with taxes, refund policies, and charges to change flights. The basic flight
information is considered to be the information that a user would probably require in
order to make an informed purchasing decision. This information is known to be
available to a travel agent.
C. The user's ranking of the clarity of the information display.
D. The user's ranking of the difficulty of the task using the site.
E. The user's ranking of the three sites as to their perceived ease of use.
42


F. The user's ranking of the three sites as to their perceived ease of use for the low
fare searches.
G. The user's ranking of the three sites as to their preference when shopping (i.e.,
which one they would prefer to shop with).
8.2 First Experiment Description
The first experiment tested the five different scenarios. These were:
A. Find the available flights and fares for a First Class or Business Class seat one-
way between two specific cities departing after 11:00 a.m. on a specific date.
B. Find the available flights and fare for an Economy Class seat round trip between
two cities departing after 11:00 a.m. on a specific date and returning after 3:00 p.m.
on a specific date. One of the target cities was to be served by multiple airports, and
the search should include all airports serving that city.
C. Find the available flights and fares for an Economy Class seat one-way between
two cities departing after 8:00 a.m. on a specific date. One of the target cities was to
be served exclusively or primarily by one or more commuter airlines flying propeller
driven aircraft and regional jets.
D. Find the lowest fare for an Economy Class seat round trip between two cities
departing at any time on a specific date and returning any time on another specific
date. The user had the flexibility to leave one day prior to the specified departure
date and return one day after the specified return date if that would provide a lower
air fare.
E. Find the lowest fare for an Economy Class seat round trip between two cities
departing at any time on a specific date and returning any time on another specific
date. The user's goal was to take a seven day vacation, and could leave three days
prior to or after the specified departure date and return three days prior to or after the
specified return date.
These scenarios were designed to test various types of searches performed by
shoppers and to incorporate various problem areas identified in our research.
The tests were performed at various locations in the Denver metropolitan area. The
tests were conducted on computers with Internet Explorer compatible browsers. The
43


test subjects were accompanied by a facilitator that read the test scenarios to the user,
counted mouse clicks, recorded time, and made general observational notes. This
allowed the facilitator to free the user of the tasks of recording information and to free
the user to focus on the task. The facilitator could also note special problems that
users were experiencing with each Web site.
The tests were conducted under the guidelines of the Human Subjects Research
Committee of the University of Colorado at Denver. Human Subjects Research
Protocol #728 required that the tests last no longer than one hour. The subjects could
also quit a scenario or quit the test entirely at their request or when it was observed
they were becoming tired or frustrated.
8.3 Second Experiment Description
The second experiment tested two scenarios. The scenarios asked the user to imagine
that they were a business traveler, and they were to attend a two-day meeting. They
were to plan their travel according to certain guidelines provided by their company.
These scenarios were:
A. Find the lowest fare between two specified cities departing at any time on a
specified date and returning at any time on a specified date. The company had
authorized travel one day prior to the specified departure date and one day after the
specified return date if an air fare could be found that was at least $200.00 less than
an air fare found on the specified dates of travel.
B. Same scenario as above, except that there was no required threshold for the
savings, only that the lowest air fare be found.
The tests were performed remotely at various locations in the Denver metropolitan
area and around the United States. The tests were conducted on computers with
Internet Explorer compatible browsers. The test subjects were provided the written
test scenarios and asked to perform the tasks as described. The second round of tests
were performed in this manner as it was determined that a facilitator was not required
to be present to view the tests. The basic data obtained from the completed results
would provide sufficient information to judge how the users performed.
The tests were conducted under the guidelines of the Human Subjects Research
Committee of the University of Colorado at Denver. Human Subjects Research
Protocol #752 required that the tests last no longer than one hour. The subjects could
44


also quit a scenario or quit the test entirely at their request or when it was observed
they were becoming tired or frustrated.
45


9. Experiment Results
The first round of experiments were conducted in the summer of 2001. The second
round of experiments were conducted during the summer and fall of 2002. The
results of each experiment are explained further in the section dedicated to each
experiment.
9.1 First Experiment Results
There were seventeen subjects tested for the first experiment. The age range of the
subjects was from 18 to 65, with a mean age of 38.5 years and a median of 40 years.
The results for the successful completion of a query for Questions 1-5 from the first
experiment are shown in Table 1. A query was considered successful if the user was
able to obtain a search result for the correct city pairs, date or dates, and cabin. A
user failed to successfully complete a query if any of the search criteria were wrong
or if they quit the search.
Table 1 Successful Completion of a Query
Web Site Attempts Successful Completions Completion Rate
Expedia 52 31 59.61%
Travelocity 37 19 51.35%
Prototype 64 56 87.5%
The completion rates in Table 1 are significant in that they may be some indication of
how many users are actually able to perform a successful query using the production
sites. While the five tasks used were different, they each required an understanding
of the query interface. Many users became lost trying to manipulate the multiple
query pages using Expedia and Travelocity and would quit a search scenario. This is
especially true with any search scenario that required the advanced search page, such
as the one-way search scenarios. Many users tried to make the compact search
interface on the home page work for the search scenarios. They failed to see the link
46


that would lead them to an advanced search page where they could provide more
information for the search or specify a specific type of search, i.e., one-way search.
The results for the successful answering of basic flight and fare information, tasks
directly related to testing of the new design guidelines, are shown in Table 2. An
attempt was considered successful if the user was able to answer all of the questions
asked. If a user failed to answer any of the basic questions the attempt was
considered a failure.
Table 2 Successful Answering of Basic Information
Web Site Attempts to Answer Questions Successful Completions Completion Rate
Expedia 40 13 32.5%
Travelocity 31 0 0.0%
Prototype 63 51 80.95%
Table 2 is significant in that it is an indication of what occurs to users after they
successfully navigate the search interface and receive a resultant display. The
questions asked are pertinent to understanding the basic flight and fare information
necessary to make a purchasing decision. These questions were of the type that a
travel agent could answer for a customer. It is important to note that Travelocity had
no successes because it did not provide all of the information available through a CRS
or a travel agent, mainly the breakdown of the fare into the basic fare and applicable
taxes. Both Expedia and Travelocity suffered from a design problem in that the
available information was scattered across multiple pages. During the tests the
Travelocity site also experienced technical problems that prevented users from using
the site. The prototype experienced a technical problem during one user trial that
prevented the user from attempting the tests.
The prototype failed general in Scenario 3 when the user was asked if the flight was a
code share flight and which airline operated the flight. The prototype failed to
translate the design guideline into a display which the user could easily see and
interpret this information. The topic of code share flights and their identification is
important as the U.S. government is directing airlines to fully identify these flights.
More research into how to properly display this information to the user would be
beneficial.
47


The prototype was more successful for two reasons. The first is that the design
guidelines specify that the same information available to a travel agent be available
through the site. The second reason is the use of information scent and providing
information paths for the user to follow to the information desired. These two
guidelines, when coupled together, appear to address many of the usability problems
associated with travel sites.
Table 3 shows how the users ranked the clarity of the information display for each
site. The users were asked to rank the Web sites for clarity on a scale from 1 to 5
with a 1 indicating the Web site was very difficult to understand and a 5 indicating
the Web site was very easy to understand. This question was asked after each attempt
at a shopping scenario while the experience was still fresh in the users mind.
Table 3 Ranking of Clarity of Information Display
Web Site Mean Stan. Error of Mean
Expedia 2.669 0.268
Travelocity 2.779 0.224
Prototype 4.304 0.183
Note: Ratings based on Likert scale from 1 (very difficult) to 5 ( very easy).
The Friedman test resulted in a chi-square value of X2=19.882, df=2, p < 0.001.
Users ranked the clarity of the information display of the prototype more highly than
the other two sites, which confirms the test results when users were asked specific
questions regarding a flight or flight combination.
Table 4 shows how users ranked the difficulty of performing a task for each site.
The users were asked to rank the Web sites for difficulty in performing a task on a
scale from 1 to 5 with a 1 indicating the Web site was very difficult to use for the task
and a 5 indicating the Web site was very easy to use for a task. This question was
asked after each attempt at a shopping scenario while the experience was still fresh in
the users mind.
48


Table 4 Ranking of Difficulty of Performing a Task
Web Site Mean Stan. Error of Mean
Expedia 2.680 0.209
Travelocity 2.831 0.192
Prototype 4.337 0.131
Note: Ratings based on Likert scale from 1 (very difficult) to 5 (very easy).
The Friedman test resulted in a chi-square value of X2=24.088, df=2, p < 0.001.
Once again users ranked the prototype as easier to use to perform the tasks when
compared to the production sites. This agrees with the amount of success users
demonstrated with the prototype.
Users were asked to rank the three sites as 1, 2 and 3 as to their ease of use, with a 1
being the best and 3 the worst. The results are shown in Table 5.
Table 5 Ranking of Perceived Ease of Use
Web Site Mean Stan. Error of Mean
Expedia 2.142 0.150
Travelocity 2.529 0.125
Prototype 1 0.0
The Friedman test resulted in a chi-square value of X2=24.088, df=2, p < 0.001.
Users ranked the prototype as the easiest site to use, with Expedia second and
Travelocity third. Travelocity appears to be ranked third because several pieces of
information that were asked for during the tests were not available on Travelocity or
very difficult to locate.
Users were also asked to rank the sites as 1, 2 and 3 for the ease of use for the low
fare searches. The results are shown in Table 6.
49


Table 6 Ranking of Perceived Ease of Use for Low Fare Searches
Web Site Mean Stan. Error of Mean
Expedia 2.444 0.176
Travelocity 2.556 0.176
Prototype 1 0.0
The Friedman test resulted in a chi-square value of X =13.556, df=2, p < 0.005.
There were problems with the design of the flexibility slider for the prototype in that
users complained they did not have a frame of reference regarding the number of days
which the scale spanned. However, even with this problem, many users successfully
used the slider and found this an improvement over the low fare search approaches of
the production sites.
User rankings of preference of use for shopping are shown in Table 7, based on a
ranking scale of 1, 2 and 3.
Table 7 Ranking of Preference When Shopping
Web Site Mean Stan. Error of Mean
Expedia 2.412 0.173
Travelocity 2.471 0.125
Prototype 1.118 0.081
The Friedman test resulted in a chi-square value of X2=19.882, df=2, p < 0.001.
Users appeared to prefer the prototype because they could more easily answer the
questions during each scenario and they could use the advanced search tool for low
fare shopping.
The mean time and mean clicks for the users for Scenario 1 is shown in Table 8.
50


Table 8 Performance for Scenario 1
Web Site Successful Completion Rate Mean Time Mean Clicks
Expedia 68% 8:23 14.3125
Travelocity 57% 7:28 18.41667
Prototype 100% 6:26 12.76471
By the end of the first three scenarios the mean time and clicks were reduced as the
users learned the interfaces. The mean clicks for Travelocity became lower than the
other interfaces as users learned that they could not answer all of the questions about
the fares and so stopped trying to locate the information. The mean time for the
prototype remained below the others as users could more quickly locate the requested
information.
An interesting effect was that of the number of users willing to perform each scenario
with a specific site. The numbers dropped quickly for the production sites as users
became frustrated and less willing to use those sites to perform searches. The users
were less frustrated with the prototype and more willing to try certain scenarios. This
could translate into the amount of repeat customer traffic for a site.
Scenarios 4 and 5 pertained to the low fare search functions. Many users did not try
these for Expedia and Travelocity as they were having difficulty manipulating the
interfaces and understanding the query results. There was also a limit on the amount
of time that users could be tested, so this limited the number of users that were tested
on these scenarios for the first experiment. The results from those that were tested are
shown in Tables 9 and 10.
Table 9 Performance for Scenario 4
Web Site No. of Users Tested Successful Completion Rate Mean Time Mean Clicks
Expedia 8 37% 5:28 11.625
Travelocity 7 14% 6:40 15
Prototype 14 78% 7:06 8.357143
51


Table 10 Performance for Scenario 5
Web Site No. of Users Tested Successful Completion Rate Mean Time Mean Clicks
Expedia 6 16% 5:00 5.5
Travelocity 5 40% 5:34 6.666667
Prototype 8 77% 5:14 5.625
During the Expedia tests for Scenarios 4 and 5 one user found the low fare search tool
during each scenario. For Travelocity one found the tool for Scenario 4 and two for
Scenario 5. The remainder of the users that successfully completed the tasks
performed individual searches on the production sites. The amount of time for the
prototype may or may not reflect that of a production site due to the delay page that
was used to slow down the search process. However, the times and number of clicks
is comparable among the three sites, while the successful completion rate for the
prototype is higher than the production sites and their low fare search tool.
9.2 Second Experiment Results
There were nineteen subjects tested for the second experiment. The age range of the
subjects was from 19 to 49, with a mean age of 31.7 years.
The success rates for the second experiment are shown in Table 11. An answer for
both scenarios was considered successful for Expedia or Travelocity if four or more
searches were conducted by the user, which would cover the search space as
required. An answer for the prototype for the second scenario was considered
successful if the user found the target fare of $300.00, which was not on the specified
dates of travel but within the flexibility space of the scenario. An answer to the first
scenario using the prototype was considered correct in the user reported a fare other
than the $300.00 fare and managed to search the search space, either through using
the flexibility indicator or through four manual searches.
52


Table 11 Overall Task Completion Rates
Web Site Attempts to Answer Questions Successful Completions Completion Rate
Expedia 38 10 26.3%
Travelocity 38 11 28.9%
Prototype 38 27 71.1%
The results for the production sites were not unexpected. Many users were not
willing to go through four separate searches in order to locate the information as
required by the shopping scenarios. It is interesting to note that many of the users
would try two or three searches and then consider that they had searched the search
space entirely, not realizing that, because of the use of yield management and seat
availability, the best fare may be on the date or dates that the user had not searched.
The success rate for the prototype was much better than those of the production sites.
Many of the users were able to understand the use of the slider in conjunction with
the desired savings box. There was still some confusion over how to use the slider,
however, even with the new scale indicating that the time span for flexibility was
within one week of the intended travel dates.
An unexpected behavior was observed in several of the users that failed the scenarios
using the prototype. For the second round of tests the users were guided through the
scenarios in the same order, starting with Expedia, then on to Travelocity and finally
to the prototype. Several of the users expected the prototype to operate in the same
manner as the production sites. Therefore, these users felt that they had to perform
multiple searches using the prototype in the same manner that they did with the
production sites. These users would try two or more searches with the prototype, and
failed to notice the top of the results display that informed the users of the dates that
had been searched using the criteria input on the query page. This has implications
for any type of new interface and search method provided to users. If the users do not
adequately grasp that the new product differs from other products that they are
familiar with, they may attempt to use the new product incorrectly and thus defeat the
new features.
53


Table 12 Ranking of Ease of Use When Shopping for Lowest Fare
Web Site Mean Stan. Error of Mean
Expedia 2.1 0.186
Tra velocity 2.3 0.154
Prototype 1.6 0.192
Table 12 shows how users ranked the ease of use of the sites when shopping for the
lowest fare, with 1 being the best and 3 the worst. The Friedman test for Table 12
resulted in a chi-square value of X2=5.47, df=2, p < 0.05. While not statistically
significant, the results show a tendency toward preference for the prototype design.
However, these results are clouded by the difficulty of the test scenarios and the fact
that some users never used the savings tool. This result confirms the intuitive idea
that a site can have very good search tools, yet they will not have a significant impact
unless users can find them and properly use them.
Table 13 Rankings for Shopping for Lowest Fare with Threshold
Web Site Mean Stan. Error of Mean
Expedia 2.3 0.154
Travelocity 2.5 0.140
Prototype 1.2 0.086
Table 13 shows how users ranked the ease of use when shopping for the lowest fare
with a threshold, with 1 being the best and 3 the worst. The Friedman test for Table
13 resulted in a chi-square value of X2=20.63, df=2, p < 0.001. This question is
significant in that this pertains to the savings threshold question. The results of this
question confirm that those who located the savings tool found the prototype easier to
use than the conventional sites. Those who did not find the savings tool were still
able to have a higher success rate and therefore rated the prototype higher due to the
expanded search provided by the prototype and the display of the results.
54


Table 14 Ranking of Preference When Shopping
Web Site Mean Stan. Error of Mean
Expedia 2.3 0.150
Tr a velocity 2.2 0.191
Prototype 1.6 0.192
Table 14 shows how users ranked the sites for their preference when shopping, with
1 indicating their first choice and 3 indicating their least preferred site. The
Friedman test resulted in a chi-square value of X2=5.16, df=2, p < 0.05. Again, while
this result is not statistically significant, it does show a tendency of the users to prefer
the prototype design over the production sites.
i
55


10. Conclusions
The experiment results are encouraging regarding the ability of the proposed industry
specific guidelines to aid in designing more usable sites for the flight and fare search
function. While the prototype was not perfect in all areas of translating the proposed
guidelines to reality, it did outperform the production sites in the basic tests. Users
preferred the prototype over the production sites, and they were more successful in
answering basic shopping questions and completing searches using the prototype. It
would be interesting to see other designers translate the guidelines into production
sites and compare usability with the production sites to see if they can perform even
better than the prototype. This work has also been encouraging in that it does show
that there are simple ways to improve usability of travel sites for these functions.
Further research should be performed regarding how to effectively display the flight
and fare information to the shopper.
These experiments have also shown that the intents of the users for a specific type of
airline fare search can be extracted with a couple of questions based on industry
specific knowledge and a non-definitive, relative answering mechanism. The search
space can be adjusted automatically for the user to complete the search in one cycle
even if it involves various inter- and cross-dependent constraints. This type of search
mechanism also ensures that the user's intent is properly interpreted by performing an
extra search in an expanded search space and providing a suggestion box with
alternatives. Our user experiments demonstrated that our methods provide a smaller,
more compact interface and substantially enhance the overall usability for an e-
commerce site.
The drawbacks of this technique are that some users prefer more precise control of
their searches. This is not available through this technique, but can be provided with
a link to an advanced search page or other mechanism. It is also possible for some
users to misinterpret how to use the slider in conveying their intentions and thereby
perform searches and receive information that they do not desire. Some users also
displayed problems with the prototype in that they expected it to work like the other
sites, and so they attempted repeated searches, thus defeating the new design and
algorithm. Further research should be performed to determine how to overcome these
difficulties.
A specific point that can be made from this research is that airline and travel agency
sites need additional work to improve the usability of search and display functions for
56


flight and fare information. An encouraging fact, shown by our research, is that
usability can be improved greatly with industry-specific design guidelines, advanced
search techniques and focus upon the needs of the users.
57


A. First Round Test Questionnaire
QUESTIONNAIRE
(Facilitators Version)
Subject Number: ______________
Test Date: _____________
Start Time: ____________
Age: _____________
Gender: Male Female
Occupation: _________________________
Have this person ever worked in a reservation center, travel agency, or other travel-related industry?
Yes No
If Yes, what was their occupation and their employer? ______________________
Has this person ever shopped for merchandise or services on the Web? Yes No
If Yes, how often do they shop on the Web for merchandise or services? ____________
Has this person ever searched for airline flight information on the Web? Yes No
If Yes, how often do they shop on the Web for this information ? ___________
Has this person ever traveled in an aircraft that was propeller driven? Yes No
Is flying on a propeller driven aircraft a concern? Yes No
Has this person ever heard the term "codeshare" before? Yes No
What does the term "codeshare" mean? ________________________________________________
How important is it to this person to understand the types of restrictions and refund policies associated
with the purchase of an airline ticket, including overnight stay requirements and penalties for changing
a flight? Answer with a 1 being not important at all and a 5 being very important.
1 2 3 4 5
58


How important is it to this person to understand the types of services provided on a flight, such as
meals and entertainment? Answer with a 1 being not important at all and a 5 being very important.
1 2 3 4 5
Proceed to http://132.194.22.95/air/index.htm
Part I Expedia
Scenario 1
Find the available flights and fares for a First class or Business class seat from Cincinnati to
Washington, D.C.'s Dulles Airport on September 19, departing after 11 a.m.
Time: _____________
Question 1: Look at the first flight in the display and answer the following questions:
a. Airline and flight number: ____________
b. Departure time: _____________
c. Arrival time: ____________
d. What is the base fare (charge excluding taxes)?
e. What is the total fare with taxes included? __
f. Is the fare refundable? Yes No
g. Is there a charge to change the flight? Yes No
h. If there is a charge, how much is it? ________
Question 2: Do you feel that you understand the advance purchase requirements, travel restrictions
and refund policies for the fares? Yes No
Question 3: Can you tell if you would be flying on a jet or a propeller aircraft?
Yes No
Question 4: Do you feel that you understand the number of stops the flight will make and where?
Yes No
Question 5: Do you feel that you understand if a plane change is required and where?
Yes No
Question 6: Do you understand which cabin you will be in? Yes No
Question 7: Do you feel that you understand the type of service provided on the flight, such as meals
and entertainment? Yes No
Time: _____________
Question 8: Total time of search:
59


Question 9: Total number of mouse clicks: _______________
Question 10: Did the subject quit the search? Yes No
Question 11: How would you rate the clarity of the information display of this Web site with a 1 being
very difficult to understand and a 5 being easy to understand?
1 2 3 4 5
Question 12: How would you rate the difficulty of this task using this Web site with a 1 being very
difficult and a 5 being very easy?
1 2 3 4 5
Scenario 2
Find the available flights and fares for an Economy class seat from Cincinnati to Washington, D.C.
departing on September 19, departing after 11 a.m., and returning on September 26 after 3 p.m. You
are not searching for the lowest fares, just the available flights and times. You want to look at ALL
airports in the Washington, D.C. area.
Time: _____________
Question 1: Look at the first flight in the display and answer the following questions:
a. Airline and flight number of outbound flight:
b. Departure time: ____________
c. Arrival time: ____________
d. Arrival airport: ___________
e. Airline and flight number of return flight: __
f. Departure time: ____________
g. Arrival time: ____________
h. What is the base fare (charge excluding taxes)?
i. What is the total fare with taxes included? __
j. Is the fare refundable? Yes No
k. Is there a charge to change the flight? Yes No
l. If there is a charge, how much is it? ________
Question 2: Do you feel that you understand the advance purchase requirements, travel restrictions
and refund policies for the fares? Yes No
Question 3: Can you tell if you would be flying on a jet or a propeller aircraft?
Yes No
Question 4: Do you feel that you understand the number of stops the flight will make and where?
Yes No
60


Question 5: Do you feel that you understand if a plane change is required and where?
Yes No
Question 6: Do you understand which cabin you will be in? Yes No
Question 7: Do you feel that you understand the type of service provided on the flight, such as meals
and entertainment? Yes No
Time: _____________
Question 8: Total time of search: ______________
Question 9: Total number of mouse clicks: ______________
Question 10: Did the subject quit the search? Yes No
Question 11: How would you rate the clarity of the information display of this Web site with a 1 being
very difficult to understand and a 5 being easy to understand?
1 2 3 4 5
Question 12: How would you rate the difficulty of this task using this Web site with a 1 being very
difficult and a 5 being very easy?
1 2 3 4 5
Scenario 3
Find the available flights and fares for a one-way economy ticket from Cincinnati to Charleston, West
Virginia on September 24, departing after 8 a.m. You are not searching for the lowest fares, just the
available flights and times.
Time: _____________
Question 1: Look at the first flight in the display and answer the following questions:
a. Airline and flight number: ___________
b. Departure time: ____________
c. Arrival time: ______________
d. What is the base fare (charge excluding taxes)?
e. What is the total fare with taxes included? __
f. Is the fare refundable? Yes No
g. Is there a charge to change the flight? Yes No
h. If there is a charge, how much is it? _________
i. Which airline operates the flight? ___________
j. Is this a codeshare flight?___________
61


Question 2: Do you feel that you understand the advance purchase requirements, travel restrictions
and refund policies for the fares? Yes No
Question 3: Can you tell if you would be flying on a jet or a propeller aircraft?
Yes No
Question 4: Do you feel that you understand the number of stops the flight will make and where?
Yes No
Question 5: Do you feel that you understand if a plane change is required and where?
Yes No
Question 6: Do you understand which cabin you will be in? Yes No
Question 7: Do you feel that you understand the type of service provided on the flight, such as meals
and entertainment? Yes No
Time:
Question 8: Total time of search: ______________
Question 9: Total number of mouse clicks: ______________
Question 10: Did the subject quit the search? Yes No
Question 11: How would you rate the clarity of the information display of this Web site with a 1 being
very difficult to understand and a 5 being easy to understand?
1 2 3 4 5
Question 12: How would you rate the difficulty of this task using this Web site with a 1 being very
difficult and a 5 being very easy?
1 2 3 4 5
Scenario 4
Find the lowest fare for an Economy class seat from Cincinnati to Baltimore departing on September
19 and returning on September 26. You have the flexibility in your schedule to leave at any time on
September 18 and return at any time on September 27 if that will provide a lower air fare.
Time: ______________
Question 1: Answer the following questions for the lowest fare that you found:
a. Airline and flight number of outbound flight:
b. Departure date: __________
c. Departure time: __________
62


d.
e.
f.
g-
h.
i.
j-
k.
l.
Time:
Question 2:
Question 3:
Question 4:
Arrival time: ___________
Airline and flight number of return flight: __
Return date: ____________
Departure time: ____________
Arrival time: ___________
What is the base fare (charge excluding taxes)?
What is the total fare with taxes included? __
Is the fare refundable? Yes No
Is there a charge to change the flight? Yes No
If there is a charge, how much is it? ________
Total time of search:
Total number of mouse clicks:
Did the subject quit the search? Yes No
Question 5: Total number of separate searches required to obtain the information: ________________
Question 6: How would you rate the clarity of the information display of this Web site with a 1 being
very difficult to understand and a 5 being easy to understand?
1 2 3 4 5
Question 7: How would you rate the difficulty of this task using this Web site with a 1 being very
difficult and a 5 being very easy?
1 2 3 4 5
Scenario 5
Find the lowest fare for an Economy class seat from Cincinnati to Washington, D.C. departing on
September 19 and returning on September 25. You are a vacation traveler, and you wish to take a 7-
day vacation. You have the flexibility in your schedule to leave at any time from September 16
through September 22, and return at any time from September 22 through September 28 if that will
provide a lower air fare. You do not care which Washington, D.C. area airport you fly into or out of.
Time: _____________
Question 1: Answer the following questions for the lowest fare that you found:
a. Airline and flight number of outbound flight:
b. Departure date: __________
c. Departure time: __________
d. Arrival time: __________
63


e. Arrival airport: ____________
f. Airline and flight number of return flight: __
g. Return date: _____________
h. Departure time: _____________
i. Arrival time: _________
j. What is the base fare (charge excluding taxes)?
k. What is the total fare with taxes included? __
l. Is the fare refundable? Yes No
m. Is there a charge to change the flight? Yes No
n. If there is a charge, how much is it? _______
Time: ______________
Question 2: Total time of search: ___________
Question 3: Total number of mouse clicks: _____________
Question 4: Did the subject quit the search? Yes No
Question 5: Total number of separate searches required to obtain the information: ____________
Question 6: How would you rate the clarity of the information display of this Web site with a 1 being
very difficult to understand and a 5 being easy to understand?
1 2 3 4 5
Question 7: How would you rate the difficulty of this task using this Web site with a 1 being very
difficult and a 5 being very easy?
1 2 3 4 5
Part II Tra velocity
Scenario 1
Find the available flights and fares for a First class or Business class seat from Cleveland to New
York's LaGuardia Airport on September 19, departing after 11 a.m.
Time: ______________
Question 1: Look at the first flight in the display and answer the following questions:
a. Airline and flight number:
b. Departure time: __________
c. Arrival time: ____________
64


*
d. What is the base fare (charge excluding taxes)?
e. . What is the total fare with taxes included? _
f. Is the fare refundable? Yes No
g. Is there a charge to change the flight? Yes No
h. If there is a charge, how much is it? __________
Question 2: Do you feel that you understand the advance purchase requirements, travel restrictions
and refund policies for the fares? Yes No
Question 3: Can you tell if you would be flying on a jet or a propeller aircraft?
Yes No
Question 4: Do you feel that you understand the number of stops the flight will make and where?
Yes No
Question 5: Do you feel that you understand if a plane change is required and where?
Yes No
Question 6: Do you understand which cabin you will be in? Yes No
Question 7: Do you feel that you understand the type of service provided on the flight, such as meals
and entertainment? Yes No
Time:
Question 8: Total time of search:
Question 9: Total number of mouse clicks:
Question 10: Did the subject quit the search?
Yes
No
Question 11: How would you rate the clarity of the information display of this Web site with a 1 being
very difficult to understand and a 5 being easy to understand?
1 2 3 4 5
Question 12: How would you rate the difficulty of this task using this Web site with a 1 being very
difficult and a 5 being very easy?
1 2 3 4 5
Scenario 2
Find the available flights and fares for an Economy class seat from Cleveland to New York departing
on September 19, departing after 11 a.m., and returning on September 26 after 3 p.m. You are not
searching for the lowest fares, just the available flights and times. You want to look at all airports in
the New York area.
Time: ____________
65


Question 1: Look at the first flight in the display and answer the following questions:
a. Airline and flight number of outbound flight:
b. Departure time: ____________
c. Arrival time: ____________
d. Arrival airport: ___________
e. Airline and flight number of return flight: __
f. Departure time: ____________
g. Arrival time: ____________
h. What is the base fare (charge excluding taxes)?
i. What is the total fare with taxes included? __
j. Is the fare refundable? Yes No
k. Is there a charge to change the flight? Yes No
l. If there is a charge, how much is it? ________
Question 2: Do you feel that you understand the advance purchase requirements, travel restrictions
and refund policies for the fares? Yes No
Question 3: Can you tell if you would be flying on a jet or a propeller aircraft?
Yes No
Question 4: Do you feel that you understand the number of stops the flight will make and where?
Yes No
Question 5: Do you feel that you understand if a plane change is required and where?
Yes No
Question 6: Do you understand which cabin you will be in? Yes No
Question 7: Do you feel that you understand the type of service provided on the flight, such as meals
and entertainment? Yes No
Time:
Question 8: Total time of search:
Question 9: Total number of mouse clicks:
Question 10: Did the subject quit the search? Yes No
Question 11: How would you rate the clarity of the information display of this Web site with a 1 being
very difficult to understand and a 5 being easy to understand?
1 2 3 4 5
Question 12: How would you rate the difficulty of this task using this Web site with a 1 being very
difficult and a 5 being very easy?
1 2 3 4 5
66


Scenario 3
Find the available flights and fares for a one-way economy ticket from Cleveland to Charleston, West
Virginia on September 24, departing after 8 a.m. You are not searching for the lowest fares, just the
available flights and times.
Time: ______________
Question 1: Look at the first flight in the display and answer the following questions:
a. Airline and flight number: ____________
b. Departure time: _____________
c. Arrival time: ____________
d. What is the base fare (charge excluding taxes)?
e. What is the total fare with taxes included? __
f. Is the fare refundable? Yes No
g. Is there a charge to change the flight? Yes No
h. If there is a charge, how much is it? _________
i. Which airline operates the flight? ___________
j. Is this a codeshare flight? ___________
Question 2: Do you feel that you understand the advance purchase requirements, travel restrictions
and refund policies for the fares? Yes No
Question 3: Can you tell if you would be flying on a jet or a propeller aircraft?
Yes No
Question 4: Do you feel that you understand the number of stops the flight will make and where?
Yes No
Question 5: Do you feel that you understand if a plane change is required and where?
Yes No
Question 6: Do you understand which cabin you will be in? Yes No
Question 7: Do you feel that you understand the type of service provided on the flight, such as meals
and entertainment? Yes No
Time:
Question 8:
Question 9:
Question 10:
Total time of search: _______________
Total number of mouse clicks: ________________
Did the subject quit the search? Yes
No
67


Question 11: How would you rate the clarity of the information display of this Web site with a 1 being
very difficult to understand and a 5 being easy to understand?
1 2 3 4 5
Question 12: How would you rate the difficulty of this task using this Web site with a 1 being very
difficult and a 5 being very easy?
1 2 3 4 5
Scenario 4
Find the lowest fare for an Economy class seat from Cleveland to Newark departing on September 19
and returning on September 26. You have the flexibility in your schedule to leave at any time on
September 18 and return at any time on September 27 if that will provide a lower air fare.
Time: ____________
Question 1:
b.
c.
d.
e.
f.
g-
h.
i.
j-
k.
l.
Time:
Question 2:
Question 3:
Question 4:
Answer the following questions for the lowest fare that you found:
Airline and flight number of outbound flight: ________
Departure date: __________
Departure time: __________
Arrival time: ____________
Airline and flight number of return flight: ________
Return date: _____________
Departure time: __________
Arrival time: ____________
What is the base fare (charge excluding taxes)? __________
What is the total fare with taxes included? ________
Is the fare refundable? Yes No
Is there a charge to change the flight? Yes No
If there is a charge, how much is it? _________
Total time of search:
Total number of mouse clicks:
Did the subject quit the search? Yes No
Question 5:
Total number of separate searches required to obtain the information:
68


Question 6: How would you rate the clarity of the information display of this Web site with a 1 being
very difficult to understand and a 5 being easy to understand?
1 2 3 4 5
Question 7: How would you rate the difficulty of this task using this Web site with a 1 being very
difficult and a 5 being very easy?
1 2 3 4 5
Scenario 5
Find the lowest fare for an Economy class seat from Cleveland to New York departing on September
19 and returning on September 25. You are a vacation traveler, and you wish to take a 7-day vacation.
You have the flexibility in your schedule to leave at any time from September 16 through September
22, and return at any time from September 22 through September 28 if that will provide a lower air
fare. You do not care which New York airport you fly into or out of.
Time: ______________
Question 1:
a.
b.
c.
d.
e.
f.
g-
h.
i.
j-
k.
l.
Time:
Question 2:
Question 3:
Question 4:
Answer the following questions for the lowest fare that you found:
Airline and flight number of outbound flight: ________
Departure date: ___________
Departure time: ___________
Arrival time: _____________
Arrival airport: __________
Airline and flight number of return flight: __________
Return date: ______________
Departure time: ___________
Arrival time: _____________
What is the base fare (charge excluding taxes)? __________
What is the total fare with taxes included? __________
Is the fare refundable? Yes No
Is there a charge to change the flight? Yes No
If there is a charge, how much is it? _________
Total time of search:
Total number of mouse clicks:
Did the subject quit the search? Yes No
Question 5:
Total number of separate searches required to obtain the information:
69


Question 6: How would you rate the clarity of the information display of this Web site with a 1 being
very difficult to understand and a 5 being easy to understand?
1 2 3 4 5
Question 7: How would you rate the difficulty of this task using this Web site with a 1 being very
difficult and a 5 being very easy?
12 3 4 5
Part III -AURARIA
Scenario 1
Find the available flights and fares for a First class or Business class seat from Denver to Los Angeles
International Airport on August 15, departing after 11 a.m.
Time: ______________
Question 1: Look at the first flight in the display and answer the following questions:
a. Airline and flight number: ____________
b. Departure time: _____________
c. Arrival time: ____________
d. What is the base fare (charge excluding taxes)?
e. What is the total fare with taxes included? __
f. Is the fare refundable? Yes No
g. Is there a charge to change the flight? Yes No
h. If there is a charge, how much is it? ________
Question 2: Do you feel that you understand the advance purchase requirements, travel restrictions
and refund policies for the fares? Yes No
Question 3: Can you tell if you would be flying on a jet or a propeller aircraft?
Yes No
Question 4: Do you feel that you understand the number of stops the flight will make and where?
Yes No
Question 5: Do you feel that you understand if a plane change is required and where?
Yes No
Question 6: Do you understand which cabin you will be in?
Yes
No
70


Question 7: Do you feel that you understand the type of service provided on the flight, such as meals
and entertainment? Yes No
Time: _____________
Question 8: Total time of search: ______________
Question 9: Total number of mouse clicks: ______________
Question 10: Did the subject quit the search? Yes No
Question 11: How would you rate the clarity of the information display of this Web site with a 1 being
very difficult to understand and a 5 being easy to understand?
1 2 3 4 5
Question 12: How would you rate the difficulty of this task using this Web site with a 1 being very
difficult and a 5 being very easy?
1 2 3 4 5
Scenario 2
Find the available flights and fares for an Economy class seat from Denver to Los Angeles departing
on August 15, departing after 11 a.m., and returning on August 21 after 3 p.m. You are not searching
for the lowest fares, just the available flights and times. You want to look at all airports in the Los
Angeles area.
Time: ____________
Question 1: Look at the first flight in the display and answer the following questions:
a. Airline and flight number of outbound flight:
b. Departure time: _____________
c. Arrival time: ____________
d. Arrival airport: ____________
e. Airline and flight number of return flight: __
f. Departure time: _____________
g. Arrival time: ____________
h. What is the base fare (charge excluding taxes)?
i. What is the total fare with taxes included? __
j. Is the fare refundable? Yes No
k. Is there a charge to change the flight? Yes No
l. If there is a charge, how much is it? ________
Question 2: Do you feel that you understand the advance purchase requirements, travel restrictions
and refund policies for the fares? Yes No
71


Question 3: Can you tell if you would be flying on a jet or a propeller aircraft?
Yes No
Question 4: Do you feel that you understand the number of stops the flight will make and where?
Yes No
Question 5: Do you feel that you understand if a plane change is required and where?
Yes No
Question 6: Do you understand which cabin you will be in?
Yes
No
Question 7: Do you feel that you understand the type of service provided on the flight, such as meals
and entertainment? Yes No
Time:
Question 8: Total time of search: ______
Question 9: Total number of mouse clicks:
Question 10: Did the subject quit the search? Yes No
Question 11: How would you rate the clarity of the information display of this Web site with a 1 being
very difficult to understand and a 5 being easy to understand?
1 2
3 4 5
Question 12: How would you rate the difficulty of this task using this Web site with a 1 being very
difficult and a 5 being very easy?
1 2 3 4 5
Scenario 3
Find the available flights and fares for a one-way economy ticket from Denver to Santa Barbara,
California on August 16, departing after 8 a.m. You are not searching for the lowest fares, just the
available flights and times.
Time: _____________
Question 1: Look at the first flight in the display and answer the following questions:
a. Airline and flight number: ___________
b. Departure time: ____________
c. Arrival time: ___________
d. What is the base fare (charge excluding taxes)?
e. What is the total fare with taxes included? _
f. Is the fare refundable? Yes No
g. Is there a charge to change the flight? Yes No
72


h. If there is a charge, how much is it?
i. Which airline operates the flight? _
j. Is this a codeshare flight? ______
Question 2: Do you feel that you understand the advance purchase requirements, travel restrictions
and refund policies for the fares? Yes No
Question 3: Can you tell if you would be flying on a jet or a propeller aircraft?
Yes No
Question 4: Do you feel that you understand the number of stops the flight will make and where?
Yes No
Question 5: Do you feel that you understand if a plane change is required and where?
Yes No
Question 6: Do you understand which cabin you will be in?
Yes
No
Question 7: Do you feel that you understand the type of service provided on the flight, such as meals
and entertainment? Yes No
Time:
Question 8: Total time of search:
Question 9: Total number of mouse clicks: _______________
Question 10: Did the subject quit the search? Yes No
Question 11: How would you rate the clarity of the information display of this Web site with a 1 being
very difficult to understand and a 5 being easy to understand?
1 2 3 4 5
Question 12: How would you rate the difficulty of this task using this Web site with a 1 being very
difficult and a 5 being very easy?
1 2 3 4 5
Scenario 4
Find the lowest fare for an Economy class seat from Denver to Orange County departing on August 14
and returning on August 21. You have the flexibility in your schedule to leave at any time on August
13 and return at any time on August 22 if that will provide a lower air fare.
Time: _____________
73


Question 1:
b.
c.
d.
e.
f.
g-
h.
i.
j-
k.
l.
Time:
Question 2:
Question 3:
Question 4:
Answer the following questions for the lowest fare that you found:
Airline and flight number of outbound flight: ___________
Departure date: ____________
Departure time: ____________
Arrival time: ___________
Airline and flight number of return flight: ________
Return date: ____________
Departure time: ____________
Arrival time: ___________
What is the base fare (charge excluding taxes)? _________
What is the total fare with taxes included? ________
Is the fare refundable? Yes No
Is there a charge to change the flight? Yes No
If there is a charge, how much is it? _________
Total time of search: ______________
Total number of mouse clicks: _______________
Did the subject quit the search? Yes No
Question 5:
Question 6:
Question 7:
Question 8:
very difficult
Total number of separate searches required to obtain the information: _____________
Did the "extra search box" come up? Yes No
Dates actually searched (1st attempt): ____________through______________
How would you rate the clarity of the information display of this Web site with a 1 being
to understand and a 5 being easy to understand?
1 2 3 4 5
Question 9: How would you rate the difficulty of this task using this Web site with a 1 being very
difficult and a 5 being very easy?
1 2 3 4 5
Question 10: Were you satisfied with the information returned on the first attempt of the search, even
if it did not meet all of your date requirements? Yes No
Question 11: Did the amount of time you had to wait for the results of the comprehensive search
irritate you? Please rate your level of irritation with the wait using a 1 for very irritating and a 5 for not
irritating.
1 2 3 4 5
74


Question 12: Was the amount of time you had to wait for the results of the search worth while when
compared to the ability to perform a more comprehensive search with a single search entry?
Yes No
Scenario 5
Find the lowest fare for an Economy class seat from Denver to Los Angeles departing on August 13
and returning on August 19. You are a vacation traveler, and you wish to take a 7-day vacation. You
have the flexibility in your schedule to leave at any time from August 10 through August 16, and
return at any time from August 16 through August 22 if that will provide a lower air fare. You do not
care which Los Angeles airport you fly into or out of.
Time: ___________
Question 1:
b.
c.
d.
e.
f.
g-
h.
i.
j-
k.
l.
m.
n.
Time:
Answer the following questions for the lowest fare that you found:
Airline and flight number of outbound flight: ___________
Departure date: ____________
Departure time: ____________
Arrival time: ___________
Arrival airport: ___________
Airline and flight number of return flight: _________
Return date: ____________
Departure time: ____________
Arrival time: ___________
What is the base fare (charge excluding taxes)? _________
What is the total fare with taxes included? _________
Is the fare refundable? Yes No
Is there a charge to change the flight? Yes No
If there is a charge, how much is it? _________
Question 2: Total time of search:
Question 3: Total number of mouse clicks:
Question 4: Did the subject quit the search?
Yes
No
Question 5: Total number of separate searches required to obtain the information:
Question 6: Did the "extra search box" come up? Yes No
Question 7: Dates actually searched (1st attempt): ___________through___________
75


Question 8: How would you rate the clarity of the information display of this Web site with a 1 being
very difficult to understand and a 5 being easy to understand?
1 2 3 4 5
Question 9: How would you rate the difficulty of this task using this Web site with a 1 being very
difficult and a 5 being very easy?
1 2 3 4 5
Question 10: Were you satisfied with the information returned on the first attempt of the search, even
if it did not meet all of your date requirements? Yes No
Question 11: Did the amount of time you had to wait for the results of the comprehensive search
irritate you? Please rate your level of irritation with the wait using a 1 for very irritating and a 5 for not
irritating.
1 2 3 4 5
Question 12: Was the amount of time you had to wait for the results of the search worth while when
compared to the ability to perform a more comprehensive search with a single search entry?
Yes No
Scenario 6
Find the lowest fare for an Economy class seat from Denver to Long Beach departing on August 11
and returning on August 15. You are visiting relatives in Long Beach, so you have absolute flexibility
as far as your travel dates, but you wish to fly into Long Beach airport.
Time: ______________
Question 1: Answer the following questions for the lowest fare that you found:
a. Airline and flight number of outbound flight: _
b. Departure date: ___________
c. Departure time: ___________
d. Arrival time: ___________
e. Arrival airport: __________
f. Airline and flight number of return flight: _
g. Return date: ____________
h. Departure time: ___________
i. Arrival time: ___________
j. What is the base fare (charge excluding taxes)?
k. What is the total fare with taxes included? _
l. Is the fare refundable? Yes No
m. Is there a charge to change the flight? Yes No
n. If there is a charge, how much is it? _______
Time: _____________
76


Question 2:
Question 3:
Question 4:
Total time of search: _______________
Total number of mouse clicks: ________________
Did the subject quit the search? Yes
No
Question 5:
Question 6:
Question 7:
Total number of separate searches required to obtain the information:
Did the "extra search box" come up? Yes No
Dates actually searched (1st attempt): ____________through____________
Question 8: How would you rate the clarity of the information display of this Web site with a 1 being
very difficult to understand and a 5 being easy to understand?
Question 9: How would you rate the difficulty of this task using this Web site with a 1 being very
difficult and a 5 being very easy?
12 3 4 5
Question 10: Were you satisfied with the information returned on the first attempt of the search, even
if it did not meet all of your date requirements? Yes No
Question 11: Did the amount of time you had to wait for the results of the comprehensive search
irritate you? Please rate your level of irritation with the wait using a 1 for very irritating and a 5 for not
irritating.
1 2 3 4 5
Question 12: Was the amount of time you had to wait for the results of the search worth while when
compared to the ability to perform a more comprehensive search with a single search entry?
Yes No
AURARIA Specific Questions
Question 1: Did you like the "slider" as an input device to indicate your flexibility? Yes
No
Question 2: Did you understand the concept behind the slider (i.e., how it works) by viewing it?
Yes No
Question 3: Did you like the ability to perform a more comprehensive search with one entry?
Yes No
77


Overall Comparisons
Question 1: Comparing the overall display of information based upon the questions asked following
each task, how would you rank the Web sites according to their ease of use? Rank the best as 1 and
the worst as 3.
Expedia_______ Travelocity______ AURARIA___________
Question 2: Comparing the overall ease of use of the Web sites in the tasks of searching for the
lowest fare over a period of time (Scenario 4 and Scenario 5), how would you rank the Web sites
according to their ease of use? Rank the best as 1 and the worst as 3.
Expedia_______ Travelocity______ AURARIA___________
Question 3: Comparing the overall display of information based upon your own personal preference
(for example, how you would shop), how would you rank the Web sites according to their ease of use?
Rank the best as 1 and the worst as 3.
Expedia_______ Travelocity ___ AURARIA___________
End Time:
78


B. Second Round Test Questionnaire
QUESTIONNAIRE
Second Round of Tests
(Facilitators Version)
Subject Number: ______________
Test Date: ____________
Start Time: ___________
Age: _____________
Gender: Male Female
Occupation: ________________________
Have this person ever worked in a reservation center, travel agency, or other travel-related industry?
Yes No
If Yes, what was their occupation and their employer? _____________________
Has this person ever shopped for merchandise or services on the Web? Yes No
If Yes, how often do they shop on the Web for merchandise or services? ___________
Has this person ever searched for airline flight information on the Web? Yes No
If Yes, how often do they shop on the Web for this information? ___________
Proceed to http://carbon.cudenver.edu/~ccharito/index.htm
Part I Expedia
Scenario 1
You are flying from Cincinnati to Baltimore on a business trip for your company. You have all-day
meetings on October 23 and October 24. You want to depart anytime on October 22 and return
anytime on October 25. However, you are somewhat flexible in your travel dates. Your company has
authorized you to leave on October 21 and / or return on October 26 if you can find a fare that will
save at least $200.00 off of what you would pay if you left on October 22 and returned on October 25.
79


Start Time:
Question 1: Look at the first flight that meets the criteria of the scenario and answer the following
questions:
a. Airline and flight number of outbound flight:
b. Departure date: __________
c. Departure time: __________
d. Arrival time: __________
e. Airline and flight number of return flight: _
f. Return date: ___________
g. Departure time: __________
h. Arrival time: __________
i. What is the total fare with taxes included? _
Finish Time: _____________
Question 2: Total time of search: ____________
Question 3: Total number of mouse clicks: ____________
Question 4: Did the subject quit the search? Yes No
Question 5: Total number of separate searches required to obtain the information: _______________
Question 6: How would you rate the clarity of the information display of this Web site with a 1 being
very difficult to understand and a 5 being easy to understand?
1 2 3 4 5
Question 7: How would you rate the difficulty of this task using this Web site with a 1 being very
difficult and a 5 being very easy?
1 2 3 4 5
Scenario 2
You are flying from Cincinnati to Atlanta on a business trip for your company. You have all-day
meetings on October 23 and October 24. You want to depart anytime on October 22 and return
anytime on October 25. However, you are somewhat flexible in your travel dates. Your company has
authorized you to leave on October 21 and / or return on October 26 if that will provide you with the
lowest fare.
Start Time: ________________
Question 1: Answer the following questions for the lowest fare that you found:
a. Airline and flight number of outbound flight: _______________
80


b. Departure date: ____________
c. Departure time: ____________
d. Arrival time: ____________
e. Arrival airport: ___________
f. Airline and flight number of return flight:
g. Return date: _____________
h. Departure time: ____________
i. Arrival time: ____________
j. What is the total fare with taxes included?
Finish Time: ______________
Question 2: Total time of search: ___________
Question 3: Total number of mouse clicks: ___________
Question 4: Did the subject quit the search? Yes
No
Question 5: Total number of separate searches required to obtain the information: ______________
Question 6: How would you rate the clarity of the information display of this Web site with a 1 being
very difficult to understand and a 5 being easy to understand?
1 2 3 4 5
Question 7: How would you rate the difficulty of this task using this Web site with a 1 being very
difficult and a 5 being very easy?
1 2 3 4 5
Proceed to http://carbon.cudenver.edu/~ccharito/index.htm
Part II Travelocity
Scenario 1
You are flying from Cleveland to Newark on a business trip for your company. You have all-day
meetings on October 23 and October 24. You want to depart anytime on October 22 and return
anytime on October 25. However, you are somewhat flexible in your travel dates. Your company has
authorized you to leave on October 21 and / or return on October 26 if you can find a fare that will
save at least $200.00.
Start Time: ________________
81


Question 1:
a.
b.
c.
d.
e.
f.
g-
h.
i.
Finish Time:
Question 2:
Question 3:
Question 4:
Answer the following questions for the lowest fare that you found:
Airline and flight number of outbound flight: _______
Departure date: ___________
Departure time: ___________
Arrival time: ___________
Airline and flight number of return flight: _________
Return date: ____________
Departure time: ___________
Arrival time: ___________
What is the total fare with taxes included? _________
Total time of search:
Total number of mouse clicks:
Did the subject quit the search? Yes No
Question 5: Total number of separate searches required to obtain the information: _______________
Question 6: How would you rate the clarity of the information display of this Web site with a 1 being
very difficult to understand and a 5 being easy to understand?
1 2 3 4 5
Question 7: How would you rate the difficulty of this task using this Web site with a 1 being very
difficult and a 5 being very easy?
1 2 3 4 5
Scenario 2
You are flying from Cleveland to Houston on a business trip for your company. You have all-day
meetings on October 23 and October 24. You want to depart anytime on October 22 and return
anytime on October 25. However, you are somewhat flexible in your travel dates. Your company has
authorized you to leave on October 21 and / or return on October 26 if that will provide you with the
lowest fare.
Start Time:
Question 1: Answer the following questions for the lowest fare that you found:
a. Airline and flight number of outbound flight:
b. Departure date: __________
c. Departure time: __________
82


d. Arrival time: _____________
e. Arrival airport: __________
f. Airline and flight number of return flight:
g. Return date: ______________
h. Departure time: ___________
i. Arrival time: _____________
j. What is the total fare with taxes included?
Finish Time: ______________
Question 2: Total time of search: ___________
Question 3: Total number of mouse clicks: __________
Question 4: Did the subject quit the search? Yes
No
Question 5: Total number of separate searches required to obtain the information: ___________
Question 6: How would you rate the clarity of the information display of this Web site with a 1 being
very difficult to understand and a 5 being easy to understand?
1 2 3 4 5
Question 7: How would you rate the difficulty of this task using this Web site with a 1 being very
difficult and a 5 being very easy?
1 2 3 4 5
Proceed to http://carbon.cudenver.edu/~ccharito/index.htm
Part HI UIE
Scenario 1
You are flying from Denver to Los Angeles on a business trip for your company. You have all-day
meetings on August 15 and August 16. You want to depart anytime on August 14 and return anytime
on August 17. However, you are somewhat flexible in your travel dates. Your company has
authorized you to leave on August 13 and / or return on August 18 if you can find a fare that will save
at least $200.00.
Start Time: ________________
Question 1: Answer the following questions for the lowest fare that you found:
a. Airline and flight number of outbound flight: _______
b. Departure date: __________
83


c. Departure time: ____________
d. Arrival time: ___________
e. Airline and flight number of return flight:
f. Return date: ____________
g. Departure time: ____________
h. Arrival time: ___________
i. What is the total fare with taxes included?
Finish Time: _____________
Question 2: Total time of search: ____________
Question 3: Total number of mouse clicks: ___________
Question 4: Did the subject quit the search? Yes
No
Question 5: Total number of separate searches required to obtain the information:
Question 6: Not Applicable
Question 7: Dates actually searched (1st attempt): _______________through_______________(From top
of initial results page.)
Question 8: How would you rate the clarity of the information display of this Web site with a 1 being
very difficult to understand and a 5 being easy to understand?
1 2 3 4 5
Question 9: How would you rate the difficulty of this task using this Web site with a 1 being very
difficult and a 5 being very easy?
1 2 3 4 5
Question 10: Were you satisfied with the information returned on the first attempt of the search, even
if it did not meet all of your date requirements? Yes No
Question 11: Did the amount of time you had to wait for the results of the comprehensive search
irritate you? Please rate your level of irritation with the wait using a 1 for very irritating and a 5 for not
irritating.
1 2 3 4 5
Question 12: Was the amount of time you had to wait for the results of the search worth while when
compared to the ability to perform a more comprehensive search with a single search entry?
Yes No
84


Scenario 2
You are flying from Denver to Los Angeles on a business trip for your company. You have all-day
meetings on August 15 and August 16. You want to depart anytime on August 14 and return anytime
on August 17. However, you are somewhat flexible in your travel dates. Your company has
authorized you to leave on August 13 and / or return on August 18 if that will provide you with the
lowest fare.
Start Time:
Question 1:
a.
b.
c.
d.
e.
f.
g-
h.
i.
j-
Finish Time:
Question 2:
Question 3:
Question 4:
Answer the following questions for the lowest fare that you found:
Airline and flight number of outbound flight: __________
Departure date: ____________
Departure time: ____________
Arrival time: ___________
Arrival airport: ___________
Airline and flight number of return flight: _________
Return date: ____________
Departure time: ____________
Arrival time: ___________
What is the total fare with taxes included? _________
Total time of search: _______________
Total number of mouse clicks: ________________
Did the subject quit the search? Yes No
Question 5:
Question 6:
Question 7:
Total number of separate searches required to obtain the information:
Not Applicable
________through___________
Dates actually searched (1st attempt):
of initial results page.)
. (From top
Question 8: How would you rate the clarity of the information display of this Web site with a 1 being
very difficult to understand and a 5 being easy to understand?
1 2 3 4 5
Question 9: How would you rate the difficulty of this task using this Web site with a 1 being very
difficult and a 5 being very easy?
1 2 3 4 5
85


Question 10: Were you satisfied with the information returned on the first attempt of the search, even
if it did not meet all of your date requirements? Yes No
Question 11: Did the amount of time you had to wait for the results of the comprehensive search
irritate you? Please rate your level of irritation with the wait using a 1 for very irritating and a 5 for not
irritating.
1 2 3 4 5
Question 12: Was the amount of time you had to wait for the results of the search worth while when
compared to the ability to perform a more comprehensive search with a single search entry?
Yes No
UIE Specific Questions
Question 1: Did you like the "slider as an input device to indicate your flexibility? Yes
No
Question 2: Did you understand the concept behind the slider (i.e., how it works) by viewing it?
Yes No
Question 3: Did you like the ability to perform a more comprehensive search with one entry?
Yes No
Overall Comparisons
Question 1: Comparing the overall ease of use of the Web sites in the tasks of searching for the
lowest fare, how would you rank the Web sites according to their ease of use? Rank the best as 1 and
the worst as 3.
Expedia______ Travelocity______ UIE________
Question 2: Comparing the overall ease of use of the Web sites in the tasks of searching for the
lowest fare if it meets a certain savings threshold, how would you rank the Web sites according to their
ease of use? Rank the best as 1 and the worst as 3.
Expedia______ Travelocity______ UIE________
Question 3: Comparing the overall display of information based upon your own personal preference
(for example, how you would shop), how would you rank the Web sites according to their ease of use?
Rank the best as 1 and the worst as 3.
Expedia______ Travelocity______ UIE________
End Time:
86


References
1. Adler, T. Air Travelers 2001: What Do They Tell Us About the Future of U.S. Air
Travel, 2001. Available from http://www.surveycafe.com/surveycafe/air/air.asp.
2. Carr, D. F. "How Travelocity Gives Consumers Rock-Bottom Airfares," Internet
World, May 17, 1999. Available from
http://www.intemetworld.com/print/1999/05/17/webdev/19990517-how.html.
3. Daudel, S., & G. Vialle. Yield Management: Applications to Air Transport and
Other Service Industries. Paris: Les Presses de L'Institute du Transport Aerien,
1994.
4. Detweiler, M., & R. Omanson. Ameritech Web Page User Interface Standards
and Design Guidelines. 1996. Available from http://www.ameritech.com.
5. Donoghue, J. The Future is Now, Air Transport World, April 2001, pp. 78-79.
6. Fogg, B., J. Marshall, L. Othman, A. Osipovich, C. Varma, N. Fang, J. Paul, A.
Rangnekar, J. Shon, P. Swani, & M. Treinen. What Makes Web Sites Credible? A
Report on a Large Quantitative Study, Conference Proceedings of CHI 2001, New
York, 2001, pp. 61-68.
7. Frpkjaer, E., M. Hertzum, & K. Hombaek. Measuring Usability: Are
Effectiveness, Efficiency, and Satisfaction Really Correlated? Proceedings of the
CHI 2000 Conference on Human Factors in Computing Systems, New York, 2000,
pp. 345-352.
8. IBM. IBM Ease of Use: Web Design Guidelines. Available from
http://www.ibm.com/easy.
9. Nielsen, J., & J. Levy. Measuring Usability: Preference vs. Performance,
Communications of the ACM 37(4), 1994, pp. 66-75.
10. Nielsen, J. What is "Usability"? DevHead, 1998. Available from
http://www.zdnet.eom/devhead/stories/articles/0,4413,213767 L00.html.
87


11. "No Frills Flights to Save Business £l.lbn This Year," Ananova, August 26,
2002. Available from http://ananova.com/business/story/sm_657168.html.
12. Palmer, J. W., & D. A. Griffith. An Emerging Model of Web Site Design for
Marketing, Communications of the ACM, March 1998, pp. 44-51.
13. Pu, P., & B, Fairings. "Enriching Buyers' Experiences: The SmartClient
Approach, CHI Letters 2(1), 2000.
14. Rohn, J. Creating Usable E-commerce Sites, Standard View 6(3), 1998, pp.
110-115.
15. Selvidge, P. "How Usable Are the Airline Web Sites, Usability News, 1999.
Available from http://wsupsy.psy.twsu.edu/usability/%20news/volumel_winter/
U sability_airline .htm.
88