- Permanent Link:
- Ethnographically-informed design, development and testing of a mobile media collection and editing application for installation and maintenance of non-linear context-aware prompts in warehouse environments
- Fry, Rachel A.
- Place of Publication:
- Denver, Colo.
- University of Colorado Denver
- Publication Date:
- Master's ( Master of Science)
- Degree Grantor:
- University of Colorado Denver
- Degree Divisions:
- Department of Bioengineering, CU Denver
- Degree Disciplines:
- Source Institution:
- University of Colorado Denver Collections
- Holding Location:
- Auraria Library
- Rights Management:
- Copyright RACHEL A. FRY. Permission granted to University of Colorado Denver to digitize and display this item for non-profit research and educational purposes. Any reuse of this item in excess of fair use or other copyright exemptions requires permission of the copyright holder.
This item has the following downloads:
ETHNOGRAPHICALLY-INFORMED DESIGN, DEVELOPMENT AND TESTING OF A MOBILE MEDIA COLLECTION AND EDITING APPLICATION FOR INSTALLATION AND MAINTENANCE OF NON-LINEAR CONTEXT-AWARE PROMPTS IN WAREHOUSE ENVIRONMENTS
RACHEL A. FRY
B.S., Loyola Marymount University, 2014
A thesis submitted to the Faculty of the Graduate School of the University of Colorado in partial fulfillment of the requirements for the degree of Master of Science Bioengineering Program
This thesis for the Master of Science degree by Rachel A. Pry has been approved for the Department of Bioengineering by
Cathy Bodine, Advisor Cathy Bodine, Chair Levin Sliker Sarel van Vuuren
November 15, 2016
Fry, Rachel A. (M.S., Bioengineering)
Ethnographically-Informed Design, Development and Testing of a Mobile Media Collection and Editing Application For Installation and Maintenance of Non-Linear Context-Aware Prompts in Warehouse Environments
Thesis directed by Associate Professor Cathy Bodine
The third installment of the Rehabilitation Engineering Research Center for Advancing Cognitive Technologies (RERC-ACT III) grant is focused on improving employment opportunities for working-age adults with cognitive disabilities. The combined goal for the three development phases is to build and test an enterprise ready, context-aware, interactive, non-linear system to enable people with cognitive disabilities to competitively perform work tasks in a warehouse setting. A critical component of this system is an application for collecting, editing and maintaining the appropriate warehouse-specific media for the prompting system. An ethnographically-informed design approach was chosen to accommodate the complicated and dynamic nature of warehouses and the need for the application to adapt to users in the context of a warehouse environment. Through intensive fieldwork, information was elicited from four warehouse environments and combined with requirements from key stakeholders to develop a mobile media collection and editing application prototype, which was then tested with five warehouse managers.
The form and content of this abstract are approved. I recommend its publication.
Approved: Cathy Bodine
I dedicate this work to those in my life who have continuously supported my education and academic endeavors- to my grandfather, parents and brothers.
I would like to acknowledge my committee for their continued help and assistance with this project; Dr. Cathy Bodine, Dr. Levin Sliker, and Dr. Sarel van Vuuren. I would also like to acknowledge the RERC-ATACI National Advisory Council for their input on this project, as well as the research funding from the RERC-ATACI grant number H133E09003 from the National Institute on Disability, Independent Living, and Rehabilitation Research, Association for Community Living, Department of Health and Human Services. Last, but certainly not least, I would like to acknowledge the Coleman Institute for Cognitive Disabilities for support and Coleman Fellowship funding.
TABLE OF CONTENTS
1. Introduction.............................................................. 1
1.1 Living with a cognitive disability in the United States.............. 1
1.2 Employment opportunities in the warehousing industry................. 4
1.3 RERC-ACT III: Technology in development................................... 5
1.4 Warehouse automation and enterprise systems.......................... 8
1.5 Current technology for installing customizable prompting systems .... 9
1.6 Requirements engineering and ethnographically-informed system design . 9
1.7 Ethnography research methodologies in requirements engineering .... 13
1.8 Qualitative data analysis............................................ 15
2. Specific Aims............................................................ 18
3. Materials and Methods.................................................... 19
3.1 Ethnographic Study ...................................................... 19
3.2 Use Cases............................................................ 26
3.3 Mobile Application Development ......................................... 27
3.4 Usability Testing........................................................ 27
4. Results.................................................................. 31
4.1 Ethnographic Study ...................................................... 31
4.2 Use Cases............................................................ 67
4.3 Inspired Design...................................................... 67
4.4 Mobile Application Development ......................................... 71
4.5 Usability Testing........................................................ 80
5. Discussion and Future Work............................................... 87
5.1 Ethnographically-informed Design..................................... 87
5.2 Mobile Application Development and Usability Testing..................... 90
5.3 Future Integration with RERC-ACT III technologies.................... 91
6. Conclusion............................................................... 92
A. Warehouse Interview Questions............................................ 98
B. Code Definitions ....................................................... 100
C. Pre-Test Survey......................................................... 103
D. Usability Task Instructions............................................. 107
E. After-Scenario Questionnaire............................................ 112
F. System Usability Score Survey........................................... 114
G. Use Cases............................................................... 116
H. Pre-Test Survey Responses............................................... 135
1.1 Living with a cognitive disability in the United States
Approximately 14 million people between the age of 18 and 64 living in the United States have a cognitive disability . Unfortunately, people with cognitive disabilities face many challenges that affect their quality of life, including poverty , increased health complications , and lower wages . Only 24.2% of working-age adults with cognitive disabilities were employed in 2014, paling in comparison to the 75.4% employment rate for working-age adults without disabilities . Employment continues to be one of the largest obstacles people with cognitive disabilities face, which, if addressed successfully, could potentially mitigate these challenges ,
When discussing employment for adults with cognitive disabilities, it is important to consider both the employment and unemployment rate. The employment rate is defined by inclusion of working-age adults regardless of their employment status, while the unemployment rate only includes working-age adults in the work force and those actively seeking employment . The employment rate metric provides insight to a frustrating component of the labor force that working-age adults with cognitive disabilities encounter; the desire to work but the inability to find a job. This situation can result in what Department of Labor identifies as a discouraged worker, defined as a, person not in the labor force who want and are available for a job and who have looked for work sometime in the past 12 months but who are not currently looking because they believe there are no jobs available or there are none for which they would qualify .
To date, the federal government has spent billions of dollars and passed multiple legislative acts to combat discrimination and improve employment opportunities for people with disabilities [7, 8, 9]. Two notable legislative acts passed were the Rehabilitation Act of 1973 (RA) and the Americans with Disabilities Act of 1990 (ADA). The RA prohibited discrimination on the basis of disability in programs
conducted by federal agencies, in programs receiving federal financial assistance, in federal employment, and in the employment practices of federal contractors. Vocational Rehabilitation (VR) programs for each state were also established under Title 1 of the RA and used government assistance to help operate the programs .
While the RA made great strides in prohibiting discrimination and establishing programs to assist individuals with disabilities, it was not all encompassing for discrimination outside of government institutions and institutions that received funding from the government. The passage of the ADA in 1990 addressed this issue by prohibiting discrimination on the basis of disability in employment, state and local government, public accommodations, commercial facilities, transportation, and telecommunications. The ADA also assisted in ushering in a new paradigm that, disability is a natural and normal part of the human experience that in no way diminishes a persons right to fully participate in all aspects of society, including employment . This is in contrast to the old paradigm that considered individuals with disabilities as defective people that needed to be fixed . The new paradigm in turn supported the four goals outlined in the ADA: equality of opportunity, full participation in decision-making, independent living and economic self-sufficiency .
In addition to the RA and ADA, there were other calculated efforts to specifically address the low employment rate for working-age adults with disabilities . The Ticket to Work and Work Incentives Improvement Act of 1999 (TTW) was passed with the goal to increase earnings for working-age adults with disabilities and in turn lower benefits that cause an increased burden on tax payers . A recent report found minimal impact findings for this program; however, it is speculated that the minimal impact is a result of the program's existence for less than 10 years .
Despite these immense legislative efforts, the employment rate for working-age adults with cognitive disabilities remains nearly a third of the employment rate for working-age adults without a disability . It is also currently debated whether the
employment rate for working-age adults with cognitive disabilities has decreased, but this position is largely based in disagreement over data limitations for the target population [12, 10]. Even more concerning is the combination of the stagnant employment rate, sheltered employment and underemployment in the workforce. Neither sheltered employment or underemployment are accounted for in the employment rate, but is something that greatly affects working-age adults with cognitive disabilities .
Underemployment exists when an employed individual is working less than what qualifies as full-time employment. Working less than full time, frequently referred to as part time, often disqualifies an employee from benefits, such as access to health insurance. Unfortunately, this is a reality for working-age adults with cognitive disabilities, as they are often underemployed and paid lower wages .
Sheltered employment is a form of segregated employment for individuals with disabilities. Kregel et. al.  described it as a wide range of segregated vocational and non-vocational programs for individuals with disabilities, such as sheltered workshops, adult activity centers, work activity centers and day treatment centers. Programs that supported sheltered employment were developed because of the belief that adults with disabilities were not capable of participating successfully in a competitive employment positions . This practice has drawn sharp criticism for its inability to provide meaningful employment outcomes, the isolation of individuals from the community, and the lack of progression to competitive employment . It was not until the landmark Supreme Court case of Olmstead v L.C. in 1999 that supported employment was mandated to a certain degree for people with disabilities seeking employment . The ruling placed emphasis on integration, declaring that individuals with disabilities have a right to employment in environments that are as integrated as possible .
Even with the increased opportunities for competitive employment, people with cognitive disabilities still face many obstacles to employment. There are other factors
that contribute to the inability to find a job, such as the perception that people with cognitive disabilities are under qualified [7, 16]. Other factors include the disability itself, potential loss of federal benefits, lack of education, and employer attitudes . Assistive technology (AT) has been frequently associated with providing a means to overcome these factors through establishing competitive job placement that supports participation and increases a sense of well-being [17, 18, 19].
Fortunately, there are a few industries in the United States that are committed to addressing these obstacles to employment through the dedication of significant resources to hiring and training a working-age adults with cognitive disabilities. Two notable businesses that employ and support many people with cognitive disabilities are Goodwill Industries and PRIDE Industries. Employment in these industries consists of positions in warehouse locations, workshops and training to help prepare employees for the workforce. As of 2013, thirty thousand people with disabilities were employed at Goodwill .
1.2 Employment opportunities in the warehousing industry
One sector of the workforce identified with great opportunity for employment is in the warehousing industry. The Bureau of Labor Statistics reports that 772,000 individuals are employed in the warehousing industry as of 2015. Of those individuals, 55,790 are employed as order fillers. The average earnings for production and nonsupervisory employees as $15.52 an hour, with order fillers earning an average salary of $32,990 per year with benefits.
The order filler position within warehouses is a strong competitive employment opportunity for people with cognitive disabilities and fosters development of a variety of skills. The Warehousing Education and Research Council (WERC) defines the main responsibility of an order filler as filling customer orders and delivering them to the delivery platform. The WERC also lists six job outputs and tasks for a warehouse order filler job, which include: completing timely shipments, maintaining
a safe workplace, ensuring customer satisfaction, maintaining an efficient workplace, completing accurate transactions, and ensuring productivity measurements met.
1.3 RERC-ACT III: Technology in development
The Rehabilitation Engineering Research Center for Advancing Cognitive Technologies (RERC-ACT) was a five-year grant first awarded to University of Colorado of Denver in 2004. In subsequent years, the grant was awarded in 2008 (RERC-ACT II) and most recently in 2014 (RERC-ACT III). The focus of the RERC-ACT III grant is to improve employment opportunities for people with cognitive disabilities. There are three development phases within the grant; (Dl): Effective Configuration and Authoring of Interactive Prompting Templates, (D2): Integrating the Interactive Prompting Platform (IPP) with Enterprise Systems and Generation of Context-aware Linear Prompts, and (D3): Real-time Monitoring of Execution of Job Tasks with Nonlinear Intelligent Contextual Prompts. The overarching goal is to build and test a non-linear, context-aware system to support people with cognitive disabilities in an order filling position in a warehouse setting. Components of the system will include an indoor navigation package coupled with a smart cart, a non-linear context-aware prompting system, and a method to install, configure, and maintain prompts for the system (Figure 1.1).
One major component of this system is the interactive prompting platform (IPP). Given the immense diversity of warehouse products, preinstalled generic pictures and videos for prompts will not be sufficient for the prompting system. Before the system is successfully deployed, warehouse specific pictures, video and audio will need to be collected and edited into the appropriate prompting template format by warehouse management and employees. Warehouse logistics will play a key role in setting up the interactive prompts for the system and maintaining and editing the prompt media. The order-picking method and warehouse layout can help determine the specificity of the media, while activity profiling can identify what prompts will be most active and
the pick lists are further edited by the WMS to generate pick-lines so travel time in the warehouse is efficient. Over half of the order-picking process is spent traveling through the warehouse, while only 25% is spent searching and extracting. 
There are two types of order picking: split-case picking and carton-picking. Split-case picking is considered more labor intensive as units are not contained in large cartons and employees are required to sort through a high density of SKUs (item identifiers). Carton-picking involves employees picking cartons from shelves. This task is simpler than split-case picking, so much so that warehouses with significant capital automate the carton-picking process because rectangular cartons are easy to transport with autonomous machines. 
1.3.2 Warehouse Design: Layouts
Warehouse layouts can be complex and vary significantly. The overall goal for layout design is to optimize efficiency for inbound and outbound processes. Items with high turnover rates are placed in convenient locations dependent upon the receiving and shipping locations in a warehouse. The two most common forms of shipping and receiving configurations are U-flow configuration and flow-through configuration. 
With flow-through configurations, the shipping and receiving docks are on opposite ends of the warehouse creating convenient locations within a direct line between the two stations. With U-flow configuration, shipping and receiving docks are on the same side of the warehouse, creating convenient locations in a triangular pattern outward from the stations. In this model, convenient locations are more convenient than the flow-through configuration model, however the trade off is that inconvenient locations are more difficult to navigate to. 
Layouts are also constructed for pallet reserve, carton pick areas, and individual pick areas. Tasks that do not involve operating machinery begin at the carton pick areas, however it is possible for tall carton shelving units to require fork lifts and lifts to retrieve and replenish items. Cartons on the lower levels can be order picked by
employees with carts if the cartons are not oversized. Individual pick areas, called fast-pick areas, are located in a sub-region of the warehouse. They are often formed to improve response times to customer orders that are in high demand. These areas are general small and do not require extensive travel for pick-orders and are more easily supervised. 
1.3.3 Warehouse Performance: Activity Profiling
To secure a better understanding of where products are located and what pick tasks are completed most often, a warehouse activity profile can be generated. Amongst the statistics gathered in this profile, there are four pertaining to pick tasks: 1) the average number of pick-lines shipped per day and 2) the average number of units per pick-line 3) the average rate of introduction of new products and 4) seasonal products. A common form of activity profiling that collects information on product activity is ABC analysis. This analysis classifies SKUs as A (a small number of SKUs that require most of the activity), B (moderately important SKUs), or C (large number of SKUs but only limited activity). 
1.4 Warehouse automation and enterprise systems
The RERC-ACT III technology in development has the potential to be supplanted by autonomous technology in automated warehouses. Given current advancements and recent efforts to automate and streamline warehouse supply-chain logistics, it may be unclear as to why this system is under development. Often removed from the automation hype is discussion of the significant barriers to warehouse automation. Two of the largest barriers are cost and technology shortfalls, which are often staggering due to the inability to adapt autonomous technology to certain warehouse layouts. In addition, automation does not always mean fully automated. Many warehouses are partially automated and still require order fillers and stockers for day-to-day operations in the warehouse. 
With the goal of building an enterprise ready system, it is important to review
what makes an enterprise system successful and what obstacles exist in the deployment and maintenance of a system. Some considerations for enterprise systems are financial costs and risk, technical issues, IT adoption use and impacts, and integration .
Total Cost of Ownership (TCO) as an import factor to consider when reviewing enterprise systems. TCO is a model that analyzes the total amount of expenses for the life of an IT system, with special focus on the ongoing costs that are incurred to maintain the system. Support costs typically account for 70%-85% of TCO, thus making it an extremely applicable consideration for the enterprise ready, context-aware, interactive, non-linear system. 
1.5 Current technology for installing customizable prompting systems
Technology on the market to setup and customize prompts for certain tasks are small-scale and designed for an individual-specific prompting system. AbleLink Technologies currently sells the Visual Impact 3, an iPhone application that can be used on Apple devices for creating custom prompts. What is unique about Visual Impact 3 is the system behind prompt customization. AbleLink Instructional Media Standards (AIMS) uses an XML based protocol to design and share prompts on multiple devices. Two other customizable prompting programs from AbleLink technology are the PocketCoach and Visual Assistant, but these products have been discontinued. The AIMS platform can only be used on devices that are AIMS ready. This make the prompt authoring process easily to customize and build on one device to transfer to another. Limitations include lack of translation to large-scale work and usability information.
1.6 Requirements engineering and ethnographically-informed system design
Requirements engineering is a subdiscipline of software engineering that involves elicitation, analysis, specification, and validation , It is often cited as one of the
most important steps of the Software Development Life Cycle (SDLC) because poorly executed requirements engineering can result in costly mistakes and failed software systems [25, 26, 27, 28]. To solve this problem, it was postulated that augmenting the requirements elicitation process with social science methodologies would aid in the success of newly developed software [29, 30, 31]. Ethnography is the method that quickly emerged because it enabled developers to explore the environment of the end user , The combination and collaboration between these different scientific fields contributed to the early development of the held now known as Computer Supported Cooperative Work (CSCW) .
Ethnography is traditionally defined as a qualitative method for studying a phenomenon within its natural context . It is a research methodology often used by anthropologists to study communities, cultures, and societies , Individuals who conduct ethnographic studies, known as ethnographers, are guided by four principles: studying phenomena in natural setting, maintaining a holistic view, providing descriptive understanding and taking a members perspective . These four principles establish guidelines that bring ethnographers to real world settings where the activities are experienced firsthand. In observing these activities, it is imperative for the ethnographer to understand them in the larger context, understand them as they occur, and gain an insiders view of a situation rather than projecting personal views .
Given its roots in the social sciences, it may be hard to imagine how ethnography transitioned into the held of requirements engineering. In 1987, Dr. Lucy Such-man, an anthropologist, published seminal work that called for rethinking the design process for developing technology. In Plans and Situated Actions: The Problem of Human Machine Communication, Suchman studied human interactions between office workers and a photocopier using ethnography. Her ethnographic approach promoted, observations that capture as much of the phenomena, and presuppose as little as
possible . Her findings supported her postulation that an individual's actions are intertwined and dependent upon the evolution of an immediate situation, which contradicted the widely held view that an individual's actions are based upon a plan they constructed before the execution of actions. She also described the importance of generating an understanding of an environment, If we build an account of action, or language-use, without simultaneously building an understanding of languages relationship to the world, there is a sense in which we quite literally do not know what we are talking about, which applies to the importance of obtaining access to actual situations instead of generating mock scenarios .
Following the work by Suchman, seminal studies that first combined requirements elicitation with ethnography began in the early 1990s with technology in London Underground line control rooms , system design for air traffic control , technology in an ambulance control center , and design work in the fashion industry . Each of these studies was paramount because they identified and discussed the benefits and problems encountered when combining ethnography with requirements elicitation. While these studies provided the groundwork for combining requirements elicitation methods with ethnography, they lacked a formal process to bridge the gap between the qualitative data generated from ethnographic studies and software development requirements. In addition, the research groups did not develop and deploy a prototype in the studied environment to test the design requirements elicited from the ethnographic fieldwork. This missing component left a lingering question as to whether requirements elicitation from ethnography was beneficial at all.
One of the problems identified by researchers in these studies was adhering to traditional ethnography field time. Traditional ethnographic studies often take months or years in order to gain a comprehensive understanding of the culture and environment. This lengthy timeframe contradicts the time spent in the field for these studies, which ranged from months to days. While longer time in the field translates
to a strengthened understanding, it can be impractical for teams looking to develop technology within a specified timeframe. This altered time constraint engendered the creation of four different types of ethnography in the CSCW held: concurrent, quick and dirty, evaluative, and re-examination . The most relevant type of ethnography related to the issue of time in the held is quick and dirty, as it accompanies short periods of heldwork. Recently, rapid ethnography has emerged as a new type of ethnography . While similar to quick and dirty ethnography, rapid ethnography provides a more rigorous and clearly dehned approach for conducting heldwork .
With the increased use of technology connecting multiple locations, it became clear that small-scale ethnographic studies in a single held location for an extended period of time were too restrictive , This was yet another difference between traditional ethnography, which is often limited to one held site. With the adoption of multiple held sites, ethnography began evolving to follow people and technology to multiple sites and environments . One of the largest areas within CSCW that utilizes multiple held sites is research and development of healthcare technology. This is not surprising considering the continued overwhelming transition from paper to electronic medical records .
With the continuing evolution of ethnography in the held of CSCW, it is easy to become lost amongst the dehnitive practices. What dehnes ethnography is continuing to evolve; however, it is explicitly clear what ethnography in the held of CSCW is not. Ethnography should only inform design, not replace other traditional requirements elicitation methods [35, 39, 42, 43, 38, 32], This criterion is critical because qualitative data is produced from heldwork, and it is widely acknowledged that contextual approaches based on ethnographic techniques do not map well onto current formal specihcations , Ethnography is also concerned with determining cultural practices of communities and would therefore be incomplete in eliciting functional requirements.
Naturally, combining traditional requirements elicitation methods with ethnography in the held of CSCW has drawn criticism since its inception , There are disagreements on what qualifications are necessary for ethnographers conducting held studies, as well as the reduction in the value of ethnography when it is used to simply inform design , In addition, the evolution of ethnography within the held poses questions of what dehnes proper ethnography and how it can specifically inform design. While quantitative studies are still the dominate method in the held, ethnography is widely recognized and utilized by research groups and major global companies, including IBM, Microsoft, Intel, and Xerox [35, 45]. These companies, amongst others, employ anthropologists and ethnographic experts to help inform design for products. It may not be surprising that in 1987, Lucy Suchman's work was performed at the Intelligent Systems Laboratory at Xerox Palo Alto Research Center.
1.7 Ethnography research methodologies in requirements engineering
It is evident that ethnography in the held of CSCW is continuing to evolve as technology advances. As such, methods for combining ethnography and system design are frequently redehned and augmented. The combination of requirements engineering and ethnography is fairly new and many different approaches exist within ethnographically informed research studies.
Research methods from the hrst ethnographically-informed design studies varied in a few aspects. Each of the studies differed in heldwork length and observation recordings. There were also differences in who conducted the ethnographic studies; some studies only used ethnographers while others used computer scientists . The studies lacked thorough descriptions of the qualitative data analysis methods the ethnographers and computer scientists used for the studies. There was also no formal method to bridge the gap between the data produced from the ethnographic study and requirements elicited for design.
Studies with successful development and deployment of a software system in-
formed by ethnography began in 2000, with a majority of research produced by a group in Copenhagen, Denmark . The research group successfully developed and deployed AWARE architecture and platforms with applications AwarePhone and AwareMedia, and a Peri-Operative Coordination and Communication System (PoCCS) in hospitals [46, 47, 41]. While each of these programs and systems were developed with the help of ethnographic methods to inform design, a combination of methods were used to elicit requirements and analyze qualitative data, including activity theory and analysis [48, 46].
Given the variety of approaches and nuances, it may seem unclear as to why ethnography is selected over alternative methods for research studies. Easterbrook et. al.  identifies six different methods for acquiring information and data for software engineering research: controlled experiments, case studies, survey research, ethnographies, action research and mixed-methods. Of these different research methods, case studies are the closest related to ethnographies and often used within the CSCW held. Given the similarities between the methods, it is important to delineate the differences.
Case studies are defined as, an empirical inquiry that investigates a contemporary phenomenon within its real-life context, especially when the boundaries between phenomenon and context are not clearly evident, and require a clearly defined research question that explores how or why a phenomena occurs . In contrast, ethnography can help to understand how technical communities build a culture of practices and common strategies that enables them to perform technical work collab-oratively. Ethnography also requires a clearly defined research question that explores cultural aspects of a community. Herein lies the fundamental difference between case studies and ethnography; one method explores a phenomenon while the other explores cultural aspects. Ethnography fosters an understanding of how individuals work together in the context of the environment.
Once ethnography is defined as one of the methods for requirements elicitation, it follows a clearly defined path of formulating a research objective, devising a strategy for selecting study participants and selecting appropriate research techniques. Four common methods for selecting study participants in ethnographic research are quota, purposive, snowball, and convenience. These sampling methods are nonprobability methods and often used because of research time constraints. Both the quota and purposive strategies are used when a group is specified and screened to ensure participants are appropriate for the sample. While quota uses this method with a set number of participants, purposive does not as it is may not be feasible to set a specific number of participants. Convenience and snowball sampling are both sample as you go strategies and are appropriate for situations where it is unknown who will be able to participate. The snowball sampling strategy relies on word of mouth from previous or initial participants, as they will refer other individuals who may be good candidates for the study. Convenience sampling is used to select people who meet requirements for the fieldwork and are available and willing to participate. 
The final step after establishing the research objective and strategy for selecting study participants is selecting research techniques and approaches for the fieldwork. Participant observations and interviews are two data collection methods commonly used Methods within participant observation include think aloud observations and participant-observer techniques. Think aloud observations encouraged subjects to voice their thought processes when carrying out work tasks, and participant-observer techniques allow the observer to work with participants and become part of the team.
1.8 Qualitative data analysis
One of the most critical processes within ethnographic fieldwork is the analysis of qualitative data. Before delving into different methodologies, it is necessary to define two dominant theories that exist within qualitative data analysis methods; genera-
tion of theory and confirmation of theory. Confirmation of theory is used to confirm a theory that has been pre-identihed by strengthening and providing evidence that supports the theory. In contrast, generation of theory is used to extract a theory that is grounded in the data. Outside influences (including previous research and literature) do not alter or influence the generation of theory, as the data is solely responsible for its development. While many methods exist for analyzing qualitative data, a majority of them fall within the generation of theory framework. Common methods for analyzing qualitative data include grounded theory methods, conversation analysis, distributed cognition and activity analysis. Some of these methods have been used in previous ethnographically informed design studies, as listed in the ethnographic methodologies in requirements engineering section. 
Coding of qualitative data is an analysis method commonly used in requirements engineering projects and studies that include ethnographic fieldwork , A formal method for the coding process is content analysis, which is defined as a method that simplifies data through use of codes and categories to classify the information .
Content analysis begins once qualitative data is collected. Before coding begins, a deductive or inductive approach must be selected to drive subsequent analysis decisions. An inductive approach is taken when there is little former knowledge about the phenomenon under review, while a deductive approach is built upon previous content analysis work and results . The first step in the process is coding the data, which is also comprised of two separate approaches; a priori coding and emergent coding , A priori coding uses predetermined codes from established theories within literature to analyze the data, while emergent coding uses codes that emerge from information within the data.
Coding the data involves looking for specific items, asking questions constantly about the data, and making comparisons at various levels. Questions are asked internally by the coder to develop a better understanding of the data. Making comparisons
of data strengthens the interpretation of the final results by revealing similarities and differences. 
Naturally, the coding process is subject to researcher bias, so it is important to document justifications for the coding schema and validate results. One method frequently used for validation is data source triangulation, where multiple data sources support interpretation of the data. In addition to data source triangulation, intercoder reliability is calculated to determine if the coding is reproducible , For this procedure, objective or subjective coders are given explicit instructions regarding the coding process and code the data with the set coding schema generated by the coder. A sample set of data is coded first to ensure that instructions are understood, after which formal coding of the data from the study begins. This test is not to determine the reliability of the chosen coding schema, but rather to identify the reliability of coding the data itself. The measure of reliability is often calculated using Cohen's Kappa
where Pa is the percentage of cases the coders agree upon and Pc is the percentage of agreed cases when the data is coded by chance. A calculated value of 0.6 and above, out of a max score of 1, suggests high reliability. [52, 53]
2. Specific Aims
Specific Aim 1: Conduct an ethnographic study in warehouse environments and generate use cases with information from key stakeholders to elicit and analyze requirements for a mobile media collection and editing application.
Specific Aim 2: Develop a mobile media collection and editing application prototype.
Specific Aim 3: Conduct usability testing with the mobile media collection and editing application prototype and warehouse employees.
3. Materials and Methods
3.1 Ethnographic Study
The research objective for the ethnographic study was to understand how warehouse employees complete tasks and interact with the warehouse environment. Two research questions were developed to guide this objective:
1. What factors influence task completion by employees in a warehouse?
2. What interactions with technology do employees experience in a warehouse?
Warehouse sites were selected using the convenience sampling method. Given the timeline constraints and the difficulty gaining access to warehouses in the Denver metro area, it was not possible to use an alternative sampling strategy. Site selection began with cold calls and emails to over twenty warehouses in Denver. Of the warehouses contacted, four responded positively and agreed to an initial meeting. The following warehouses were gracious enough to grant access for the study:
1. Engine & Performance Warehouse
2. Denver Public Schools Food & Nutrition Services
3. Oskar Blues Brewery
4. Goodwill Industries eCommerce
Subjects included in the study were managers and employees working in the warehouses, excluding administrative staff. There was interaction with 34 subjects, comprised of nine managers and 25 employees. Table 3.1 shows three employee demographics; gender, primary job function, and status of employment.
The fieldwork for this study included documenting accounts and processes with a camera and audio recorder. Managers and employees who were recorded and/or photographed in the warehouses consented by signing the Colorado Multiple Institutional
Table 3.1: A list of the 34 subjects in the ethnographic study, including subject gender (Male (M) or Female (F)), employment position, and employment status (Full Time (FT) or Part Time (PT)).
Gender Position Full Time (FT) or Part Time (PT)
F Manager FT
M Manager FT
M Manager FT
M Manager FT
M Manager FT
M Manager FT
M Manager FT
M Manager FT
M Manager FT
M Order Filler FT
M Order Filler FT
F Order Filler FT
M Order Filler FT
M Order Filler FT
M Order Filler FT
M Order Filler FT
M Order Filler FT
M Order Filler FT
M Order Filler FT
M Order Filler FT
M Order Filler PT
M Item Lister PT
M Customer Service PT
M Book sorting and shipping PT
M Order Filler PT
M Book sorting and order filler PT
M Item Lister PT
F Order Filler PT
M Item Photographer PT
M Order Filler FT
M Order Filler FT
M Order Filler FT
M Order Filler FT
Review Board (COMIRB) approved consent forms under the study, An Exploratory Investigation of the Impact of the Assistive Technology Partners Product Testing Lab. These forms included a consent packet and a photograph/videotape release form. The forms were read by each subject and reviewed verbally in a quiet section of the warehouse before signing.
3.1.4 Protocol and Methods
The number of site visits varied for each warehouse due to the permission granted for the timing and duration of the visits. Table 3.2 shows the number of visits to each site, as well as the duration of the visit and data collection details. A total of 25 hours was spent in the held.
Before the first visit to each warehouse, interview questions were developed with a research colleague after a literature review of warehouse processes. Sixty questions were developed and divided into six categories; general warehouse questions, technology-related, warehouse-specific, employee-related, training-related and task-related questions. Upon review with research advisors, interview questions were narrowed down and refined into thirty questions using the binning and winnowing approach used by PROMIS investigators , The process of deleting, adding, and refining interview questions was dependent upon redundancy, relevance, specificity, and necessity to elicit more information for the system as a whole. The interview questions can be found in Appendix A.
The first site visits consisted of interviews with managers, introductions to available employees, and warehouse tours. These visits elicited information about the history of the warehouse processes and global versus day-to-day operations. After the initial site visits, subsequent visits were scheduled to shadow warehouse employees and collect information about the warehouse environment. One warehouse presented as an exception to this methodology as it was located in Longmont. At this site,
Table 3.2: Data collection details for the ethnographic fieldwork at Denver Public Schools Food & Nutrition Services and Oskar Blues Brewery.
Warehouse Site Visit Total Hours Information Collection Details
Denver Public Schools Food & Nutrition Services 1 2.75 - Interviewed one warehouse manager to elicit warehouse logistic information - Collected information of employee training and the order filling and stocking process - Reviewed equipment used to fill orders - Collected photos of the warehouse layout with a camera and observations with a held notebook
2 3 - Interviewed a manager to learn about the order delivery process - Shadowed four employees filling orders in the cooler of the warehouse - Shadowed one employee filling orders in the dry goods area of the warehouse - Interviewed manager to learn more in depth information on warehouse logistics history and current technology
3 1 - Interviewed the manager who oversees product inventory - Observed workhow change of pace at the end of the day on a Friday
4 1 - Arrived in the early morning on a Monday to observe the fruit check process for the incoming produce order - Shadowed three employees moving orders in the cooler
Oskar Blues Brewery 1 3 - Interviewed warehouse manager to learn about overall brewery and delivery logistics
- Explored the shipping and cooler areas and collected layout information with a camera and observations with a held notebook
- Shadowed four employees filling orders in the cooler and shipping dock
Table 3.3: Data collection details for the ethnographic fieldwork at Goodwill Industries eCommerce and Engine & Performance Warehouse.
Warehouse Site Visit Total Hours Information Collection Details
Goodwill Industries eCommerce 1 3 - Interviewed two warehouse managers to learn about warehouse logistics - Explored the warehouse collecting extensive layout information with a camera, and observations with a field notebook - Conversed with one employee in the photography area - Shadowed one employee in the listing area
2 3 - Interviewed two employees about daily work tasks - Shadowed one employee filling orders
3 2 - Arrived early before the start of the work day to observe the morning meeting - Interviewed four employees about daily work tasks - Shadowed one employee filling orders
Engine & Performance Warehouse 1 2 - Interviewed two managers with two research colleagues and an associate professor to collect logistics information - Warehouse tour with both managers to see warehouse layout and daily processes in action
2 2.25 - Explored the warehouse collecting extensive layout information with a camera, and observations with a field notebook - Shadowed and spoke with five employees filling orders and stocking. Discussed job responsibilities and general warehouse workflow - Spoke with warehouse manager on the warehouse floor to learn more about warehouse logistics and warehouse history - Filled an order with one employee
3 2 - Explored the warehouse collecting layout information with a camera, and observations with
a field notebook
- Spoke with warehouse manager to elicit more information about warehouse logistics
- Shadowed two employees filling orders and participated in filling orders with one employee
interviews and shadowing were completed in one half-day.
3.1.5 Data Collection
Data was collected with an audio recording device, a camera, and a held notebook. Table 3.2 shows the data collection methods for each site visit. In some situations, employees were nervous in the presence of a recording device. In order to prevent data collection of an altered persona, a held notebook was used to collect information from conversations in lieu of the dictaphone.
3.1.6 Data Analysis
Inductive content analysis was used to analyze the data. The content analysis process was completed in the following phases; preparation, organization and reporting the analysis and results (Figure 3.1).
The organization phase began with uploading all of the photos, videos, held notes, and audio recordings into the qualitative data analysis software NVivo. This software package was chosen because it, supports qualitative and mixed method research, and aids in organization and analysis of unstructured qualitative data to help hnd key insights . Each of the audio recordings was transcribed, and the data was organized by warehouse and visitation date. In total, there were 310 photos, 5.5 hours of interviews and 30 pages of held notes.
Open coding was the hrst step in the organization phase. All of the data for each warehouse was reviewed in NVivo and words and phrases were recorded in the corresponding notes section of NVivo. This initial review of the data included observations and ideas generated from looking through the data. From these notes, headings were generated, and they were then transferred to a coding sheet in Microsoft Word (Appendix B). Categories were generated from the headings and grouped under higher order headings. A conceptual model was then generated to rehect the hnal data abstraction.
Inter-rater reliability was measured with an objective coder. Ten percent of the
warehouse data from all four sites was randomly selected in NVivo for analysis. The
Figure 3.1: The inductive and deductive content analysis approaches, including steps within the preparation, organization and reporting phases. Figure borrowed with permission from .
coding session began with a description of the purpose of inter-rater reliability and how the overall process was structured. The general descriptions of the warehouses listed in the results section were read to the coder and photographs of the warehouse layouts were shown to provide an understanding of each site. The code definitions were reviewed thoroughly, and a sample piece of data was then reviewed with the coder to facilitate understanding of the coding process. Once the process was understood, the objective coder began coding the data using the code definition sheet as a reference. The method used to calculate inter-rater reliability was Cohen's kappa.
3.2 Use Cases
The objective of generating use cases was to outline functional requirements elicited from key stakeholders, which included research advisors overseeing the RERC-ACT III research and development phases and warehouse managers.
3.2.1 Subjects and Consenting
The requirements elicited from the four research advisors were driven by the requirements for the interactive prompting platform (IPP). The IPP was under development concurrently with the mobile application, so requirements gathered included core fundamentals for a non-linear context-aware prompting system. This included layers of prompts that provide specific, additional task completion information to the user in the form of photos, video and audio.
Warehouse managers served as key stakeholders because the end goal is for the mobile application to be used by managers in a warehouse environment. These requirements were for item specifics and functions to prevent disruption of processes in a warehouse. Seven warehouse managers were interviewed for logistics information at the four warehouses. The consenting process for the managers was identical to those listed in the ethnographic study methods. Managers who were recorded for interviews consented by signing the Colorado Multiple Institutional Review Board (COMIRB) approved consent forms under the study, An Exploratory Investigation of the Impact
of the Assistive Technology Partners Product Testing Lab. The consent form were read by each subject and reviewed verbally in a quiet section of the warehouse before signing.
3.2.2 Protocol and Methods
Requirements were elicited from research advisors for the duration of the research project. Information was gathered and discussed during meetings and presentations on research work. Requirements from warehouse managers were elicited from interviews about warehouse logistics and task completion. All of the requirements were compiled into the use case template format outlined in Documenting Use Cases by Geri Schneider, provided by developer Works IBM .
3.3 Mobile Application Development
The objective of mobile application development was to use the elicited functional requirements and inspired design features to build a prototype. The Create-A-Task application prototype was developed in Android Studio. The Android platform was chosen because of its ability to seamlessly integrate with the other Android platform devices in development for the non-linear, context-aware prompting system. Open source libraries were also used to incorporate specific features into the application.
3.4 Usability Testing
The objective of the usability study was to test the functionality of the Create-A-Task application, measure overall user satisfaction and usability metrics, and elicit user feedback to enhance future design iterations.
Five warehouse managers were recruited from the Denver Public Schools Food & Nutrition Services warehouse (DPS).
Usability testing was completed on-site at the DPS warehouse during a district holiday weekday. The district holiday had suspended the normal workflow in the warehouse, providing a quiet and open environment for testing.
Consenting was completed with the Colorado Multiple Institutional Review Board (COMIRB) approved consent forms under the study, An Exploratory Investigation of the Impact of the Assistive Technology Partners Product Testing Lab. These forms included a consent packet, a photograph and videotape release form, and demographic form. The forms were read by each subject and reviewed verbally in a private, quiet section of the warehouse before signing.
3.4.4 Protocol and Methods
A four-tiered plastic shelving unit (36 Wx58Hxl8L) was used for usability testing at the warehouse site (Figure 3.2). Four items were placed on the shelving unit; a small packaging box, a NetGear ProSafe desktop switch box, an empty red bin, and two home gym exercise mats. Handwritten labels were attached to the NetGear Prosafe box, the red bin, and the home gym exercise mats (Figure 3.3). The four tiers of the shelving unit were labeled individually between 101-104.
After consenting, the subject completed a brief survey on daily technology use (Appendix C). The subject was then briefed on the overall structure of the test. This included instructions to think aloud, and to troubleshoot as much as possible before asking for help.
The usability test was comprised of four separate tasks using the Create-A-Task application on a LG Nexus 5 smartphone (Appendix D). After each task, the subject completed an after-scenario questionnaire (ASQ) (Appendix E). At the conclusion of all four tasks, the subject completed a system usability score questionnaire (SUS) assessing the entire Create-A-Task application (Appendix F).
Figure 3.2: The four-tiered shelving unit with four items used for usability testing. The shelves were labeled between 101-104.
3.4.5 Data Collection
Data was collected with the usability test application Lookback and a GoPro camera fixed to the shelving unit. The Lookback application was enabled while users completed tasks and captured all activity on the screen of the phone, as well as video of the users face while completing the task. A limitation of Lookback was complete disconnection from the application when the phone camera was accessed. The GoPro camera was used to collect user information after Lookback was disconnected.
3.4.6 Data Analysis
The Lookback recordings and GoPro videos were reviewed manually and tagged at relevant time points for the time on task and number of errors metrics. The number of errors was split between critical errors and non-critical errors. Critical errors were defined as incorrectly completing a task while thinking it was completed
Figure 3.3: Item labels for the (a) empty bin of closet hooks (b) NetGear ProSafe desktop switch box and (c) exercise mat. The labels included the item name in the center, SKU in the lower left corner, and description in the lower right corner.
correctly, or asking for verbal assistance. Non-critical errors were navigational errors and deviations from the shortest completion path. Non-critical errors did not include typos that were corrected before exiting the text held. The SUS form were scored according to the appropriate scoring method. Even numbered questions were given a score contribution of 5 minus the scale position, and odd numbered questions were given a score contribution of the scale position minus I. The scores of each question were then summed and multiplied by 2.5 to calculate the SUS score. Additional feedback was also documented.
4.1 Ethnographic Study
Each warehouse presented unique information due to the variation amongst products, customer bases, and methods of operation. Each of the four sites will be described, followed by the findings and inspired design.
4.1.1 Site 1: Engine & Performance Warehouse
The first site was a mid-size warehouse in Denver that sold automotive parts. At 60,000 square feet, the warehouse had three separate floors, one loading dock, and one receiving dock (Figure 4.1a). By floor, products were arranged according to the ABC method. Group A corresponded to products pulled and shipped most often and resided on the first floor. Group B products were ordered to a lesser degree and were stored on the second floor, and C products that were rarely ordered were located on the third floor. Amongst the floors, products were arranged by product line, and alphanumerically within each product line (Figure 4.1b).
Figure 4.1: The Engine & Performance warehouse layout, (a) The three floors of the warehouse, (b) Items organized alphanumerically by product lines within an aisle.
The layout and organization of the warehouse was ordered and maintained by one
manager. This manager had been employed at the warehouse since its inception, when
there was only 6,000 square feet of space. The manager was responsible for ordering the 90 product lines stored in the warehouse. While the organization system was set alphanumerically by product line, the manager had to accommodate the unique industry quirk of reissued product lines that were out of order. Some product lines in the warehouse spanned multiple aisles and shelves and it was simply impossible to reorganize to make room for a new part within a product line that fell between the existing alphanumeric structure. As a result, products were placed as closed to the product line as possible and not all products were ordered alphanumerically by product line (Figure 4.2).
Figure 4.2: Items placed on the floor nearest to their corresponding product lines.
For employees, a typical work day began at 8am and ended at 5pm. Transfer orders were completed first upon arrival in the morning. Transfers were large orders sent to one of the 13 Engine & Performance (EP) warehouses in the United States each week, five days a week. The number of employees filling transfer orders gradually tapered off as the morning progressed, with some employees beginning to fill daily orders within a few hours of the start of the shift. Large transfer orders were complete after approximately 5-6 hours, after which all employees began work on filling orders printed out at the pick ticket station (Figure 4.3a). EP developed a custom warehouse management system (WMS) to suit the warehouse needs, and a component of this system was the print ticket station. When an order was received, a pick ticket was automatically printed at the pick ticket station. Employees that were in close proximity to the station frequently checked the pick ticket printer and placed the tickets in a designated basket. When an employee was ready to fill an order, they would remove a pick ticket from the basket, assess the items on the list, grab a cart if necessary, and fill the order. Once the order was filled, the employee returned to the shipping area near the pick ticket station and boxed the order on the shipping conveyer belt (Figure 4.3b). In addition to order fillers, there were two employees responsible for stocking items in the warehouse. These employees were highly knowledgeable of where to locate items, and helped order fillers with questions about where to locate specific items.
The process of filling orders was unique to each order filler in the warehouse. A specific pick path was not designated by management, so employees were free to develop their own method for retrieving items. Given the difficulty with keeping items organized alphanumerically by product line, new hires were paired with veteran employees to learn where items were located. According to the manager, it took approximately three weeks for a new employee to become familiar with the warehouse. If there were any questions about items, employees frequently crossed paths while
Figure 4.3: The start and end points of order picking in the warehouse, (a) The pick ticket station start point with a computer and ticket printer, (b) The shipping area end point where orders were either packaged or placed in bins.
working and questions were often answered briefly in passing. Occasionally, questions were yelled out loud to everyone in the warehouse, as the open space provided an environment for everyone to easily hear each another (Figure 4.4).
Figure 4.4: A view of the warehouse from the third floor.
4.1.2 Site 2: Denver Public Schools Food & Nutrition Services
The second site was a mid-size warehouse in Denver that provided food and nutrition services for the 190+ schools in the Denver Public Schools district. The food and nutrition services included delivery of fresh produce, frozen goods, dry goods, and supplies to kitchens in the district. At 40,000 square feet, the one-story warehouse included a freezer and cooler section separate from a dry goods section (Figure 4.5).
Figure 4.5: Denver Public Schools Food & Nutrition Services warehouse map.
The dry goods area of the warehouse was comprised of six aisles with two-tiered shelving units. Aisles one, two and three were filled with supplies and dry goods that were organized from heaviest and in the largest quantities to the lightest and individual items (Figure 4.6). Heavyweight kitchen cleaning supplies were the first items
Figure 4.6: The start and end points of the dry goods pick path, (a) The beginning of the dry goods pick path with heavy items and cleaning chemicals in aisle one. (b) The end of the dry goods pick path with individual items in aisle three.
in aisle one to prevent leakage into food items if the packaging was damaged. Items picked in the aisles were organized into the ground-level bins. Bins were numbered in descending order, with odd numbers on the left side of the aisle and even numbers on the right side of the aisle. The first number in the bin number corresponded to the aisle it was located in. Bin numbers were divided by even and odd numbers to facilitate a z-pick path. The z-pick path made the order picking process more efficient by preventing order fillers from backtracking between items in an aisle. Aisles four through six stored overstock items in large quantities (Figure 4.7). The second-tier of the shelving units in aisles one through three also stored overstock items, but in smaller quantities.
The cooler and freezer areas were adjacent to the dry goods area. The entrance to the cooler was lined with thick plastic blinds to keep the temperature at 40 F. The cooler was a large, open rectangular room, with two-tiered shelving units lining two opposing walls (Figure 4.8). An automated door served as the entrance to the freezer directly opposite of the entrance to the cooler. The freezer was comprised of two-tiered racks that aligned all of the walls, in addition to a central island of two-tiered racks. This u-shape design facilitated yet another z-pick path before exciting
Figure 4.7: An overstock aisle in the warehouse.
back into the cooler. Items in the cooler and freezer were organized by weight, with heavy and durable items in the first bin and light fragile items in the final bin (Figure 4.9). Once an order was filled, the employee wrapped the order in plastic wrap to secure the items on the pallet, marked the school name in sharpie on all four sides of the order, and placed the pallet on the side of the cooler at the end of the pick path (Figure 4.10).
Figure 4.8: A corner of the cooler with items for picking stacked on pallets and overstock items on the second-tier of the shelving racks.
A typical day for an employee began at either Gam or Sam. Employees arriving
at Gam were responsible for the quality check of incoming produce for the day, as
well as stacking items in the cooler to prepare for the daily cooler pick orders. Once
the cooler items were checked and placed in the correct layout in the cooler, the
order picking commenced. Employees attended a morning meeting and were given
productivity sheets to complete as they filled orders. The goal was to complete all of
the cooler picking by l:30-2:00pm. Employees in the cooler selected pick tickets from
a cubby unit located in the cooler, while employees in the dry goods area selected pick
Figure 4.9: The start and end points of the cooler pick path, (a) Items in boxes at the beginning of cooler pick path, (b) Fragile produce at the end of cooler pick path.
Figure 4.10: Completed orders stored in the cooler.
Figure 4.11: Pick ticket cubbies, (a) Dry goods pick ticket area, (b) Cooler pick ticket area.
tickets in a cubby at the start of the first aisle (Figure 4.11). There were typically five employees order picking in the cooler and one or two employees picking in the dry goods area.
Once cooler order picking was complete, employees who arrived at 6:00am were close to leaving work for the day, but they still assisted with the end-of-the-day tasks if cooler picking was finished early. The end-of-the-day tasks included collecting all of the trash, sweeping to collect all of the debris on the floor, and reorganizing items that remained in the cooler and freezer. Once clean up tasks were complete, items for the following day were restocked.
4.1.3 Site 3: Goodwill eCommerce
The third site was a 100,000 square foot warehouse that housed two separate programs; Goodwill eCommerce and Goodwill Good Electronics (Figure 4.12). The focus of the site visits was in the eCommerce section of the warehouse. The eCommerce department was established at Goodwill after recognizing the value of certain items that were donated to the organization. The warehouse serves as a corporate office and donation center, receiving items from 28 stores in the state of Colorado. When employees at Goodwill stores and donation centers sort through donated items,
anything that looks like it may be of greater value is placed aside and delivered to
Figure 4.12: The Goodwill warehouse space subdivided into two sections, (a) The Good Electronics section of the warehouse, (b) The eCommerce section of the warehouse.
the eCommerce warehouse. Once items are received at the eCommerce warehouse, they are sorted and either returned to the stores if they are of low value, or prepared for online listing.
The eCommerce section is comprised of eleven hubs in the open space; a receiving dock for bins of items from stores, a sorting area, a specialized listing and research area, a general listing area, a photography area, a jewelry sorting area, a media and book storage area, a book sorting area, a shipping area, an item storage area, and a staging area for items picked up in person (Figure 4.13). Items arrive at the receiving dock in large bins separated between books and all other items. The large bins of books, referred to as melons, are placed near the book-sorter conveyer belt where they are sorted using a barcode scanning system (Figure 4.14). All other bins are placed near the listing area and reviewed by employees who list items. If an item appears to be unique, the specialized listers in the research area research the item and determine its value. Items that do not have significant value are reviewed by the general item listers who are also responsible for listing items online. Once items have been flagged for listing, they are placed near the photography station where photographers take high quality photos of individual items. After photos are taken,
Figure 4.13: Four of the eleven separate hubs within the eCommerce warehouse, (a) Book and item storage racks, (b) Item listing area, (c) Photography station, (d) Staging area for item pick up.
the items are placed on a cart with the SD card and the lister lists the items online with a description and the corresponding photos.
A typical workday for an employee begins at 7am and ends between llam-5pm because many employees are part-time workers. While there were designated tasks in the hubs within the eCommerce area, it was rare for an employee to complete only a certain set of tasks in the warehouse. Employees moved around to complete different tasks depending on the level of activity in the warehouse. Book sorters often assisted with filling orders and listers helped with filling orders and photography.
Figure 4.14: The book sorting area in the warehouse, (a) Book sorting conveyor belt, (b) Bins of unsorted books.
4.1.4 Site 4: Oskar Blues Brewery
The fourth site was a 100,000 square foot brewery in Longmont that housed equipment for brewing, packaging, and shipping beer. The focus of this work was in the shipping and storage area of the brewery. The shipping area was a large, open cooler that kept the product at a low temperature to prevent flavor alterations (Figure 4.15a). Pallets with a variety of product lines were placed in the corner of the cooler. The shipping area was adjacent to another, much larger cooler that stored kegs and pallets (Figure 4.15b).
Figure 4.15: Coolers in the shipping dock of the warehouse, (a) Shipping area cooler, (b) Storage cooler adjacent to the shipping area cooler.
A typical work day began between 7am-10am, with staggered 8 hour shifts (7am-3pm, 8am-4pm, 9am-5pm, 10am-6pm) to ensure the shipping dock was covered between 7am and 6pm. Identical paper print tickets were distributed to employees at the beginning of the morning shift and large orders were filled by the group onto delivery trucks. All of the equipment used to complete order filling was powered; every employee used a forklift to fill orders, with the exception of one employee who was responsible for building pallets. Building pallets was a process of dismantling pallets of a single product and combining individual packs on separate pallets with different products. This process was required for certain orders because not all orders
Figure 4.16: Pallets built in the shipping dock are loaded onto trucks, (a) Dismantled pallets in the cooler for building orders, (b) Pallet loading instructions for load distribution requirements on the trucks.
required the exact quantity of a product line on one pallet (Figure 4.16a). Completed orders were wrapped with an automated wrapping machine and loaded onto the delivery trucks in a specific order (Figure 4.16b). The order pallets were loaded into the trucks was extremely important for weight distribution and max load limits. Truck drivers also had multiple orders on a single truck, and the orders needed to correspond to their delivery route.
4.1.5 The Findings
At the most basic level, warehouses serve as an intermediary between inbound and outbound items. While receiving and shipping processes exist in all warehouses to facilitate the flow of items, the processes between entry and departure vary drastically. Naturally, the variation is dependent upon the industry a warehouse is associated with; processes within a furniture warehouse will vary from processes in a grocery chain warehouse. Although, great differences do exist amongst warehouses within the same industry given the wide variety of decisions pertaining to logistics, storage structure, and workflow tasks.
Figure 4.17 shows the data abstraction from inductive content analysis. There
are fourteen subcategories, four main categories, and one data abstraction descrip-
tion. This abstraction identifies a description of the research topic and relates the analysis back to the research questions, and is summarized as factors that affect the implementation of technology in warehouses.
Figure 4.17: A data abstraction diagram of data collected from the ethnographic study. There are 4 main categories, 14 sub categories, and 8 further sub categories. Factors that may affect the implementation of new technology in warehouse environments is the overarching description of the categories that emerged from the data.
The environment of a warehouse was defined as everything enclosed within the walls of the warehouse, both physical and abstract. This category emerged from the different occurrences that drove task completion in the four warehouses. Similarities and differences between the environments highlighted unique forms of communication and varying equipment, layouts, temperature, and technology.
184.108.40.206 Environment: Communication
The completion of daily tasks was never a solo endeavor; it required communication with others and the environment. At DPS and Goodwill, communication began
before the start of the morning work shifts. Managers met with employees to either discuss daily work tasks, present updates on logistics changes in the warehouse, and pass out required information for task completion. One site provided more of an update on happenings in the warehouse, as well as some humorous trivia before work officially commenced:
On that note, today's leading celebrity birthday is Chick Norris. John Hamm from Mad Men is 45. Sharon Stone is 58. Quick funny story, about a year into eCommerce, I found an AARP Magazine and the cover girl was Sharon Stone. That's the moment I knew I was old.
The humor ended the morning meeting on a lighter note before work began, as almost every employee left the meeting chuckling and discussing Sharon Stone and the other celebrity birthdays. Even though the meeting was work and task-related, the manager made an effort to include trivia to lighten the mood and improve the commencement of work for the day.
This is in contrast to the other sites with no designated morning meetings. Employees arrived and knew the order filling tasks that had to be completed for the day, and began work on those tasks. However, at Oskar Blues, once the work shift began, it was heavily dependent upon communication. One employee said yelling was a necessity to communicate to others as to what order was being filled. While the employee stated this method of communication was not very efficient, he said it was necessary to avoid duplicating orders and accommodate the addition of last-minute items to the orders from the sales team. The process of filling orders at this site required building pallets and loading them onto freight trucks in a specific order as to evenly distribute the load. The employees filling the orders used forklifts and drove around quickly to retrieve pallets, rarely stopping to discuss who filled what but instead yelling it out in real-time.
Communication with others to complete work tasks was also dependent upon the employee's familiarity with the warehouse environment and task completion processes. One manager described the onboarding process as:
First you're just building stuff. 4 different types of ways we package our beers. Once you're comfortable with that and the variation of brands we have, we have 9 brands, 50 something... once you're comfortable with that, youre more of an assistant, youre not on the forklift, youre building pallets and helping them out, shouting at the forklift drivers. Once they get past that stage, youre pretty much on your own. I dont micromanage, because I dont have to. Its really simple once you get your head around what everything is.
As familiarity grows, comfort with task completion grows and results in reduced reliance on asking others for help. The level of familiarity is largely dependent upon the structure of the warehouse and how often processes change. At EP, a manager stated it took approximately three weeks for an employee to become comfortable with locating items in the warehouse. There was an increased level of task related communication at this site due to the need to find items. Even employees who worked in the warehouse for over ten years occasionally forgot where an item was located, and had to ask others where it was located.
While some employees were responsible for filling single orders individually, they occasionally communicated to others the need for physical assistance to complete a task. At one site, as an employee was filling an order, she came across a large item she was unable to pull from the shelf, This is what I mean by a bigger item. There are items this big that are really heavy. Thats what Amanda is for. She also described one order as a double header, and changed the manner in which she completed the order filling process:
Researcher: What's a double header first?
Employee: Two items on a sheet instead of one. I usually do this. One... make sure that both items are there. So the shipper will know that this is two items. I just like the shipper to know exactly whats going on. I try to be very exact with what I do.
While this form of communication was not directly with the employee, her physical method was still a way to easily communicate with the shipping employees.
Communication with the environment was just as critical as communication with others to complete work tasks. Communication with the environment included directional signs (Figure 4.18a), informative signs (Figure 4.18c), and visual cues to guide task completion (Figure 4.18b). Directional signs were dependent upon how the warehouse was organized, although at one site, they were not always accurate in regard to where items were located. Signs were both handwritten and computer generated and organized by the abbreviation of product lines or letters and numbers to reflect bin and shelving locations (Figure 4.18d).
Informative signs were signage that included instructions on how to complete tasks, what to avoid, and current updates for the day. Visual cues were not written signage, but quick reminders for employees during task completion. For example, at DPS, one delivery truck's height limit was much lower than the other delivery trucks in the fleet. The max height limit was marked via a black sharpie on the shelf unit where orders for that route were stored (Figure 4.19a). There was a fail-safe if employees were not reminded by the visual cue, which presented as a written notice pointing to another black line denoting the truck height limit before the cooler exit (Figure 4.19b).
220.127.116.11 Environment: Equipment
Work tasks in each warehouse required interfacing with some form of equipment.
Orders were filled using carts, pallet jacks, and forklifts (Figure 4.20). The type of
Figure 4.18: Communication with the environment included a mixture of directional signs, informative signs, and visual cues, (a) Directional signs at Goodwill eCom-merce. (b) Pallet loading instructions at Oskar Blues Brewery, (c) Bin numbers at DPS. (d) Plandwritten labels at Engine & Performance warehouse.
Figure 4.19: Visual cues require knowledge of warehouse task completion to be an effective form of communication, (a) Black markings for the height limit of orders for a specific truck route at the DPS warehouse, (b) An informative sign at the exit of the cooler in case the visual cue was missed.
equipment used changed throughout the day depending upon the work tasks. At DPS, other tasks outside of order filling, like restocking and organization at the end of the day, required forklifts to move larger pallets. No certification was required for manual equipment, but certification was required for forklift use. The level of power within the equipment also increased the necessity of heightened awareness of immediate surroundings. Employees that used forklifts and large pallet jacks were constantly surveying their surroundings and beeping the equipment horns to notify others of the equipment presence. Equipment certification was a limiting factor for task completion for some employees because those who were not certified were unable to use the forklifts to complete tasks.
18.104.22.168 Environment: Layout
Task completion was greatly influenced by the layout of the warehouse. The layout pertains to how and where units that house products are arranged, as well as the divides between different areas within the warehouse. At DPS, the layout was conducive to a specific pick path to enhance productivity. The aisles provided a z-pick path and the horseshoe shape layout provided a way to easily circle around items (Figure 4.21a). At EP, item storage on three separate floors required employees to complete additional steps for task completion. This included pulling items on separate floors and sending them down slides to different levels (Figure 4.21b). The layout at Goodwill was the most open with the absence of physical divides between the hubs on the floor (Figure 4.2Id). This layout made it much easier for employees to identify what work tasks needed to be completed. This is similar to Oskar Blues Brewery, where no divides existed in the shipping area and cooler (Figure 4.21c). The layouts at all four sites remained fixed; however, at DPS and Oskar Blues, layouts recently underwent a transformation to improve productivity and streamline processes.
22.214.171.124 Environment: Technology
Technology in the warehouse environment was defined as the infrastructure that
supported and maintained warehouse operations and included devices that integrated
Figure 4.20: Manual and powered equipment at DPS, Engine & Performance warehouse, Oskar Blues Brewery, and Goodwill eConnnerce. (a) An electric pallet jack at DPS. (b) Push carts at Engine & Performance warehouse, (c) A forklift at Oskar Blues Brewery, (d) A push cart at Goodwill eConnnerce.
Figure 4.21: Warehouse layouts at DPS, Engine & Performance warehouse, Oskar Blues Brewery, and Goodwill eCommerce. (a) Beginning of z-pick path in the dry goods section at DPS. (b) Third floor slide at Engine & Performance warehouse, (c) Open cooler at Oskar Blues Brewery, (d) Open space organized into hubs at Goodwill eConnnerce.
with the warehouse management system (WMS). Employees at all four sites performed order filling and restocking tasks with paper picking and restocking sheets. The absence of barcode scanners required increased attention to detail, as there was no automated feedback to check if tasks were performed correctly. Employees at DPS, EP, and Oskar Blues Brewery rarely interacted with technology while completing work tasks. The exception was the inventory specialist at EP, who was responsible for inputting all incoming products into the WMS on a computer (Figure 4.22).
Goodwill eCommerce was the only site where employees interacted directly with technology on a daily basis. ShopGoodwill was the eConnnerce marketplace where items in the warehouse were listed by a subset of employees. Before listing an item online, employees performed item research to determine its value. The process of researching and listing an item involved a desktop computer. For these employees, technology was an integral part of their job responsibilities.
Figure 4.22: The inventory area at Engine & Performance warehouse with a computer connected to the warehouse management system.
Warehouse processes are constrained by time. As one manager described, picking are the only direct hours. Thats what is generating the income. Anything else is indirect hours and it is lost time pretty much. Inefficient processes, especially within task completion, waste significant amounts of time and can negatively affect the accuracy and timely filling of orders. To ensure time is spent in the most efficient way possible, managers either implement quotas, set schedules, or define task completion steps for employees to follow. Quotas and schedules often set the tempo in the warehouse, with the exception of outside events that may alter the internal workflow.
126.96.36.199 Time: Quotas
Quotas were a means for management to ensure time was spent efficiently by employees and to enhance work task productivity. There was variation in the degree to which quotas were enforced, with one manager describing the quotas at the DPS warehouse:
Researcher: Is there a productivity threshold?
Manager: Yeah. So that's on the productivity sheet. So when we pick in the freezer and cooler, we want them picking at 1,500 pieces an hour. Freezer by itself, on Friday, 100 cases an hour. When we do the dry, its a 104 cases per hour. When we're just doing the cooler, we do the cooler on it's own, or separate on Fridays, they need to be 1990.
These fixed quotas were closely tracked and required to be met by all employees. If quotas were not fulfilled, employees were audited by a manager, which entailed shadowing the order filler during a work shift. This is in contrast to lenient quotas with different task completion requirements. One employee at Goodwill describes:
Researcher: Is there a quota they want you to reach each day?
Employee: Usually there is, although Im glad we have some leniency with that because if you dont have the items in the system, I have to go and photograph things and then come back and list. They are aware of that.
The different task completion requirements was key for the degree of leniency for quotas, as another employee at the same site but with a different work task described certain quotas as unobtainable:
What their goal is, or what Im told, is they want us to scan 21,000 books a day. And out of that, that will give us maybe 5-7,000 acceptable books.
The problem with that, is wed need three more slides to get 21,000. On a good day, we only get 8-9,000. For some reason they thought we could get more through here.
The two warehouses without quotas were driven by the tasks that needed to be completed by the end of the work day. With the absence of quotas, there was no way to track individual employee contributions, which could potentially enhance training if an employee needed additional help or was not contributing equally.
188.8.131.52 Time: Tempo
Tempo defined the pace of workflow in the warehouse. It was either seasonal, monthly or daily depending upon the industry. Goodwill's eCommerce tempo was dependent upon textbook season when the academic year began, the holidays, and spring cleaning. The tempo during these seasonal rushes was extremely fast, with employees quickly sorting through books and items daily, and pulling items for shipping. EP also had a seasonal tempo driven by annual national car shows customers prepared for. The tempo at DPS was dependent on the school calendar. The busiest time of the year was when school began in the fall:
It's good, it's fun, it gets a little hectic and high tension sometimes. The beginning of the school year is bad. I wouldn't say work environment bad, but we're getting hit so hard with orders in the beginning because the schools are starting to restock and everything is just huge. It's quite, head down, and we just want to go home, for about 2-3 months.
This is the complete opposite of the tempo described during summer break, when tasks are much more relaxed and housekeeping and light reorganization happens in the warehouse.
184.108.40.206 Time: Schedules
Schedules provided the closest resemblance to routine as they were a guideline of when tasks should be completed. What differentiates schedules from tempo is the calculated and predetermined time frames that bind a schedule. While tempo was more dependent upon external forces influencing a warehouse, schedules were the time frames set by managers for task completion.
At DPS and Oskar Blues, the morning work schedules were solely order filling related, and the afternoon was focused on clean up reorganization. Order picking in the cooler at DPS was scheduled for completion between l:30-2:00pm given the number of employees and picks per employee that was calculated at the start of each day. At Oskar Blues, the schedule was to complete large truck orders by 12pm. After the morning schedule was complete at both sites, the afternoon work tasks were able to commence. Employees at EP had a similar schedule, but it deviated after morning task completion. EP order fillers completed large transfer orders in the morning, however, the afternoon schedule was to complete additional orders instead of clean up and restocking. One manager stated afternoons in the warehouse at EP is hectic with employees running around to complete remaining orders for the day.
The Goodwill eCommerce schedule was centered around employee work shifts and
completing quotas. The schedule was dependent upon what work was available that
day. Outside of the scheduled morning meeting, schedules were not dominant and task completion was driven by quotas and the overall tempo within the warehouse that day.
Organization for task completion was dependent upon the continued presence or absence of mandates and environment alterations from managers. This category drew inspiration from the concept of a conservative force, where the work measured is independent of the path taken between two points (Figure 4.23). Certain tasks, especially those related to order filling, began at a start point of a pick ticket and concluded with a completed order. Theoretically, in every situation, there was a progression of steps an employee could take to complete the order filling process in the most efficient way possible. The structure of the steps completed from start to finish were controlled by management who either required step-by-step or environmental methods for filling orders, or left the methods for the employees to decide. The work of filling an order was always the same and independent of the means used to complete the task, no matter how efficient the process was. Organization within a warehouse is targeted at these means to enhance productivity as much as possible.
220.127.116.11 Organization: Restructuring
Restructuring organization consists of processes that undergo alterations from
management as a consequence of need and desire. Optimizing task completion in a warehouse is often a necessity to meet performance standards set managers or supervisors.
DPS underwent intensive restructuring over five years ago when a manger with previous experience in highly automated warehouses was hired. The manager described the state of the order picking process as having, no rhyme or reason... it was not very organized, with daily task completion described as, probably only 2 of the eight hours were spent picking for each of them. Everything else was setting up and putting away.
Figure 4.23: A schematic of the variety of possible paths to fill an order. The most efficient path with no errors is denoted with the blue arrow, and a less efficient path with some errors is denoted with the red arrow. Regardless of the path taken to fill an order, the end result is always a completed order.
Alterations to the environment included the addition of racking and assigning every item a location for a pick path. The dry goods section of the warehouse was restructured to a z-pick path with items organized according to weight and size to avoid reorganizing an order after it was picked. In addition to environmental changes, the manager implemented productivity sheets to track the effectiveness of restructuring decisions and provide insight for ways to improve productivity:
Productivity sheets track not only the direct picking, but also the indirect hours that are spent, trying to get everything ready before you can start picking. Picking are the only direct hours. Thats what is generating the income. Anything else is indirect hours and it is lost time pretty much.
These sheets were carried on a clipboard by every employee daily and required documentation of when picking for an order began and concluded. Activities, such as restocking items or searching for items, was also documented for totaling indirect hours. After productivity sheets were adopted, the manager noticed restocking accounted for a large amount of lost time because it was completed while employees were filling orders, instead of in advance of filling daily orders:
They could just pick, take as long as they want to pick, as long as they finished it by the end of their shift, they didn't have any standards for anything. There was really no organization in how they picked... if you needed something in a pick bin, and it was back here in a reserve bin, they would have to just drive up and down the aisles to find the item to put it into the pick bin. And it was right in the middle of the picking process because the pickers just went and got whatever they needed as they were picking it. So there was no standards and there was no organizations, within the department.
As a result, the manager implemented stocking sheets:
The next thing I put in place was the stocking sheets. Which allows us to be able to stock before we start picking, so once the picker starts picking there should be no reason that the picker has to stop in the middle.
The reorganization of the layout, productivity sheets, and stocking sheets all served as major vehicles to restructure the organization of the warehouse. The decision to alter the layout of the warehouse was similar to the restructuring that occurred at Oskar Blues, and for similar reasons. As the manager at DPS described, employees were required to set up and take down items for picking each day before the processes were restructured. The manager at Oskar Blues describes the transition:
In the past, it [the beer] laid out over the floor and it'd be sweating. The cooler was so small we had to pull everything out just to get to items that were behind it and it would stay out for 10 hours sweating. We can now store 3 times as much. Rotating stock was a full 2-3 hours out of someones shift!
The reorganization at both sites significantly reduced the amount of time, wasted, or the amount of time not filling orders. However, even though both warehouses underwent restructuring, the individual processes were either conformed to a designated structure, or remained largely unstructured.
18.104.22.168 Organization: Structured
Structured organization is a required set of steps that must be completed by employees when completing tasks. In essence, this is the right way for employees to complete work tasks. A structured organization was often a direct result of past restructuring, as evident at the DPS warehouse with the z-pick path and horseshoe layout. The Oksar Blues Brewery shipping area, while it underwent restructuring to alter work tasks, did not change the process of filling orders on forklifts. The freedom
to fill orders how the employee sees fit is similar to the unstructured approach at EP, which will be discussed in the next section.
The Goodwill eCommerce warehouse was a site that had a structured organization, but it was not a result of recent restructuring. The structure of task completion was mandated by managers to ensure items were listed correctly and for the correct price. The flow of items rarely deviated from the following workflow; receiving, research or listing, photography, storage, retrieval after purchase, and shipping. Even though employees at the warehouse participated in more than one work task, there was always an order to completing each task. For example, the photographers in the eCommerce area were required to take pictures of items a certain way. Four pictures were needed per item, one on each side, with the appropriate lighting and highlighting of important information on the item. Photo quality was important because items with poor quality photos resulted in increased questions to customer service or potential disinterest in an item.
22.214.171.124 Organization: Unstructured
Unstructured organization enables employees full power to decide how they complete tasks between receiving a pick ticket and completing an order and the environment is conducive to the ability to select a variety of methods. Management at DPS and Goodwill eCommerce provided specific rules for task completion, while management at EP and Oskar Blues both gave employees freedom to choose task completion methods.
The unstructured organization at EP was evident once an employee selected a pick sheet from the pick ticket station. The items listed were in no specific order, so the employee had to determine if the items were split by floor. According to the manager, a majority of the pick tickets were for engine kits with components stored on the first floor of the warehouse. Two employees shadowed at this warehouse had been employed for approximately two weeks, and they already had vastly different
methods and pace for locating items. One employee constructed his own cheat sheet by listing product lines that resided on each level. He already had constructed a set pick path for himself to fill engine kits, and was completing orders at a fast rate. The other employee felt much less comfortable with the warehouse, and was often asking where items were located, and had not developed a set pick path for filling common orders. This contrast demonstrates the striking differences in completing work tasks and how they are dependent upon the organizational structure.
The organization at Oskar Blues was similar to that of EP, but affected work tasks in a different manner. Employees filled large orders together onto freight trucks and the structure of filling components of the order was dependent upon what others had completed. While employees were given the freedom to designate how orders were filled from management, that autonomy was not present amongst the order fillers themselves. Conforming the unstructured organization to the group objective limited the options each employee had for retrieving and loading items.
Perception was how employees related to their jobs and task completion, specifically with personal feelings. This category was not immediately recognized in the data, but emerged after employees continued to provide honest opinions about work.
126.96.36.199 Perception: Positive
A mixture of employees and managers held highly positive perceptions. One manager at DPS had a positive perception despite the difficulties he may face day-to-day:
There is so much, especially with the customer service part because there are so many people that we have to deal with, that I have to deal with, because, my drivers are the face of the company. And I myself, the face of the company. I have to deal with secretaries, principles, custodians, area suprivisors here, area suprivisors at our service building, suprivisors for
our overall custodians, administrative directors, executive directors, it's just, there is a lot of people that you come in contact with and sometimes that can be difficult because people tend to be difficult. I'm a pretty hands on type of person and it's a challenge, and I look forward to the challenge. It makes it exciting to work here. Sometimes you have to go above and beyond to help people out, you know, office staff here, we get a lot of last minute requests for things and to try to keep our customers pleased with the service we're providing.
His perception also influenced his work tasks, as he was often the go-to person for training employees, I try to do training with the order selecting because Im patient with people.
At Goodwill, an employee enjoyed his work tasks for a different reason outside of challenges, I like the customer service desk because its a little bit different everyday. I like the different questions.
188.8.131.52 Perception: Negative
Negative perceptions often arose from intensive restructuring that altered task completion, especially if additional work was added to tasks. The restructuring at the DPS warehouse received a lot of pushback from employees when it was implemented: It was easy to catch on [to the changes]... but with the veterans... anywhere you go, change is hard. They are set in their ways. It wasn't received well at the time, but [the manager] kind of had me forefront the campaign and say its good. They'd come to me and I'd have to explain it'll be better, it'll be easier, but they just didn't believe me. Change is not easy.
The restructuring process not only brought about change, but added additional work employees were required to fulfill while picking orders. These productivity sheets were not well received:
So, again it was like, every time you did an order you'd have to stop and fill it out and everyone thought it was pointless and why are we doing this... it was bad for a while. But it works!
This is in contrast to Oskar Blues, where employees were elated with the time saved filling orders after the restructuring process. One employee said it was less stressful and much more straightforward.
184.108.40.206 Perception: Neutral
A neutral perception was indifference to the job and associated tasks. With employees at all sites expressing both the positive and negative aspects of their jobs, there were some with the attitude of, this job is just a job. Employees with a neutral perception completed work tasks on time and correctly, but lacked the positivity that was evident with other managers and employees. One employees neutral perception was his feelings towards the lack of upward mobility where he was working. Employed as an item lister for two years, he felt there was no opportunity to progress forward, but still completed his work tasks on time and correctly. The lack of upward mobility was not a negative perception either, as it was simply accepted by this employee. A negative perception in regard to lack of upward mobility would likely result in poor task completion, or leaving the job entirely.
4.1.6 Inter-rater reliability
Inter-rater reliability measured the level of agreement between two coders that coded the data with the previously discussed coding scheme and findings. The interrater reliability was in the range of slight agreement, with a kappa value of 0.14. The percent agreement and disagreement was 91.7% and 8.26% respectively. The percentage of the subjective coder coding a source that the objective coder did not code was 7%, and the percentage of the objective coder coding a source that the subjective coder did not code was 1.1%. The percent chance of randomly coding the data was 90.4%.
4.2 Use Cases
Overall System Description: Create-A-Task Application
The Create-A-Task mobile application (CTA) will collect, edit and store media for a non-linear context aware prompting system in warehouse environments. Warehouse managers and employees will use the application to assist in the installment and maintenance of the system to enable warehouse employees to perform work tasks.
Figure 4.24: The overall system use case for the Create-A-Task application. The system has seven main use cases; Login, Create Prompt, Prompt Library, Review Dashboard, Delete Prompt, Edit Prompt, and Update Status. The actors are either an employee or manager. The system use cases interact with the non-linear context-aware prompting system.
Figure 4.24 shows the overall system use case for the application. The seven use cases and 14 subordinate use cases are listed in Appendix G.
4.3 Inspired Design
The mobile application design incorporated the 21 developed use cases (Figure 4.24) and ethnographic study data. The use cases provided guidance solely for the functionality of the application, but there remained design decisions for how the application would present with mobile features. The feature decisions were grounded in the ethnographic findings, with a driving design consideration that the application should emulate common mobile application features found on smartphones. The
Figure 4.25: A list of the five application features informed by the ethnographic fieldwork. Each feature was inspired by the specific findings listed below the features.
categories from data abstraction were grouped together to form descriptions of each feature, which were then transformed into a physical feature (Figure 4.25).
4.3.0. 1 Feature: Unique login and password for each user
As previously discussed, driving factors for warehouse success and sustainability are filling orders accurately and shipping orders on time. Consequences for a warehouse that is unable to fulfill these two requirements may be diminished profits, the loss of customers, high employee turnover, and a host of other negative issues. The most notable characteristic for each employee was individual perception for work tasks and warehouse employment as a whole. Relating perception to a feature required revisiting the purpose of the mobile application development, which was to provide a means for managers and employees to install and maintain prompt media for employees with cognitive disabilities. If the prompt media in the system is incorrect, incomplete, or otherwise ineffective for the end user with a cognitive disability, the prompting system would contribute to violations of the two core requirements of a warehouse. Employees with cognitive disabilities using the prompting system require the correct media and information, or they may fill the order incorrectly and spend excessive amounts of time calling for assistance.
Managers and employees who view additional work as pointless and irritating, or who would likely use the application incorrectly, may not be the most effective users of the application. Employees and managers with a positive perception about work tasks may be more motivated to use application and should have access to the system as well as feedback regarding the quality of the item information and prompts collected. A login in feature will provide a means of inclusion and feedback for employees with positive perceptions, and exclude employees with negative perceptions who may not use the system correctly. The user interface of the feature mimicked traditional login screens found on mobile devices that require text input for a username and password.
4.3.0. 2 Feature: Fast alteration options in the review dashboard
The review dashboard function was outlined in the system level use case; however, the manner in which the review dashboard would display and update errors was undefined. If the organization of the warehouse is unstructured or undergoing restructuring, there are bound to be mistakes with order filling which require the use of the review dashboard. Given the potential for a variety of the total number of errors, as well as the urgency in which errors need to be resolved, a fast alteration feature was incorporated into the review dashboard. The fastest option would allow users to edit prompts and items directly in the review dashboard by reducing navigation time. A swipe to delete feature was implemented in the review dashboard to enable a fast alterations. It addition to quickly altering prompt and item errors, the review dashboard alteration feature allows a user to quickly resolve the status of an item anywhere in the warehouse. This will ensure order filling continues without any major disruptions, and allow corrections to be made at the direct source of the error in the prompting system.
220.127.116.11 Feature: Prompt editing within the application
The ability to edit item information and media for non-linear prompts in the application is important because it allows the user to edit prompts anywhere in the warehouse. Initially an editing program was going to be created in conjunction with this application; however, after the revelation by a manager that walking back and forth from a location in the warehouse to the office where his computer was located was a significant waste of time. The ability to stand at an item in the warehouse and collect and edit media so it is ready for the system will save time and remove the necessity of learning yet another program on the computer in addition to the application.
4.3.0. 4 Feature: Adaptable to a variety of search inputs
Each employee in the warehouse is operating within the understanding of their own, personal world. When recalling an item or attempting to locate an item, the thought process is unique to each employee. One employee might recall an item by where it was located, while another employee may recall it by the SKU number because they have seen it so many times. This variability with memory and recall needs to be accounted for within the application for employees to find it easy to use. Adapting the search feature to a variety of inputs will enable employees to search for items based on their personal recollection. Instead of only including the item name and SKU, the information will include the location. If an employee does not want to walk across the warehouse to locate the item and SKU, they can type in a location and the application will list the items in that area and the employee can locate the item that way.
4.3.0. 5 Feature: Navigation Bar
The amount of time required to learn new technology can directly affect an employee's perception of their work tasks, especially if the technology is for a new process. A feature that preserves time by decreasing the number of navigational selections in applications is bottom bar navigation. A bottom bar navigation feature will not only save time for navigation, but the time required to learn the new technology. The bottom bar navigation feature is common amongst smartphones and is used in native phone, email and music applications. There are many other third-party applications with the feature as well, including Spotify, Instagram, and Google Maps.
4.4 Mobile Application Development
The Create-A-Task mobile application prototype was developed in Android Studio using the requirements outlined in the use cases and design features inspired by the ethnographic fieldwork. A total of two open source libraries were used in the
application; BottomBar and SwipeMenuListView. These libraries greatly enhanced
features in the application, and both licenses are included in the source code at: www.github.com/rfry5 / CreateATask.
Login and Home Screen
The login screen is the first screen displayed after the Create-A-Task application icon is selected (Figure 4.26). The screen includes the name of the application, a graphic of an individual authoring a prompt, and input fields for a username and password. The username and password for the user were pre-registered using the register button at the bottom of the screen.
Q E 8 13 u 11:57
Welcome To Create-A-Task!
Select an Option
Q SEARCH PROMPT
Si REVIEW DASHBOARD
Figure 4.26: The Create-A-Task application Login and Home screen.
The home screen is displayed after the user successfully logs in to the application (Figure 4.26). A welcome message, and an instruction to select an option are located above five button selections: Create Prompt, Search Prompt Library, Review Dashboard, and Logout. Each button navigates the user to a new screen corresponding to the button description. Button icons were added to correspond with bottom bar
navigation, which will be discussed in a subsequent section. The icons were selected
from the Android graphics library, with the objective to closely resemble the function of the button description. The logout button logs the user out of the home screen and navigates back to the login screen.
The create task screen presents with a variety of features and options (Figure 4.27). There are four buttons: take photo, take video, upload media, and save. The take photo button accesses the camera on the device to capture a picture. The take video button accesses the camera on the device to capture a video. The upload media button opens a dialog box, prompting the user to select the option of uploading a photo or video from the gallery on the device. The save. button saves all of the item information, media, and prompt command lines to the SQLite database on the device.
In addition to the four buttons, the user interface includes media review windows to review the photo captured with the camera, and replay the video recorded with the camera (Figure 4.28). If the user is unsatisfied with the photo or video captured, they are able to take a new picture or video that updates in the media review window.
The item name and SKU fields were incorporated based on item information used to label warehouse products. The location, description, item type, item size, and specific notes were added to provide addtional information that may be required by the prompting system for unusual items in a warehouse. The prompt command lines are text that can be read into the prompting system for audio cues.
The search library screen displays select information for items and corresponding media in the database (Figure 4.29). A thumbnail of the item is displayed, as well at the item name, SKU, location and description. The user is able to search items by item name, SKU, and location. The search feature uses a search filter, so as the user is typing in the item name, SKU, or location, the results are automatically updated
tem Size -
TAKE VIDEO SAVE
Prompt Commands: 1. Scan barcode 1 k
2. Remove item from shelf
3. Place on scale -3 '
A Q. 3
Create Home Library Review
Figure 4.27: The Create-A-Task application Create screen, (a) Access to the device camera to capture a picture after selecting take photo. (b) Access to the device camera to record a video after selecting take video. (c) Drop-down menus to select the item type and item size, (d) The dialog box that appears after upload media is selected. The upload photo and upload ^ video buttons navigate to the photo gallery on the device, (e) Scrolling section to add multiple prompt commands.
I NITOtJH.fr ^Xw*10*
,-rt n/(W g j | ll.Ptf .ri, w.*^ jflH j
I NetGear prosafe 5 port 10/100
TAKE VIDEO Prompt Commands: 1. Scan barcode
2. Remove item from shelf
3. Place on scale
A Q. a
Create Home Library Review
r v r o
qwertyuiop a s d f g h j k I zxcvbnmQ
4. Put on cart
5. Add command
test 1 the 4
1t34 9 * 7 t 0
q w e r t y u i o p
a s d f 9 h j k 1
z x c V b n m a
Figure 4.28: The Create-A-Task application Create screen with item information and media. The photo and video captured with the camera appear in the media review windows. The video can be played back in the window by clicking the play button, (a) A keyboard appears when the text fields are selected, (b) The keyboard appears for the scrolling prompt command lines text fields.
according to the text input. The list display scrolls smoothly through all of the items, no matter how many items are in the list. Any item in the list can be selected, which navigates the user to the edit task screen. The final feature in the search library is the ability swipe to delete an item and its corresponding media. Once the delete option is selected, the list of items automatically updates and the item is removed from the database on the device.
Q. [Search for Items or Media Recently Added Items
Ikea table legs 57212.011 Shelf 201 table legs
Dell desktop monitor 16.032.456 Shelf 205 monitor
Home and Gym Exercise Mat 11.300.010 Shelf 102 floor mat
Figure 4.29: The Create-A-Task application Search screen, (a) The search bar uses a text filter that automatically updates the search results with each letter and/or number entered by the user. The user can search by item name, SKU, or location, (b) The swipe to delete feature in the search library, (c) The search library can be viewed by scrolling through items, and scrolls smoothly regardless of the number of items in the library.
The edit task screen is virtually identical to the create task screen, with the exception of the save button (Figure 4.30). An update button is in its place to prompt the user to update the existing item information, instead of saving a new item.
1. Scan barcode
2. Remove item from shelf
3. Place on scale
A Q. B
Create Home Library Rewew
Figure 4.30: The Create-A-Task application Edit screen. The Edit screen is identical to the Create screen, with the exception of the update button in place of the save button.
The review dashboard screen displays the items currently flagged for review (Figure 4.31). A flagged item may be in the review dashboard because the incorrect media was collected for the item, or item information has changed in the warehouse and was not updated in the system. The list display is identical to the search library screen, with a thumbnail displaying a picture of the item, as well as the corresponding item name, SKU, location and description. Any item can be selected, and it will navigate the user to the edit prompt screen. If the item information and media are edited to the correct information, the user can swipe to mark the item as resolved. This will remove the item from the review dashboard. If the item is currently out of stock and requires removal from the system until restocking, the user can select out of
stock. This will remove remove the item and corresponding information from the search library and the review dashboard.
U S -O Â£ 8:40
Prompt Errors by Urgency
Dell desktop monitor 16.032.456 Shelf 205 monitor
Home and Gym Exercise Mat 11.300.010 Shelf 102
NetGear ProSafe 5-Port
A Q. a
Create Home Library Review
Figure 4.31: The Create-A-Task application Review Dashboard screen with the swipe to mark an item as resolved or out of stock feature.
Bottom Bar Navigation
A navigation bar is present at the bottom of every screen, excluding the login and home screen (Figure 4.32). This enables the user to navigate quickly between screens by minimizing button clicks. The icons used for the bottom navigation bar correspond to the icons on the home screen buttons, with the exception of the home button. Text was added underneath each icon to assist the user with navigation if the icons were not recognized.
Figure 4.32: The Create-A-Task application Bottom Bar. The bottom bar includes four key words and icons that match the function of the words; Create, Home, Search, and Review. The bottom bar is present on all screens, with the exception of the Home screen. The Home screen includes the bottom bar icons associated with the button function that match the navigation buttons.
4.5 Usability Testing
The metrics analyzed in the usability study were time on task, number of errors, and efficiency. Pre-test survey data of 11 questions were also collected to review each test subjects familiarity with mobile devices. The figures not included for the survey data can be found in Appendix H.
All five subjects were male warehouse managers, ranging in age from 35 to 57 years old. Every subject owned a smartphone, with one subject owning both a smartphone and mobile phone. Four subjects owned a smartphone for over six years, while one subject only owned a smartphone between 2-4 years (Figure 4.33a). All five subjects used smartphones to text, make phone calls and take photos (Figure 4.33c). Only four subjects used smartphones email, entertainment, and taking videos (Figure 4.33c). Three subjects used smartphones over 20 times per day, while the other two subjects use smartphones only 5-10 times per day and 10-15 times per day (Figure 4.33b). All subjects used text input options on smartphones multiple times per day. Three subjects captured photos with smartphones once per week. One subject captured photos with a smartphone once per day, and one subject captured photos with a smartphone multiple times per day. Subjects used smartphones to record video less frequently, with three subjects only recording video less than once a month. One subject recorded videos once per day, and one subject recorded videos once every two weeks. Three subjects used other touch screen devices in addition to smartphones. None of the subjects were colorblind, and all of the subjects wore glasses.
How long have you owned a smartphone/mobile phone?
0-6 Months 6 Months- 2 Years- 4 Years- 6+Years
2 Y ears 4 Y ears 6 Y ears
How often do you use a smartphone/mobile phone? 6 r
0-5 5-10 10-15 15-20 20+
times per day times per day times per day times per day times per day
What do you use your phone for?
Texting Phone Calls Email Entertainment Taking Taking
Figure 4.33: Responses from three of the eleven survey questions from all five subjects, (a) Survey response for duration of smartphone ownership, (b) Survey responses for frequency of smartphone use. (c) Survey responses for smartphone usage.
Figure 4.34 shows the average time to complete each task and the distribution of completion times amongst the subjects. The login task was completed at the fastest rate, with an average time of 1 minute and 4 seconds. The search, edit and delete prompt (SED) task was completed at the slowest rate, with an average time of 8 minutes and 14 seconds. This task had one outlier, with one subject completing the task in under 5 minutes. The review dashboard task was completed in an average of 2 minutes and 27 seconds, and the create prompt task was completed in an average of 8 minutes and 11 seconds. The create prompt task had an outlier as well, with one
subject completing the task in approximately 14 minutes.
Time On Task
11 -10 9 8 7 6 5 4 3 2 1 0L
Login Create S/E/D
O Data points Average time
Figure 4.34: Box plot of time spent completing each task with a sample size of n=5. Individual data points for each subject are listed in blue, and the average time to complete each task is listed in red.
As discussed in the methods, critical errors were defined as incorrectly completing
a task while thinking it was completed correctly, or asking for verbal assistance. Non-
critical errors were navigational errors and deviations from the shortest completion
path. Non-critical errors did not include typos that were corrected before exiting the
text field. The average number of critical errors was highest for the search, edit and
delete task, with an average of 3.6 critical errors (Figure 4.35). The review dashboard
task had the lowest average number of critical errors at 0.4 of a critical error (Figure
Number of Critical Errors for Each Task
Figure 4.35: Box plot of the number of critical errors for each task with a sample size of n=5. Individual data points for each subject are listed in blue, and the average number of critical errors is listed in red.
4.36) . The average number of critical errors for the create and login tasks were 1.2 and 0.8 errors respectively. The task with the highest average number of non-critical errors was the search, edit and delete task, with an average of 9.8 errors. The task with the lowest average of non-critical errors was the login task, with an average of 1.4 errors. The average number of non-critical errors for the create and review dashboard tasks were 2.8 and 6.2 errors respectively.
The after-scenario questionnaires (ASQ) were scored on a seven point scale, with 1 corresponding to strongly agree and 7 corresponding to strongly disagree. From the ASQ surveys completed after each task, user satisfaction for ease of completion was greatest across each task, with the exception of the create prompt task (Figure
4.37) . The login, search, edit and delete, and review dashboard task had an average overall satisfaction for ease of completion score of 2.4, 3.4 and 2.2 respectively. For the create task, ease of completion tied with time to complete the task with an average score of 3.8 (Figure 4.37b).
User satisfaction was lowest in the time required to complete the task for the login and search, edit and delete tasks (Figure 4.37). The average scores in user satisfaction for these tasks were 3.4 and 4.2 respectively. For the create and review
Number of Non-Critical Errors for Each Task
Login Create S/E/D Task
Figure 4.36: Box plot of the number of non-critical errors for each task with a sample size of n=5. Individual data points for each subject are listed in blue, and the average number of non-critical errors is listed in red.
dashboard tasks, the user interface had the lowest user satisfaction with an average score of 4.2 and 3.0 respectively (Figure 4.37b).
Averaging all ASQ scores across the four tasks, user satisfaction was greatest for ease of completion with an average score of 3.0 (Figure 4.38). User satisfaction was lowest for the user interface with an average score of 3.6. The average ASQ score for the time to complete the tasks was 3.5.
The average system usability scale (SUS) score was 67.5, with a low score of 35 points and a high score of 95 points (Figure 4.39). The standard deviation of the SUS scores was 24 points, and the median SUS score was 77.5.
Overall Satisfaction for Login Task
Overall Satisfaction for Create Task
O Data Points
Average ASQ Score
Ease of Completion Time to Complete
Overall Satisfaction for Search, Edit and Delete Task
Overall Satisfaction for Review Dashboard Task
Figure 4.37: ASQ scores for the login, create, search/edit/delete, and review dashboard tasks with a sample size of n=5. Individual data points for each subject are listed in blue, and the average ASQ score is listed in red. (a) Box plot of ASQ scores for login task, (b) Box plot of ASQ scores for create task, (c) Box plot of ASQ scores for search, edit and delete task, (d) Box gj plot of ASQ scores for review dashboard task.
Overall Satisfaction for All Tasks
Figure 4.38: Box plot of ASQ scores for all tasks with a sample size of n=5. Individual data points for each subject are listed in blue, and the average ASQ score is listed in red.
Subject SUS Scores
Subject 1 Subject 2 Subject 3 Subject 4 Subject 5
Figure 4.39: System usability scale (SUS) scores from all five subjects.
5. Discussion and Future Work
5.1 Ethnographically-Informed Design
The ethnographic study not only provided a plethora of information to inspire design features for the Create-A-Task (CTA) application, but also information on work tasks and employee relations within a variety of warehouses. The inter-rater reliability of the inductive content analysis results was measured with Cohen's kappa, and the inspired design results were incorporated into the development of the application for usability testing. While these results were both under the umbrella of the ethnographic study, they provided insight to entirely different components of this research.
The low kappa value of 0.14 suggests only slight agreement between the two coders. This low agreement is likely the result of using an objective coder to code the data. The primary coder, who spent 25 hours in the held, coded more nodes in the data compared to the objective coder, suggesting experience from the held provided information that was not understood by the objective coder. Future work should include testing inter-rater reliability with subjective coders who spend an equal amount of time in the held and develop the coding scheme together.
However, seeking to improve inter-rater reliability will not necessarily affect the ethnographically-informed design results. While it is possible for future work to uncover new hndings and rehne the existing coding categories, the new hndings may not be able to directly affect the design of the mobile application. For example, more information about the culture amongst warehouse employees may surface, but its relation to inform design may be slight. In addition, the heart of the ethnographically-informed design process is transforming heldwork results into design choices in the absence of a formal method (but only augmenting traditional methods and never as a standalone approach). That is, the design choices may be meticulously justified and closely mirror the ethnographic hndings, but there is no surefire formula that all
researchers use to translate qualitative data into meaningful design choices. Therefore, given the nuances of ethnography in the area of design, a mixed methods approach may be more useful for eliciting additional information moving forward with the fieldwork. Future work should include an increased focus on the completion of work tasks within the environments through the lens of activity theory and activity analysis, as well as distributed cognition.
Despite the low Cohens kappa value, the design features inspired by the ethnographic study translated well into the Create-A-Task application. The SUS score of 67.5 was high for a first prototype and there was positive feedback about feature familiarity from all five subjects. Initially, subjects were unable to find the swipe to delete feature in the search, edit and delete task, which was also the task with the highest average critical error rate of 3.6 errors. This was likely a direct result of the vague test instructions of delete the item, instead of swipe to delete the item. This was a critical error for all five subjects, although each of them expressed their knowledge of the feature and that briefly stating how to delete the item would enable them to locate the delete option with ease. This is supported by the number of critical errors in the review dashboard task, with no subject requiring assistance to locate the resolve and out of stock swipe functions. One subject enjoyed this feature the most because it was large and did not require reading smaller buttons. Each subject also expressed they would feel comfortable with the device after a brief, informal training. Future work should measure learnability in addition to other metrics, as there may be a decrease in critical errors and time on task for features that users are familiar with. A control group of users who are not warehouse managers should also be included in future work to provide a basis for comparison for the usability data.
The search feature and bottom bar also need further testing, as users did not use them frequently during task completion. This is likely due to the segmentation of the tasks and the size of the prompt library where users could simply scroll to locate
the correct item. The features in the application that were inspired by time should also be tested under circumstances more similar to tasks completed in a warehouse. This might include locating existing items in different aisles and bins, as well as items spread throughout the entirety of the warehouse. Extended and more strenuous tasks may present more of an effort to save time using the application, which may translate into using those features. The subjects who used the search feature only searched by the item name instead of its SKU or location, which is likely due to the linear nature of the instructions. To test the search feature's ability to search according to how users recall item information, items employees are familiar with should be included in future studies and presented as generically as possible in the usability test instructions.
These suggestions for future work highlight the disconnect between successfully fulfilling tasks with the application and overall application utility. While users may complete tasks quickly and with ease during a test scenario, knowledge of the true utility of the application is absent without the context of the actual warehouse environment. The ethnographically-informed design approach aimed to address this disconnect by informing design features on the basis of the context of the warehouse environment; however, it was not measured directly with usability testing. Future work should include deploying a prototype in a warehouse without disrupting or changing any component of the warehouse and how it operates. A held study like this would not only present an opportunity to elicit more information about the previously mentioned features, but the use of the entire application in the context of the warehouse environment.
A usability test in the held may also highlight how the application can contribute value to a warehouse customer base. No matter how easy the application is to use, or how rehned and enhanced the features become, its adoption will suffer without contributing to a strong value proposition. The intrinsic value of the application is
to enable warehouses to hire additional employees with cognitive disabilities through support of an automated job coach system, but does not include specific value for the warehouse management customer segment. The value can be bolstered by presenting an application that integrates and is beneficial to other technology in warehouses, such as inventory management applications and technology that interfaces with the warehouse management system. Strengthening the value of this individual component of the non-linear context-aware system will in turn strengthen the value proposition of the system as a whole.
5.2 Mobile Application Development and Usability Testing
Usability testing in relation to the functionality of the application uncovered two bugs that require code alterations. Overall, the application fulfilled the functional requirements with only two instances of the application crashing. The use cases generated in this study were core functionalities of non-linear prompt structures that included inherent characteristics of items in warehouses. The functionality will likely change after development and testing of the interactive prompting platform (IPP) to include additional requirements, such as integration with other systems. Future requirements may also change to reflect the varying levels of automation and the different processes employees complete. A few of the task completion steps observed in the field for this study that were not included in the CTA were the asset-management method of first in first out (FIFO), adjusting the quantity of items that can be pulled, how to stack and organize orders on pallets and carts for shipping, securing specific items, and how the order will be processed in the shipping area.
Feedback from the after-scenario questionnaire (ASQ) shows the user interface should be revisited and the non-critical error rate suggests navigation features should more closely mimic those on smartphones. The most commonly observed navigation error was swiping on the device to hide keyboard. The ability for the interface to adapt to usability features in smartphones should also be explored, specifically increasing
font and button size. Four of the five subjects expressed a desire for larger buttons, with one saying he used the increased user interface size on the iPhone.
5.3 Future Integration with RERC-ACT III technologies
Apprehensions for this research may be of the form, why develop the collection application before the non-linear prompting system? If the prompt media is dependent upon the structure and needs of the IPP, it may seem development of the media collection method is placing the proverbial cart before the horse. Developing the Create-A-Task application was critical because it elicited feedback from the end user as early in the system development phase as possible. Final deployment of the IPP system would likely suffer without significant efforts to incorporate requirements from end users running the system. This study suggests that developing an application that closely mimics current simple features on mobile devices will likely have a positive outcome with the end user. Continuation of this work will help ensure the IPP is used to its full potential by the greatest number of non-technical users when development is complete.
A mobile media collection and editing application for warehouse managers to install and maintain non-linear prompt information and media was developed by combining traditional requirements elicitation methods and ethnographically-informed system design. This research was a component of a non-linear, context-aware system that will enable adults with cognitive disabilities to competitively perform work tasks in a warehouse environment. Twenty five hours of ethnographic fieldwork was completed in four warehouses to inspire design for five application features, and interviews with key stakeholders were used to generate the functional requirements represented with 21 use cases. The inter-rater reliability calculated with the qualitative data from the fieldwork was 0.14 with an objective coder, suggesting only slight agreement. The mobile application was developed in Android Studio and deployed on an LG Nexus 5 device. The informed design approach was measured indirectly with usability testing, which showed successful functionality of the application and high user satisfaction with an SUS score of 67.5.
 S. Stoddard, 2014 Disability Statistics Annual Report, 2014.
 L. Lindstrom, B. Doren, and J. Miesch, Waging a living: Career development and long-term employment outcomes for young adults with disabilities, Exceptional children, vol. 77, no. 4, pp. 423-434, 2011.
 B. S. Barnow, The employment rate of people with disabilities., Monthly Labor Review, vol. 131, pp. 44-50, 2008.
 R. Kober and I. R. C. Eggleton, The effect of different types of employment on quality of life., Journal of intellectual disability research : JIDR, vol. 49, no. October, pp. 756-760, 2005.
 G. N. Siperstein, R. C. Parker, and M. Drascher, National snapshot of adults with intellectual disabilities in the labor force, Journal of Vocational Rehabilitation, vol. 39, pp. 157-165, 2013.
 B. of Labor Statistics, BLS Information Glossary. Available: http://www.bls. gov/bls/glossary.htm/tE, 2016. [Online]. Accessed: Nov. 13 2016.
 D. Gilbride, J. S. Mitus, J. Coughlin, and V. Scott, The consortium for employment success: Collaboration as a strategy to optimize employment outcomes for people with disabilities, Journal of Rehabilitation, vol. 73, no. 3, pp. 45-55, 2007.
 H. J. Bitterman, State and Federal Grants-in-Aid, pp. 1-16, 2006.
 R. Silverstein, An Overview of the Emerging Disability Policy Framework: A Guidepost for Analyzing Public Policy, Iowa Law Review, vol. 85, pp. 1757-1776, 2000.
 R. Silverstein, G. Julnes, and R. Nolan, What policymakers need and must demand from research regarding the employment rate of persons with disabilities, Behavioral Sciences and the Law, vol. 23, pp. 399-448, 2005.
 D. Stapleton, B. O. Day, and D. Wittenburg, Ticket to Work at the Crossroads : A Solid Foundation with an Uncertain Future, no. 202, 2008.
 D. C. Stapleton and R. V. Burkhauser, Has the Employment Rate of People Policy Brief, no. December, 2004.
 J. Kregel and D. H. Dean, Sheltered vs Supported Employment : A Direct Comparison of Long-Term Earnings Outcomes for Individuals with Cognitive Disabilities, pp. 63-84, 2000.
 P. Wehman, W. G. Revell, and V. Brooke, Competitive Employment: Has It Become the First Choice Yet?, Journal of Disability Policy Studies, vol. 14, pp. 163-173, 2003.
 A. Migliore, D. Manka, T. Grossia, and P. Rogan, Integrated employment or sheltered workshops: Preferences of adults with intellectual disabilities, their families, and staff, Journal of Vocational Rehabilitation, vol. 26, pp. 5-19, 2007.
 B. Hernandez and C. Keys, Employer Attitudes Toward Workers with Disabilities and their ADA Employment Rights: A Literature Review,
 P. Yeager, H. S. Kaye, M. Reed, and T. M. Doe, Assistive technology and employment: experiences of Californians with disabilities., Work, vol. 27, pp. 333 344, 2006.
 A. J. Langton and H. Ramseur, Enhancing employment outcomes through job accommodation and assistive technology resources and services, pp. 27-37,
 M. L. Wehmeyer, S. B. Palmer, S. J. Smith, W. Parent, D. K. Davies, and
S. Stock, Technology use by people with intellectual and developmental disabilities to support employment activities: A single-subject design meta analysis,
Journal of Vocational Rehabilitation, vol. 24, pp. 81-86, 2006.
 C. Sarmiento, Goodwill Works to Support Employment of People
with Disabilities. Available: http://www.goodwill.org/press-releases/
[Online]. Accessed: Nov. 13 2016.
 J. Bartholdi and S. Hankman Available: http://www2.isye.gatech.edu/ jjb/wh/book/editions/wh-sci-0.96.pdf
 M. L. Markus and C. Tanis, The Enterprise System Experience From Adoption to Success, Framing the Domains of IT Management: Projecting the Future Through the Past, pp. 173-207, 2000.
 Y. Ghazizadeh, Total Cost of Ownership: Factors to Consider. Available: https://software.intel.com/en-us/articles/
total-cost-of-ownership-factors-to-consider, 2009. [Online]. Accessed: Nov.
 Ieee, Guide to the Software Engineering Body of Knowledge Version 3.0 (SWE-BOK Guide V3.0). 2014.
 M. Berg, The Politics of Technology: On Bringing Social Theory into Technological Design, Science, Technology & Human Values, vol. 23, no. 4, pp. 456-490,
 F. D. Davis and V. Venkatesh, Toward preprototype user acceptance testing of new information systems: implications for software project management,
Engineering Management, IEEE Transactions on, vol. 51, no. 1, pp. 31-46, 2004.
Have you owned an Apple or Android device?
Only Only Both
Which did you prefer?
Figure H.3: Responses from two of the eleven survey questions from all five subjects. These survey responses were for Apple and Android phone ownership, and preferences between Apple and Android devices.
ETHNOGRAPHICALLY-INFORMEDDESIGN,DEVELOPMENTAND TESTINGOFAMOBILEMEDIACOLLECTIONANDEDITING APPLICATIONFORINSTALLATIONANDMAINTENANCEOFNON-LINEAR CONTEXT-AWAREPROMPTSINWAREHOUSEENVIRONMENTS by RACHELA.FRY B.S.,LoyolaMarymountUniversity,2014 Athesissubmittedtothe FacultyoftheGraduateSchoolofthe UniversityofColoradoinpartialfulllment oftherequirementsforthedegreeof MasterofScience BioengineeringProgram 2016
ThisthesisfortheMasterofSciencedegreeby RachelA.Fry hasbeenapprovedforthe DepartmentofBioengineering by CathyBodine,Advisor CathyBodine,Chair LevinSliker SarelvanVuuren November15,2016 ii
Fry,RachelA.(M.S.,Bioengineering) Ethnographically-InformedDesign,DevelopmentandTestingofaMobileMediaCollectionandEditingApplicationForInstallationandMaintenanceofNon-Linear Context-AwarePromptsinWarehouseEnvironments ThesisdirectedbyAssociateProfessorCathyBodine ABSTRACT ThethirdinstallmentoftheRehabilitationEngineeringResearchCenterforAdvancingCognitiveTechnologies(RERC-ACTIII)grantisfocusedonimprovingemploymentopportunitiesforworking-ageadultswithcognitivedisabilities.Thecombinedgoalforthethreedevelopmentphasesistobuildandtestanenterpriseready, context-aware,interactive,non-linearsystemtoenablepeoplewithcognitivedisabilitiestocompetitivelyperformworktasksinawarehousesetting.Acriticalcomponent ofthissystemisanapplicationforcollecting,editingandmaintainingtheappropriate warehouse-specicmediaforthepromptingsystem.Anethnographically-informed designapproachwaschosentoaccommodatethecomplicatedanddynamicnature ofwarehousesandtheneedfortheapplicationtoadapttousersinthecontextofa warehouseenvironment.Throughintensiveeldwork,informationwaselicitedfrom fourwarehouseenvironmentsandcombinedwithrequirementsfromkeystakeholders todevelopamobilemediacollectionandeditingapplicationprototype,whichwas thentestedwithvewarehousemanagers. Theformandcontentofthisabstractareapproved.Irecommenditspublication. Approved:CathyBodine iii
DEDICATION Idedicatethisworktothoseinmylifewhohavecontinuouslysupportedmyeducation andacademicendeavors-tomygrandfather,parentsandbrothers. iv
ACKNOWLEDGMENTS Iwouldliketoacknowledgemycommitteefortheircontinuedhelpandassistance withthisproject;Dr.CathyBodine,Dr.LevinSliker,andDr.SarelvanVuuren. IwouldalsoliketoacknowledgetheRERC-ATACINationalAdvisoryCouncilfor theirinputonthisproject,aswellastheresearchfundingfromtheRERC-ATACI grantnumberH133E09003fromtheNationalInstituteonDisability,IndependentLiving,andRehabilitationResearch,AssociationforCommunityLiving,Departmentof HealthandHumanServices.Last,butcertainlynotleast,Iwouldliketoacknowledge theColemanInstituteforCognitiveDisabilitiesforsupportandColemanFellowship funding. v
TABLEOFCONTENTS 1.Introduction................................... 1 1.1LivingwithacognitivedisabilityintheUnitedStates........... 1 1.2Employmentopportunitiesinthewarehousingindustry.......... 4 1.3RERC-ACTIII:Technologyindevelopment................ 5 1.4Warehouseautomationandenterprisesystems............... 8 1.5Currenttechnologyforinstallingcustomizablepromptingsystems.... 9 1.6Requirementsengineeringandethnographically-informedsystemdesign. 9 1.7Ethnographyresearchmethodologiesinrequirementsengineering.... 13 1.8Qualitativedataanalysis........................... 15 2.SpecicAims.................................. 18 3.MaterialsandMethods............................. 19 3.1EthnographicStudy............................. 19 3.2UseCases................................... 26 3.3MobileApplicationDevelopment...................... 27 3.4UsabilityTesting............................... 27 4.Results...................................... 31 4.1EthnographicStudy............................. 31 4.2UseCases................................... 67 4.3InspiredDesign................................ 67 4.4MobileApplicationDevelopment...................... 71 4.5UsabilityTesting............................... 80 5.DiscussionandFutureWork.......................... 87 5.1Ethnographically-InformedDesign...................... 87 5.2MobileApplicationDevelopmentandUsabilityTesting.......... 90 5.3FutureIntegrationwithRERC-ACTIIItechnologies........... 91 6.Conclusion.................................... 92 vi
References ...................................... 93 Appendix A.WarehouseInterviewQuestions........................ 98 B.CodeDenitions................................ 100 C.Pre-TestSurvey................................. 103 D.UsabilityTaskInstructions........................... 107 E.After-ScenarioQuestionnaire.......................... 112 F.SystemUsabilityScoreSurvey......................... 114 G.UseCases.................................... 116 H.Pre-TestSurveyResponses........................... 135 vii
1.Introduction 1.1LivingwithacognitivedisabilityintheUnitedStates Approximately14millionpeoplebetweentheageof18and64livinginthe UnitedStateshaveacognitivedisability.Unfortunately,peoplewithcognitive disabilitiesfacemanychallengesthata!ecttheirqualityoflife,includingpoverty, increasedhealthcomplications,andlowerwages.Only24.2%ofworking-age adultswithcognitivedisabilitieswereemployedin2014,palingincomparisontothe 75.4%employmentrateforworking-ageadultswithoutdisabilities.Employment continuestobeoneofthelargestobstaclespeoplewithcognitivedisabilitiesface, which,ifaddressedsuccessfully,couldpotentiallymitigatethesechallenges. Whendiscussingemploymentforadultswithcognitivedisabilities,itisimportant toconsiderboththeemploymentandunemploymentrate.Theemploymentrateis denedbyinclusionofworking-ageadultsregardlessoftheiremploymentstatus, whiletheunemploymentrateonlyincludesworking-ageadultsintheworkforceand thoseactivelyseekingemployment.Theemploymentratemetricprovidesinsight toafrustratingcomponentofthelaborforcethatworking-ageadultswithcognitive disabilitiesencounter;thedesiretoworkbuttheinabilitytondajob.Thissituation canresultinwhatDepartmentofLaboridentiesasa"discouragedworker,"dened asa,"personnotinthelaborforcewhowantandareavailableforajobandwho havelookedforworksometimeinthepast12monthsbutwhoarenotcurrently lookingbecausetheybelievetherearenojobsavailableortherearenoneforwhich theywouldqualify". Todate,thefederalgovernmenthasspentbillionsofdollarsandpassedmultiplelegislativeactstocombatdiscriminationandimproveemploymentopportunities forpeoplewithdisabilities[7,8,9].Twonotablelegislativeactspassedwerethe RehabilitationActof1973(RA)andtheAmericanswithDisabilitiesActof1990 (ADA).TheRAprohibiteddiscriminationonthebasisofdisabilityinprograms 1
conductedbyfederalagencies,inprogramsreceivingfederalnancialassistance,in federalemployment,andintheemploymentpracticesoffederalcontractors.VocationalRehabilitation(VR)programsforeachstatewerealsoestablishedunderTitle 1oftheRAandusedgovernmentassistancetohelpoperatetheprograms. WhiletheRAmadegreatstridesinprohibitingdiscriminationandestablishing programstoassistindividualswithdisabilities,itwasnotallencompassingfordiscriminationoutsideofgovernmentinstitutionsandinstitutionsthatreceivedfunding fromthegovernment.ThepassageoftheADAin1990addressedthisissuebyprohibitingdiscriminationonthebasisofdisabilityinemployment,stateandlocalgovernment,publicaccommodations,commercialfacilities,transportation,andtelecommunications.TheADAalsoassistedinusheringinanewparadigmthat,"disability isanaturalandnormalpartofthehumanexperiencethatinnowaydiminishesa personsrighttofullyparticipateinallaspectsofsociety,includingemployment". Thisisincontrasttotheoldparadigmthatconsideredindividualswithdisabilities as"defective"peoplethatneededtobe"xed".ThenewparadigminturnsupportedthefourgoalsoutlinedintheADA:equalityofopportunity,fullparticipation indecision-making,independentlivingandeconomicself-su"ciency. InadditiontotheRAandADA,therewereothercalculatede!ortstospecically addressthelowemploymentrateforworking-ageadultswithdisabilities.The TickettoWorkandWorkIncentivesImprovementActof1999(TTW)waspassed withthegoaltoincreaseearningsforworking-ageadultswithdisabilitiesandinturn lowerbenetsthatcauseanincreasedburdenontaxpayers.Arecentreport foundminimalimpactndingsforthisprogram;however,itisspeculatedthatthe minimalimpactisaresultoftheprogram sexistenceforlessthan10years. Despitetheseimmenselegislativee!orts,theemploymentrateforworking-age adultswithcognitivedisabilitiesremainsnearlyathirdoftheemploymentratefor working-ageadultswithoutadisability.Itisalsocurrentlydebatedwhetherthe 2
employmentrateforworking-ageadultswithcognitivedisabilitieshasdecreased,but thispositionislargelybasedindisagreementoverdatalimitationsforthetargetpopulation[12,10].Evenmoreconcerningisthecombinationofthestagnantemployment rate,shelteredemploymentandunderemploymentintheworkforce.Neithersheltered employmentorunderemploymentareaccountedforintheemploymentrate,butis somethingthatgreatlya! ectsworking-ageadultswithcognitivedisabilities. Underemploymentexistswhenanemployedindividualisworkinglessthanwhat qualiesasfull-timeemployment.Workinglessthanfulltime,frequentlyreferredto asparttime,oftendisqualiesanemployeefrombenets,suchasaccesstohealth insurance.Unfortunately,thisisarealityforworking-ageadultswithcognitivedisabilities,astheyareoftenunderemployedandpaidlowerwages. Shelteredemploymentisaformofsegregatedemploymentforindividualswith disabilities.Kregelet.al.describeditas"awiderangeofsegregatedvocationalandnon-vocationalprogramsforindividualswithdisabilities,suchassheltered workshops,adultactivitycenters,workactivitycentersanddaytreatmentcenters." Programsthatsupportedshelteredemploymentweredevelopedbecauseofthebeliefthatadultswithdisabilitieswerenotcapableofparticipatingsuccessfullyina competitiveemploymentpositions.Thispracticehasdrawnsharpcriticismfor itsinabilitytoprovidemeaningfulemploymentoutcomes,theisolationofindividuals fromthecommunity,andthelackofprogressiontocompetitiveemployment. ItwasnotuntilthelandmarkSupremeCourtcaseof OlmsteadvL.C. in1999that supportedemploymentwasmandatedtoacertaindegreeforpeoplewithdisabilities seekingemployment.Therulingplacedemphasisonintegration,declaringthat individualswithdisabilitieshavearighttoemploymentinenvironmentsthatareas integratedaspossible. Evenwiththeincreasedopportunitiesforcompetitiveemployment,peoplewith cognitivedisabilitiesstillfacemanyobstaclestoemployment.Thereareotherfactors 3
thatcontributetotheinabilitytondajob,suchastheperceptionthatpeoplewith cognitivedisabilitiesareunderqualied[7,16].Otherfactorsincludethedisability itself,potentiallossoffederalbenets,lackofeducation,andemployerattitudes. Assistivetechnology(AT)hasbeenfrequentlyassociatedwithprovidingameansto overcomethesefactorsthroughestablishingcompetitivejobplacementthatsupports participationandincreasesasenseofwell-being[17,18,19]. Fortunately,thereareafewindustriesintheUnitedStatesthatarecommitted toaddressingtheseobstaclestoemploymentthroughthededicationofsignicant resourcestohiringandtrainingaworking-ageadultswithcognitivedisabilities.Two notablebusinessesthatemployandsupportmanypeoplewithcognitivedisabilitiesareGoodwillIndustriesandPRIDEIndustries.Employmentintheseindustries consistsofpositionsinwarehouselocations,workshopsandtrainingtohelpprepare employeesfortheworkforce.Asof2013,thirtythousandpeoplewithdisabilitieswere employedatGoodwill. 1.2Employmentopportunitiesinthewarehousingindustry Onesectoroftheworkforceidentiedwithgreatopportunityforemploymentis inthewarehousingindustry.TheBureauofLaborStatisticsreportsthat772,000 individualsareemployedinthewarehousingindustryasof2015.Ofthoseindividuals,55,790areemployedasorderllers.Theaverageearningsforproductionand nonsupervisoryemployeesas $ 15.52anhour,withorderllersearninganaverage salaryof $ 32,990peryearwithbenets. Theorderllerpositionwithinwarehousesisastrongcompetitiveemployment opportunityforpeoplewithcognitivedisabilitiesandfostersdevelopmentofavariety ofskills.TheWarehousingEducationandResearchCouncil(WERC)denesthe mainresponsibilityofanorderlleras"llingcustomerordersanddeliveringthem tothedeliveryplatform."TheWERCalsolistssixjoboutputsandtasksfora warehouseorderllerjob,whichinclude:completingtimelyshipments,maintaining 4
asafeworkplace,ensuringcustomersatisfaction,maintainingane"cientworkplace, completingaccuratetransactions,andensuringproductivitymeasurementsmet. 1.3RERC-ACTIII:Technologyindevelopment TheRehabilitationEngineeringResearchCenterforAdvancingCognitiveTechnologies(RERC-ACT)wasave-yeargrantrstawardedtoUniversityofColorado ofDenverin2004.Insubsequentyears,thegrantwasawardedin2008(RERC-ACT II)andmostrecentlyin2014(RERC-ACTIII).ThefocusoftheRERC-ACTIII grantistoimproveemploymentopportunitiesforpeoplewithcognitivedisabilities. Therearethreedevelopmentphaseswithinthegrant;(D1):E!ectiveConguration andAuthoringofInteractivePromptingTemplates,(D2):IntegratingtheInteractive PromptingPlatform(IPP)withEnterpriseSystemsandGenerationofContext-aware LinearPrompts,and(D3):Real-timeMonitoringofExecutionofJobTaskswithNonlinearIntelligentContextualPrompts.Theoverarchinggoalistobuildandtesta non-linear,context-awaresystemtosupportpeoplewithcognitivedisabilitiesinan orderllingpositioninawarehousesetting.Componentsofthesystemwillinclude anindoornavigationpackagecoupledwithasmartcart,anon-linearcontext-aware promptingsystem,andamethodtoinstall,congure,andmaintainpromptsforthe system(Figure1.1). Onemajorcomponentofthissystemistheinteractivepromptingplatform(IPP). Giventheimmensediversityofwarehouseproducts,preinstalledgenericpicturesand videosforpromptswillnotbesu"cientforthepromptingsystem.Beforethesystem issuccessfullydeployed,warehousespecicpictures,videoandaudiowillneedtobe collectedandeditedintotheappropriatepromptingtemplateformatbywarehouse managementandemployees.Warehouselogisticswillplayakeyroleinsettingupthe interactivepromptsforthesystemandmaintainingandeditingthepromptmedia. Theorder-pickingmethodandwarehouselayoutcanhelpdeterminethespecicityof themedia,whileactivityprolingcanidentifywhatpromptswillbemostactiveand 5
Media collection application to setup non linear context aware prompting system Indoor navigation and smart cart to guide and prompt a user through tasks Task completion with prompting system and navigation Feedback Maintenance and editing of media for prompts Figure1.1:ThesystemunderdevelopmentfortheRERC-ACTIIIgrant.Componentsofthesystemincludeamediacollectionandeditingapplicationtoinstalland maintainanon-linear,context-awarepromptingsystem,anindoornavigationpackagecoupledwithasmartcart,andanon-linear,context-awarepromptingsystemto guideemployeeswithcognitivedisabilitiesthroughworktasks. necessarypreemptivee!ortsforsettingupprompts.Areviewofwarehouselogistics willhelporienttheprojectinthecontextofwarehouseenvironments.Alargebody ofwarehousescienceandengineeringinformationisprovidedbyGeorgiaInstituteof Technologyandisusedforthefollowingreviewofwarehouselogistics. 1.3.1WarehouseTasks:OrderPicking Warehouseoperationscanbebrokendownintotwocategories;inboundandoutboundprocesses.Theorderllertaskoforderpickingisassociatedwithoutbound processes.Orderpickingistheactofretrievinganitemwithinthewarehouseand bringingittoaspecicdestination. Whilethisprocessmaysoundsimple,inrealityitcanbecomplex.Approximately 55%ofwarehouseoperatingcostsstemfromorderpicking.Naturally,optimizing orderpickingmethodshasthepotentialtodecreasecost.Thewarehousemanagement system(WMS)isausefulsoftwareoptimizationtoolthatsortscustomerordersand compilespickliststhatwilleventuallybedistributedtoemployees.Ifnecessary, 6
thepicklistsarefurthereditedbytheWMStogeneratepick-linessotraveltimein thewarehouseise" cient.Overhalfoftheorder-pickingprocessisspenttraveling throughthewarehouse,whileonly25%isspentsearchingandextracting. Therearetwotypesoforderpicking:split-casepickingandcarton-picking.Splitcasepickingisconsideredmorelaborintensiveasunitsarenotcontainedinlarge cartonsandemployeesarerequiredtosortthroughahighdensityofSKUs(item identiers).Carton-pickinginvolvesemployeespickingcartonsfromshelves.This taskissimplerthansplit-casepicking,somuchsothatwarehouseswithsignicant capitalautomatethecarton-pickingprocessbecauserectangularcartonsareeasyto transportwithautonomousmachines. 1.3.2WarehouseDesign:Layouts Warehouselayoutscanbecomplexandvarysignicantly.Theoverallgoalfor layoutdesignistooptimizee"ciencyforinboundandoutboundprocesses.Itemswith highturnoverratesareplacedinconvenientlocationsdependentuponthereceiving andshippinglocationsinawarehouse.Thetwomostcommonformsofshippingand receivingcongurationsareU-owcongurationandow-throughconguration. Withow-throughcongurations,theshippingandreceivingdocksareonoppositeendsofthewarehousecreatingconvenientlocationswithinadirectlinebetween thetwostations.WithU-owconguration,shippingandreceivingdocksareon thesamesideofthewarehouse,creatingconvenientlocationsinatriangularpattern outwardfromthestations.Inthismodel,convenientlocationsaremoreconvenient thantheow-throughcongurationmodel,howeverthetradeo!isthatinconvenient locationsaremoredi" culttonavigateto. Layoutsarealsoconstructedforpalletreserve,cartonpickareas,andindividual pickareas.Tasksthatdonotinvolveoperatingmachinerybeginatthecartonpick areas,howeveritispossiblefortallcartonshelvingunitstorequireforkliftsandlifts toretrieveandreplenishitems.Cartonsonthelowerlevelscanbeorderpickedby 7
employeeswithcartsifthecartonsarenotoversized.Individualpickareas,called fast-pickareas,arelocatedinasub-regionofthewarehouse.Theyareoftenformed toimproveresponsetimestocustomerordersthatareinhighdemand.Theseareas aregeneralsmallanddonotrequireextensivetravelforpick-ordersandaremore easilysupervised. 1.3.3WarehousePerformance:ActivityProling Tosecureabetterunderstandingofwhereproductsarelocatedandwhatpick tasksarecompletedmostoften,awarehouseactivityprolecanbegenerated. Amongstthestatisticsgatheredinthisprole,therearefourpertainingtopicktasks: 1)theaveragenumberofpick-linesshippedperdayand2)theaveragenumberof unitsperpick-line3)theaveragerateofintroductionofnewproductsand4)seasonalproducts.Acommonformofactivityprolingthatcollectsinformationon productactivityisABCanalysis.ThisanalysisclassiesSKUsasA(asmallnumber ofSKUsthatrequiremostoftheactivity),B(moderatelyimportantSKUs),orC (largenumberofSKUsbutonlylimitedactivity). 1.4Warehouseautomationandenterprisesystems TheRERC-ACTIIItechnologyindevelopmenthasthepotentialtobesupplanted byautonomoustechnologyinautomatedwarehouses.Givencurrentadvancements andrecente!ortstoautomateandstreamlinewarehousesupply-chainlogistics,itmay beunclearastowhythissystemisunderdevelopment.Oftenremovedfromtheautomationhypeisdiscussionofthesignicantbarrierstowarehouseautomation.Two ofthelargestbarriersarecostandtechnologyshortfalls,whichareoftenstaggering duetotheinabilitytoadaptautonomoustechnologytocertainwarehouselayouts. Inaddition,automationdoesnotalwaysmeanfullyautomated.Manywarehouses arepartiallyautomatedandstillrequireorderllersandstockersforday-to-dayoperationsinthewarehouse. Withthegoalofbuildinganenterprisereadysystem,itisimportanttoreview 8
whatmakesanenterprisesystemsuccessfulandwhatobstaclesexistinthedeploymentandmaintenanceofasystem.Someconsiderationsforenterprisesystemsare nancialcostsandrisk,technicalissues,ITadoptionuseandimpacts,andintegration . TotalCostofOwnership(TCO)asanimportfactortoconsiderwhenreviewing enterprisesystems.TCOisamodelthatanalyzesthetotalamountofexpensesfor thelifeofanITsystem,withspecialfocusontheongoingcoststhatareincurred tomaintainthesystem.Supportcoststypicallyaccountfor70%-85%ofTCO,thus makingitanextremelyapplicableconsiderationfortheenterpriseready,contextaware,interactive,non-linearsystem. 1.5Currenttechnologyforinstallingcustomizablepromptingsystems Technologyonthemarkettosetupandcustomizepromptsforcertaintasksare small-scaleanddesignedforanindividual-specicpromptingsystem.AbleLinkTechnologiescurrentlysellstheVisualImpact3,aniPhoneapplicationthatcanbeused onAppledevicesforcreatingcustomprompts.WhatisuniqueaboutVisualImpact 3isthesystembehindpromptcustomization.AbleLinkInstructionalMediaStandards(AIMS)usesanXMLbasedprotocoltodesignandsharepromptsonmultiple devices.TwoothercustomizablepromptingprogramsfromAbleLinktechnologyare thePocketCoachandVisualAssistant,buttheseproductshavebeendiscontinued. TheAIMSplatformcanonlybeusedondevicesthatare"AIMSready."Thismake thepromptauthoringprocesseasilytocustomizeandbuildononedevicetotransfer toanother.Limitationsincludelackoftranslationtolarge-scaleworkandusability information. 1.6Requirementsengineeringandethnographically-informedsystem design Requirementsengineeringisasubdisciplineofsoftwareengineeringthatinvolves elicitation,analysis,specication,andvalidation.Itisoftencitedasoneofthe 9
mostimportantstepsoftheSoftwareDevelopmentLifeCycle(SDLC)becausepoorly executedrequirementsengineeringcanresultincostlymistakesandfailedsoftware systems[25,26,27,28].Tosolvethisproblem,itwaspostulatedthataugmenting therequirementselicitationprocesswithsocialsciencemethodologieswouldaidin thesuccessofnewlydevelopedsoftware[29,30,31].Ethnographyisthemethodthat quicklyemergedbecauseitenableddeveloperstoexploretheenvironmentoftheend user.Thecombinationandcollaborationbetweenthesedi!erentscienticelds contributedtotheearlydevelopmentoftheeldnowknownasComputerSupported CooperativeWork(CSCW). Ethnographyistraditionallydenedas"aqualitativemethodforstudyingaphenomenonwithinitsnaturalcontext".Itisaresearchmethodologyoftenusedby anthropologiststostudycommunities,cultures,andsocieties.Individualswho conductethnographicstudies,knownasethnographers,areguidedbyfourprinciples:studyingphenomenainnaturalsetting,maintainingaholisticview,providing descriptiveunderstandingandtakingamembersperspective.Thesefourprinciplesestablishguidelinesthatbringethnographerstorealworldsettingswherethe activitiesareexperiencedrsthand.Inobservingtheseactivities,itisimperativefor theethnographertounderstandtheminthelargercontext,understandthemasthey occur,andgainaninsidersviewofasituationratherthanprojectingpersonalviews . Givenitsrootsinthesocialsciences,itmaybehardtoimaginehowethnography transitionedintotheeldofrequirementsengineering.In1987,Dr.LucySuchman,ananthropologist,publishedseminalworkthatcalledforrethinkingthedesign processfordevelopingtechnology.In PlansandSituatedActions:TheProblemofHumanMachineCommunication ,Suchmanstudiedhumaninteractionsbetweeno"ce workersandaphotocopierusingethnography.Herethnographicapproachpromoted, "observationsthatcaptureasmuchofthephenomena,andpresupposeaslittleas 10
possible".Herndingssupportedherpostulationthatanindividual sactions areintertwinedanddependentupontheevolutionofanimmediatesituation,which contradictedthewidelyheldviewthatanindividual sactionsarebaseduponaplan theyconstructedbeforetheexecutionofactions.Shealsodescribedtheimportance ofgeneratinganunderstandingofanenvironment,"Ifwebuildanaccountofaction, orlanguage-use,withoutsimultaneouslybuildinganunderstandingoflanguagesrelationshiptotheworld,thereisasenseinwhichwequiteliterallydonotknowwhat wearetalkingabout,"whichappliestotheimportanceofobtainingaccesstoactual situationsinsteadofgeneratingmockscenarios. FollowingtheworkbySuchman,seminalstudiesthatrstcombinedrequirements elicitationwithethnographybeganintheearly1990swithtechnologyinLondon Undergroundlinecontrolrooms,systemdesignforairtra"ccontrol,technologyinanambulancecontrolcenter,anddesignworkinthefashionindustry .Eachofthesestudieswasparamountbecausetheyidentiedanddiscussedthe benetsandproblemsencounteredwhencombiningethnographywithrequirements elicitation.Whilethesestudiesprovidedthegroundworkforcombiningrequirements elicitationmethodswithethnography,theylackedaformalprocesstobridgethegap betweenthequalitativedatageneratedfromethnographicstudiesandsoftwaredevelopmentrequirements.Inaddition,theresearchgroupsdidnotdevelopanddeploy aprototypeinthestudiedenvironmenttotestthedesignrequirementselicitedfrom theethnographiceldwork.Thismissingcomponentleftalingeringquestionasto whetherrequirementselicitationfromethnographywasbenecialatall. Oneoftheproblemsidentiedbyresearchersinthesestudieswasadheringtotraditionalethnographyeldtime.Traditionalethnographicstudiesoftentakemonths oryearsinordertogainacomprehensiveunderstandingofthecultureandenvironment.Thislengthytimeframecontradictsthetimespentintheeldforthese studies,whichrangedfrommonthstodays.Whilelongertimeintheeldtranslates 11
toastrengthenedunderstanding,itcanbeimpracticalforteamslookingtodevelop technologywithinaspeciedtimeframe.Thisalteredtimeconstraintengenderedthe creationoffourdi!erenttypesofethnographyintheCSCWeld:concurrent,"quick anddirty,"evaluative,andre-examination.Themostrelevanttypeofethnographyrelatedtotheissueoftimeintheeldis"quickanddirty,"asitaccompanies shortperiodsofeldwork.Recently,rapidethnographyhasemergedasanewtypeof ethnography.Whilesimilarto"quickanddirty"ethnography,rapidethnography providesamorerigorousandclearlydenedapproachforconductingeldwork. Withtheincreaseduseoftechnologyconnectingmultiplelocations,itbecame clearthatsmall-scaleethnographicstudiesinasingleeldlocationforanextended periodoftimeweretoorestrictive.Thiswasyetanotherdi!erencebetween traditionalethnography,whichisoftenlimitedtooneeldsite.Withtheadoption ofmultipleeldsites,ethnographybeganevolvingtofollowpeopleandtechnology tomultiplesitesandenvironments.OneofthelargestareaswithinCSCW thatutilizesmultipleeldsitesisresearchanddevelopmentofhealthcaretechnology. Thisisnotsurprisingconsideringthecontinuedoverwhelmingtransitionfrompaper toelectronicmedicalrecords. WiththecontinuingevolutionofethnographyintheeldofCSCW,itiseasyto becomelostamongstthedenitivepractices.Whatdenesethnographyiscontinuing toevolve;however,itisexplicitlyclearwhatethnographyintheeldofCSCWis not.Ethnographyshouldonlyinformdesign,notreplaceothertraditionalrequirementselicitationmethods[35,39,42,43,38,32].Thiscriterioniscriticalbecause qualitativedataisproducedfromeldwork,anditiswidelyacknowledgedthatcontextualapproachesbasedonethnographictechniquesdonotmapwellontocurrent formalspecications.Ethnographyisalsoconcernedwithdeterminingcultural practicesofcommunitiesandwouldthereforebeincompleteinelicitingfunctional requirements. 12
Naturally,combiningtraditionalrequirementselicitationmethodswithethnographyintheeldofCSCWhasdrawncriticismsinceitsinception.Thereare disagreementsonwhatqualicationsarenecessaryforethnographersconductingeld studies,aswellasthereductioninthevalueofethnographywhenitisusedtosimplyinformdesign.Inaddition,theevolutionofethnographywithintheeld posesquestionsofwhatdenesproperethnographyandhowitcanspecicallyinformdesign.Whilequantitativestudiesarestillthedominatemethodintheeld, ethnographyiswidelyrecognizedandutilizedbyresearchgroupsandmajorglobal companies,includingIBM,Microsoft,Intel,andXerox[35,45].Thesecompanies, amongstothers,employanthropologistsandethnographicexpertstohelpinformdesignforproducts.Itmaynotbesurprisingthatin1987,LucySuchman sworkwas performedattheIntelligentSystemsLaboratoryatXeroxPaloAltoResearchCenter. 1.7Ethnographyresearchmethodologiesinrequirementsengineering ItisevidentthatethnographyintheeldofCSCWiscontinuingtoevolveas technologyadvances.Assuch,methodsforcombiningethnographyandsystemdesignarefrequentlyredenedandaugmented.Thecombinationofrequirementsengineeringandethnographyisfairlynewandmanydi! erentapproachesexistwithin ethnographicallyinformedresearchstudies. Researchmethodsfromtherstethnographically-informeddesignstudiesvaried inafewaspects.Eachofthestudiesdi!eredineldworklengthandobservation recordings.Therewerealsodi!erencesinwhoconductedtheethnographicstudies; somestudiesonlyusedethnographerswhileothersusedcomputerscientists.The studieslackedthoroughdescriptionsofthequalitativedataanalysismethodsthe ethnographersandcomputerscientistsusedforthestudies.Therewasalsonoformal methodtobridgethegapbetweenthedataproducedfromtheethnographicstudy andrequirementselicitedfordesign. Studieswithsuccessfuldevelopmentanddeploymentofasoftwaresystemin13
formedbyethnographybeganin2000,withamajorityofresearchproducedby agroupinCopenhagen,Denmark.Theresearchgroupsuccessfullydeveloped anddeployedAWAREarchitectureandplatformswithapplicationsAwarePhone andAwareMedia,andaPeri-OperativeCoordinationandCommunicationSystem (PoCCS)inhospitals[46,47,41].Whileeachoftheseprogramsandsystemswere developedwiththehelpofethnographicmethodstoinformdesign,acombination ofmethodswereusedtoelicitrequirementsandanalyzequalitativedata,including activitytheoryandanalysis[48,46]. Giventhevarietyofapproachesandnuances,itmayseemunclearastowhy ethnographyisselectedoveralternativemethodsforresearchstudies.Easterbrook et.al.identiessixdi! erentmethodsforacquiringinformationanddatafor softwareengineeringresearch:controlledexperiments,casestudies,surveyresearch, ethnographies,actionresearchandmixed-methods.Ofthesedi! erentresearchmethods,casestudiesaretheclosestrelatedtoethnographiesandoftenusedwithinthe CSCWeld.Giventhesimilaritiesbetweenthemethods,itisimportanttodelineate thedi!erences. Casestudiesaredenedas,"anempiricalinquirythatinvestigatesacontemporaryphenomenonwithinitsreal-lifecontext,especiallywhentheboundariesbetween phenomenonandcontextarenotclearlyevident,"andrequireaclearlydenedresearchquestionthatexploreshoworwhyaphenomenaoccurs.Incontrast, ethnographycanhelptounderstandhowtechnicalcommunitiesbuildacultureof practicesandcommonstrategiesthatenablesthemtoperformtechnicalworkcollaboratively.Ethnographyalsorequiresaclearlydenedresearchquestionthatexplores culturalaspectsofacommunity.Hereinliesthefundamentaldi!erencebetweencase studiesandethnography;onemethodexploresaphenomenonwhiletheotherexploresculturalaspects.Ethnographyfostersanunderstandingofhowindividuals worktogetherinthecontextoftheenvironment. 14
Onceethnographyisdenedasoneofthemethodsforrequirementselicitation,it followsaclearlydenedpathofformulatingaresearchobjective,devisingastrategy forselectingstudyparticipantsandselectingappropriateresearchtechniques.Four commonmethodsforselectingstudyparticipantsinethnographicresearcharequota, purposive,snowball,andconvenience.Thesesamplingmethodsarenonprobability methodsandoftenusedbecauseofresearchtimeconstraints.Boththequotaand purposivestrategiesareusedwhenagroupisspeciedandscreenedtoensureparticipantsareappropriateforthesample.Whilequotausesthismethodwithaset numberofparticipants,purposivedoesnotasitismaynotbefeasibletosetaspecicnumberofparticipants.Convenienceandsnowballsamplingarebothsampleas yougostrategiesandareappropriateforsituationswhereitisunknownwhowillbe abletoparticipate.Thesnowballsamplingstrategyreliesonwordofmouthfrom previousorinitialparticipants,astheywillreferotherindividualswhomaybegood candidatesforthestudy.Conveniencesamplingisusedtoselectpeoplewhomeet requirementsfortheeldworkandareavailableandwillingtoparticipate. Thenalstepafterestablishingtheresearchobjectiveandstrategyforselecting studyparticipantsisselectingresearchtechniquesandapproachesfortheeldwork. Participantobservationsandinterviewsaretwodatacollectionmethodscommonly used.Methodswithinparticipantobservationincludethinkaloudobservationsand participant-observertechniques.Thinkaloudobservationsencouragedsubjectsto voicetheirthoughtprocesseswhencarryingoutworktasks,andparticipant-observer techniquesallowtheobservertoworkwithparticipantsandbecomepartoftheteam.  1.8Qualitativedataanalysis Oneofthemostcriticalprocesseswithinethnographiceldworkistheanalysisof qualitativedata.Beforedelvingintodi!erentmethodologies,itisnecessarytodene twodominanttheoriesthatexistwithinqualitativedataanalysismethods;genera15
tionoftheoryandconrmationoftheory.Conrmationoftheoryisusedtoconrma theorythathasbeenpre-identiedbystrengtheningandprovidingevidencethatsupportsthetheory.Incontrast,generationoftheoryisusedtoextractatheorythatis groundedinthedata.Outsideinuences(includingpreviousresearchandliterature) donotalterorinuencethegenerationoftheory,asthedataissolelyresponsible foritsdevelopment.Whilemanymethodsexistforanalyzingqualitativedata,amajorityofthemfallwithinthegenerationoftheoryframework.Commonmethodsfor analyzingqualitativedataincludegroundedtheorymethods,conversationanalysis, distributedcognitionandactivityanalysis.Someofthesemethodshavebeenused inpreviousethnographicallyinformeddesignstudies,aslistedintheethnographic methodologiesinrequirementsengineeringsection. Codingofqualitativedataisananalysismethodcommonlyusedinrequirements engineeringprojectsandstudiesthatincludeethnographiceldwork.Aformal methodforthecodingprocessiscontentanalysis,whichisdenedasamethodthat simpliesdatathroughuseofcodesandcategoriestoclassifytheinformation. Contentanalysisbeginsoncequalitativedataiscollected.Beforecodingbegins, adeductiveorinductiveapproachmustbeselectedtodrivesubsequentanalysisdecisions.Aninductiveapproachistakenwhenthereislittleformerknowledgeaboutthe phenomenonunderreview,whileadeductiveapproachisbuiltuponpreviouscontent analysisworkandresults.Therststepintheprocessiscodingthedata,whichis alsocomprisedoftwoseparateapproaches;aprioricodingandemergentcoding. Aprioricodingusespredeterminedcodesfromestablishedtheorieswithinliterature toanalyzethedata,whileemergentcodingusescodesthatemergefrominformation withinthedata. Codingthedatainvolveslookingforspecicitems,askingquestionsconstantly aboutthedata,andmakingcomparisonsatvariouslevels.Questionsareaskedinternallybythecodertodevelopabetterunderstandingofthedata.Makingcomparisons 16
ofdatastrengthenstheinterpretationofthenalresultsbyrevealingsimilaritiesand di !erences. Naturally,thecodingprocessissubjecttoresearcherbias,soitisimportant todocumentjusticationsforthecodingschemaandvalidateresults.Onemethod frequentlyusedforvalidationisdatasourcetriangulation,wheremultipledatasources supportinterpretationofthedata.Inadditiontodatasourcetriangulation,intercoderreliabilityiscalculatedtodetermineifthecodingisreproducible.Forthis procedure,objectiveorsubjectivecodersaregivenexplicitinstructionsregardingthe codingprocessandcodethedatawiththesetcodingschemageneratedbythecoder. Asamplesetofdataiscodedrsttoensurethatinstructionsareunderstood,after whichformalcodingofthedatafromthestudybegins.Thistestisnottodetermine thereliabilityofthechosencodingschema,butrathertoidentifythereliabilityof codingthedataitself.ThemeasureofreliabilityisoftencalculatedusingCohen s Kappa = P a P c 1 P c (1.1) where P a isthepercentageofcasesthecodersagreeuponand P c isthepercentageof agreedcaseswhenthedataiscodedbychance.Acalculatedvalueof0.6andabove, outofamaxscoreof1,suggestshighreliability.[52,53] 17
2.SpecicAims SpecicAim1: Conductanethnographicstudyinwarehouseenvironmentsand generateusecaseswithinformationfromkeystakeholderstoelicitandanalyzerequirementsforamobilemediacollectionandeditingapplication. SpecicAim2: Developamobilemediacollectionandeditingapplicationprototype. SpecicAim3: Conductusabilitytestingwiththemobilemediacollectionand editingapplicationprototypeandwarehouseemployees. 18
3.MaterialsandMethods 3.1EthnographicStudy Theresearchobjectivefortheethnographicstudywastounderstandhowwarehouseemployeescompletetasksandinteractwiththewarehouseenvironment.Two researchquestionsweredevelopedtoguidethisobjective: 1.Whatfactorsinuencetaskcompletionbyemployeesinawarehouse? 2.Whatinteractionswithtechnologydoemployeesexperienceinawarehouse? 3.1.1Sites Warehousesiteswereselectedusingtheconveniencesamplingmethod.Given thetimelineconstraintsandthedi"cultygainingaccesstowarehousesintheDenver metroarea,itwasnotpossibletouseanalternativesamplingstrategy.Siteselection beganwithcoldcallsandemailstoovertwentywarehousesinDenver.Ofthewarehousescontacted,fourrespondedpositivelyandagreedtoaninitialmeeting.The followingwarehousesweregraciousenoughtograntaccessforthestudy: 1.Engine&PerformanceWarehouse 2.DenverPublicSchoolsFood&NutritionServices 3.OskarBluesBrewery 4.GoodwillIndustrieseCommerce 3.1.2Subjects Subjectsincludedinthestudyweremanagersandemployeesworkinginthe warehouses,excludingadministrativesta!.Therewasinteractionwith34subjects, comprisedofninemanagersand25employees.Table3.1showsthreeemployeedemographics;gender,primaryjobfunction,andstatusofemployment. 3.1.3Consenting Theeldworkforthisstudyincludeddocumentingaccountsandprocesseswitha cameraandaudiorecorder.Managersandemployeeswhowererecordedand/orphotographedinthewarehousesconsentedbysigningtheColoradoMultipleInstitutional 19
Table3.1:Alistofthe34subjectsintheethnographicstudy,includingsubjectgender (Male(M)orFemale(F)),employmentposition,andemploymentstatus(FullTime (FT)orPartTime(PT)). Gender Position FullTime(FT)orPartTime(PT) F Manager FT M Manager FT M Manager FT M Manager FT M Manager FT M Manager FT M Manager FT M Manager FT M Manager FT M OrderFiller FT M OrderFiller FT F OrderFiller FT M OrderFiller FT M OrderFiller FT M OrderFiller FT M OrderFiller FT M OrderFiller FT M OrderFiller FT M OrderFiller FT M OrderFiller FT M OrderFiller PT M ItemLister PT M CustomerService PT M Booksortingandshipping PT M OrderFiller PT M Booksortingandorderller PT M ItemLister PT F OrderFiller PT M ItemPhotographer PT M OrderFiller FT M OrderFiller FT M OrderFiller FT M OrderFiller FT 20
ReviewBoard(COMIRB)approvedconsentformsunderthestudy,"AnExploratory InvestigationoftheImpactoftheAssistiveTechnologyPartnersProductTesting Lab."Theseformsincludedaconsentpacketandaphotograph/videotaperelease form.Theformswerereadbyeachsubjectandreviewedverballyinaquietsection ofthewarehousebeforesigning. 3.1.4ProtocolandMethods Thenumberofsitevisitsvariedforeachwarehouseduetothepermissiongranted forthetiminganddurationofthevisits.Table3.2showsthenumberofvisitstoeach site,aswellasthedurationofthevisitanddatacollectiondetails.Atotalof25 hourswasspentintheeld. Beforetherstvisittoeachwarehouse,interviewquestionsweredevelopedwith aresearchcolleagueafteraliteraturereviewofwarehouseprocesses.Sixtyquestionsweredevelopedanddividedintosixcategories;generalwarehousequestions, technology-related,warehouse-specic,employee-related,training-relatedandtaskrelatedquestions.Uponreviewwithresearchadvisors,interviewquestionswerenarroweddownandrenedintothirtyquestionsusingthe"binningandwinnowing" approachusedbyPROMISinvestigators.Theprocessofdeleting,adding,and reninginterviewquestionswasdependentuponredundancy,relevance,specicity, andnecessitytoelicitmoreinformationforthesystemasawhole.Theinterview questionscanbefoundinAppendixA. Therstsitevisitsconsistedofinterviewswithmanagers,introductionstoavailableemployees,andwarehousetours.Thesevisitselicitedinformationaboutthehistoryofthewarehouseprocessesandglobalversusday-to-dayoperations.Afterthe initialsitevisits,subsequentvisitswerescheduledtoshadowwarehouseemployees andcollectinformationaboutthewarehouseenvironment.Onewarehousepresented asanexceptiontothismethodologyasitwaslocatedinLongmont.Atthissite, 21
Table3.2:DatacollectiondetailsfortheethnographiceldworkatDenverPublicSchoolsFood&NutritionServicesandOskar BluesBrewery. WarehouseSiteVisitTotalHoursInformationCollectionDetails DenverPublicSchools12.75-Interviewedonewarehousemanagertoelicitwarehouselogisticinformation Food&NutritionServices-Collectedinformationofemployeetrainingandtheorderllingandstockingprocess -Reviewedequipmentusedtollorders -Collectedphotosofthewarehouselayoutwithacameraandobservationswithaeldnotebook 23-Interviewedamanagertolearnabouttheorderdeliveryprocess -Shadowedfouremployeesllingordersinthecoolerofthewarehouse -Shadowedoneemployeellingordersinthedrygoodsareaofthewarehouse -Interviewedmanagertolearnmoreindepthinformationonwarehouselogistics historyandcurrenttechnology 31-Interviewedthemanagerwhooverseesproductinventory -ObservedworkowchangeofpaceattheendofthedayonaFriday 41-ArrivedintheearlymorningonaMondaytoobservethefruitcheckprocessfortheincoming produceorder -Shadowedthreeemployeesmovingordersinthecooler OskarBluesBrewery13-Interviewedwarehousemanagertolearnaboutoverallbreweryanddeliverylogistics -Exploredtheshippingandcoolerareasandcollectedlayoutinformationwithacamera andobservationswithaeldnotebook -Shadowedfouremployeesllingordersinthecoolerandshippingdock 22
Table3.3:DatacollectiondetailsfortheethnographiceldworkatGoodwillIndustrieseCommerceandEngine&Performance Warehouse. WarehouseSiteVisitTotalHoursInformationCollectionDetails GoodwillIndustries13-Interviewedtwowarehousemanagerstolearnaboutwarehouselogistics eCommerce-Exploredthewarehousecollectingextensivelayoutinformationwithacamera,andobservations withaeldnotebook -Conversedwithoneemployeeinthephotographyarea -Shadowedoneemployeeinthelistingarea 23-Interviewedtwoemployeesaboutdailyworktasks -Shadowedoneemployeellingorders 32-Arrivedearlybeforethestartoftheworkdaytoobservethemorningmeeting -Interviewedfouremployeesaboutdailyworktasks -Shadowedoneemployeellingorders Engine&Performance12-Interviewedtwomanagerswithtworesearchcolleaguesandanassociateprofessortocollect Warehouselogisticsinformation -Warehousetourwithbothmanagerstoseewarehouselayoutanddailyprocessesinaction 22.25-Exploredthewarehousecollectingextensivelayoutinformationwithacamera,and observationswithaeldnotebook -Shadowedandspokewithveemployeesllingordersandstocking.Discussedjobresponsibilities andgeneralwarehouseworkow -Spokewithwarehousemanageronthewarehouseoortolearnmoreaboutwarehouselogistics andwarehousehistory -Filledanorderwithoneemployee 32-Exploredthewarehousecollectinglayoutinformationwithacamera,andobservationswith aeldnotebook -Spokewithwarehousemanagertoelicitmoreinformationaboutwarehouselogistics -Shadowedtwoemployeesllingordersandparticipatedinllingorderswithoneemployee 23
interviewsandshadowingwerecompletedinonehalf-day. 3.1.5DataCollection Datawascollectedwithanaudiorecordingdevice,acamera,andaeldnotebook. Table3.2showsthedatacollectionmethodsforeachsitevisit.Insomesituations, employeeswerenervousinthepresenceofarecordingdevice.Inordertoprevent datacollectionofanalteredpersona,aeldnotebookwasusedtocollectinformation fromconversationsinlieuofthedictaphone. 3.1.6DataAnalysis Inductivecontentanalysiswasusedtoanalyzethedata.Thecontentanalysis processwascompletedinthefollowingphases;preparation,organizationandreportingtheanalysisandresults(Figure3.1). Theorganizationphasebeganwithuploadingallofthephotos,videos,eld notes,andaudiorecordingsintothequalitativedataanalysissoftwareNVivo.This softwarepackagewaschosenbecauseit,"supportsqualitativeandmixedmethod research,andaidsinorganizationandanalysisofunstructuredqualitativedatato helpndkeyinsights".Eachoftheaudiorecordingswastranscribed,andthe datawasorganizedbywarehouseandvisitationdate.Intotal,therewere310photos, 5.5hoursofinterviewsand30pagesofeldnotes. Opencodingwastherststepintheorganizationphase.Allofthedataforeach warehousewasreviewedinNVivoandwordsandphraseswererecordedinthecorrespondingnotessectionofNVivo.Thisinitialreviewofthedataincludedobservations andideasgeneratedfromlookingthroughthedata.Fromthesenotes,headingswere generated,andtheywerethentransferredtoacodingsheetinMicrosoftWord(AppendixB).Categoriesweregeneratedfromtheheadingsandgroupedunderhigher orderheadings.Aconceptualmodelwasthengeneratedtoreectthenaldata abstraction. Inter-raterreliabilitywasmeasuredwithanobjectivecoder.Tenpercentofthe warehousedatafromallfoursiteswasrandomlyselectedinNVivoforanalysis.The 24
Figure3.1:Theinductiveanddeductivecontentanalysisapproaches,includingsteps withinthepreparation,organizationandreportingphases.Figureborrowedwith permissionfrom. 25
codingsessionbeganwithadescriptionofthepurposeofinter-raterreliabilityand howtheoverallprocesswasstructured.Thegeneraldescriptionsofthewarehouses listedintheresultssectionwerereadtothecoderandphotographsofthewarehouse layoutswereshowntoprovideanunderstandingofeachsite.Thecodedenitions werereviewedthoroughly,andasamplepieceofdatawasthenreviewedwiththe codertofacilitateunderstandingofthecodingprocess.Oncetheprocesswasunderstood,theobjectivecoderbegancodingthedatausingthecodedenitionsheetasa reference.Themethodusedtocalculateinter-raterreliabilitywasCohen skappa. 3.2UseCases Theobjectiveofgeneratingusecaseswastooutlinefunctionalrequirements elicitedfromkeystakeholders,whichincludedresearchadvisorsoverseeingtheRERCACTIIIresearchanddevelopmentphasesandwarehousemanagers. 3.2.1SubjectsandConsenting Therequirementselicitedfromthefourresearchadvisorsweredrivenbytherequirementsfortheinteractivepromptingplatform(IPP).TheIPPwasunderdevelopmentconcurrentlywiththemobileapplication,sorequirementsgatheredincluded corefundamentalsforanon-linearcontext-awarepromptingsystem.Thisincluded layersofpromptsthatprovidespecic,additionaltaskcompletioninformationtothe userintheformofphotos,videoandaudio. Warehousemanagersservedaskeystakeholdersbecausetheendgoalisforthe mobileapplicationtobeusedbymanagersinawarehouseenvironment.Theserequirementswereforitemspecicsandfunctionstopreventdisruptionofprocessesin awarehouse.Sevenwarehousemanagerswereinterviewedforlogisticsinformationat thefourwarehouses.Theconsentingprocessforthemanagerswasidenticaltothose listedintheethnographicstudymethods.Managerswhowererecordedforinterviews consentedbysigningtheColoradoMultipleInstitutionalReviewBoard(COMIRB) approvedconsentformsunderthestudy,"AnExploratoryInvestigationoftheImpact 26
oftheAssistiveTechnologyPartnersProductTestingLab."Theconsentformwere readbyeachsubjectandreviewedverballyinaquietsectionofthewarehousebefore signing. 3.2.2ProtocolandMethods Requirementswereelicitedfromresearchadvisorsforthedurationoftheresearch project.Informationwasgatheredanddiscussedduringmeetingsandpresentations onresearchwork.Requirementsfromwarehousemanagerswereelicitedfrominterviewsaboutwarehouselogisticsandtaskcompletion.Alloftherequirementswere compiledintotheusecasetemplateformatoutlinedin"DocumentingUseCases"by GeriSchneider,providedbydeveloperWorksIBM. 3.3MobileApplicationDevelopment Theobjectiveofmobileapplicationdevelopmentwastousetheelicitedfunctional requirementsandinspireddesignfeaturestobuildaprototype.TheCreate-A-Task applicationprototypewasdevelopedinAndroidStudio.TheAndroidplatformwas chosenbecauseofitsabilitytoseamlesslyintegratewiththeotherAndroidplatform devicesindevelopmentforthenon-linear,context-awarepromptingsystem.Open sourcelibrarieswerealsousedtoincorporatespecicfeaturesintotheapplication. 3.4UsabilityTesting TheobjectiveoftheusabilitystudywastotestthefunctionalityoftheCreateA-Taskapplication,measureoverallusersatisfactionandusabilitymetrics,andelicit userfeedbacktoenhancefuturedesigniterations. 3.4.1Subjects FivewarehousemanagerswererecruitedfromtheDenverPublicSchoolsFood& NutritionServiceswarehouse(DPS). 27
3.4.2Site Usabilitytestingwascompletedon-siteattheDPSwarehouseduringadistrict holidayweekday.Thedistrictholidayhadsuspendedthenormalworkowinthe warehouse,providingaquietandopenenvironmentfortesting. 3.4.3Consenting ConsentingwascompletedwiththeColoradoMultipleInstitutionalReviewBoard (COMIRB)approvedconsentformsunderthestudy,"AnExploratoryInvestigationof theImpactoftheAssistiveTechnologyPartnersProductTestingLab."Theseforms includedaconsentpacket,aphotographandvideotapereleaseform,anddemographic form.Theformswerereadbyeachsubjectandreviewedverballyinaprivate,quiet sectionofthewarehousebeforesigning. 3.4.4ProtocolandMethods Afour-tieredplasticshelvingunit(36"Wx58"Hx18"L)wasusedforusability testingatthewarehousesite(Figure3.2).Fouritemswereplacedontheshelving unit;asmallpackagingbox,aNetGearProSafedesktopswitchbox,anemptyredbin, andtwohomegymexercisemats.HandwrittenlabelswereattachedtotheNetGear Prosafebox,theredbin,andthehomegymexercisemats(Figure3.3).Thefour tiersoftheshelvingunitwerelabeledindividuallybetween101-104. Afterconsenting,thesubjectcompletedabriefsurveyondailytechnologyuse (AppendixC).Thesubjectwasthenbriefedontheoverallstructureofthetest.This includedinstructionstothinkaloud,andtotroubleshootasmuchaspossiblebefore askingforhelp. TheusabilitytestwascomprisedoffourseparatetasksusingtheCreate-A-Task applicationonaLGNexus5smartphone(AppendixD).Aftereachtask,thesubject completedanafter-scenarioquestionnaire(ASQ)(AppendixE).Attheconclusion ofallfourtasks,thesubjectcompletedasystemusabilityscorequestionnaire(SUS) assessingtheentireCreate-A-Taskapplication(AppendixF). 28
Figure3.2:Thefour-tieredshelvingunitwithfouritemsusedforusabilitytesting. Theshelveswerelabeledbetween101-104. 3.4.5DataCollection DatawascollectedwiththeusabilitytestapplicationLookbackandaGoPro cameraxedtotheshelvingunit.TheLookbackapplicationwasenabledwhileusers completedtasksandcapturedallactivityonthescreenofthephone,aswellasvideo oftheusersfacewhilecompletingthetask.AlimitationofLookbackwascomplete disconnectionfromtheapplicationwhenthephonecamerawasaccessed.TheGoPro camerawasusedtocollectuserinformationafterLookbackwasdisconnected. 3.4.6DataAnalysis TheLookbackrecordingsandGoProvideoswerereviewedmanuallyandtagged atrelevanttimepointsforthetimeontaskandnumberoferrorsmetrics.The numberoferrorswassplitbetweencriticalerrorsandnon-criticalerrors.Critical errorsweredenedasincorrectlycompletingataskwhilethinkingitwascompleted 29
(a) (b) (c) Figure3.3:Itemlabelsforthe(a)emptybinofclosethooks(b)NetGearProSafe desktopswitchboxand(c)exercisemat.Thelabelsincludedtheitemnameinthe center,SKUinthelowerleftcorner,anddescriptioninthelowerrightcorner. correctly,oraskingforverbalassistance.Non-criticalerrorswerenavigationalerrors anddeviationsfromtheshortestcompletionpath.Non-criticalerrorsdidnotinclude typosthatwerecorrectedbeforeexitingthetexteld.TheSUSformwerescored accordingtotheappropriatescoringmethod.Evennumberedquestionsweregiven ascorecontributionof5minusthescaleposition,andoddnumberedquestionswere givenascorecontributionofthescalepositionminus1.Thescoresofeachquestion werethensummedandmultipliedby2.5tocalculatetheSUSscore.Additional feedbackwasalsodocumented. 30
4.Results 4.1EthnographicStudy Eachwarehousepresenteduniqueinformationduetothevariationamongstproducts,customerbases,andmethodsofoperation.Eachofthefoursiteswillbedescribed,followedbythendingsandinspireddesign. 4.1.1Site1:Engine&PerformanceWarehouse Therstsitewasamid-sizewarehouseinDenverthatsoldautomotiveparts.At 60,000squarefeet,thewarehousehadthreeseparateoors,oneloadingdock,and onereceivingdock(Figure4.1a).Byoor,productswerearrangedaccordingtothe ABCmethod.GroupAcorrespondedtoproductspulledandshippedmostoftenand residedontherstoor.GroupBproductswereorderedtoalesserdegreeandwere storedonthesecondoor,andCproductsthatwererarelyorderedwerelocatedon thethirdoor.Amongsttheoors,productswerearrangedbyproductline,and alphanumericallywithineachproductline(Figure4.1b). (a) (b) Figure4.1:TheEngine&Performancewarehouselayout.(a)Thethreeoorsofthe warehouse.(b)Itemsorganizedalphanumericallybyproductlineswithinanaisle. Thelayoutandorganizationofthewarehousewasorderedandmaintainedbyone manager.Thismanagerhadbeenemployedatthewarehousesinceitsinception,when 31
therewasonly6,000squarefeetofspace.Themanagerwasresponsibleforordering the90productlinesstoredinthewarehouse.Whiletheorganizationsystemwas setalphanumericallybyproductline,themanagerhadtoaccommodatetheunique industryquirkofreissuedproductlinesthatwereoutoforder.Someproductlines inthewarehousespannedmultipleaislesandshelvesanditwassimplyimpossible toreorganizetomakeroomforanewpartwithinaproductlinethatfellbetween theexistingalphanumericstructure.Asaresult,productswereplacedasclosedto theproductlineaspossibleandnotallproductswereorderedalphanumericallyby productline(Figure4.2). Figure4.2:Itemsplacedontheoornearesttotheircorrespondingproductlines. 32
Foremployees,atypicalworkdaybeganat8amandendedat5pm.Transfer orderswerecompletedrstuponarrivalinthemorning.Transferswerelargeorders senttooneofthe13Engine&Performance(EP)warehousesintheUnitedStates eachweek,vedaysaweek.Thenumberofemployeesllingtransferordersgraduallytaperedo!asthemorningprogressed,withsomeemployeesbeginningtoll dailyorderswithinafewhoursofthestartoftheshift.Largetransferorderswere completeafterapproximately5-6hours,afterwhichallemployeesbeganworkon llingordersprintedoutatthepickticketstation(Figure4.3a).EPdevelopeda customwarehousemanagementsystem(WMS)tosuitthewarehouseneeds,anda componentofthissystemwastheprintticketstation.Whenanorderwasreceived, apickticketwasautomaticallyprintedatthepickticketstation.Employeesthat wereincloseproximitytothestationfrequentlycheckedthepickticketprinterand placedtheticketsinadesignatedbasket.Whenanemployeewasreadytollan order,theywouldremoveapickticketfromthebasket,assesstheitemsonthelist, grabacartifnecessary,andlltheorder.Oncetheorderwaslled,theemployee returnedtotheshippingareanearthepickticketstationandboxedtheorderon theshippingconveyerbelt(Figure4.3b).Inadditiontoorderllers,thereweretwo employeesresponsibleforstockingitemsinthewarehouse.Theseemployeeswere highlyknowledgeableofwheretolocateitems,andhelpedorderllerswithquestions aboutwheretolocatespecicitems. Theprocessofllingorderswasuniquetoeachorderllerinthewarehouse. Aspecicpickpathwasnotdesignatedbymanagement,soemployeeswerefree todeveloptheirownmethodforretrievingitems.Giventhedi"cultywithkeeping itemsorganizedalphanumericallybyproductline,newhireswerepairedwithveteran employeestolearnwhereitemswerelocated.Accordingtothemanager,ittook approximatelythreeweeksforanewemployeetobecomefamiliarwiththewarehouse. Iftherewereanyquestionsaboutitems,employeesfrequentlycrossedpathswhile 33
(a) (b) Figure4.3:Thestartandendpointsoforderpickinginthewarehouse.(a)Thepick ticketstationstartpointwithacomputerandticketprinter.(b)Theshippingarea endpointwhereorderswereeitherpackagedorplacedinbins. workingandquestionswereoftenansweredbrieyinpassing.Occasionally,questions wereyelledoutloudtoeveryoneinthewarehouse,astheopenspaceprovidedan environmentforeveryonetoeasilyheareachanother(Figure4.4). 34
4.1.2Site2:DenverPublicSchoolsFood&NutritionServices Thesecondsitewasamid-sizewarehouseinDenverthatprovidedfoodand nutritionservicesforthe190+schoolsintheDenverPublicSchoolsdistrict.The foodandnutritionservicesincludeddeliveryoffreshproduce,frozengoods,dry goods,andsuppliestokitchensinthedistrict.At40,000squarefeet,theone-story warehouseincludedafreezerandcoolersectionseparatefromadrygoodssection (Figure4.5). Figure4.5:DenverPublicSchoolsFood&NutritionServiceswarehousemap. Thedrygoodsareaofthewarehousewascomprisedofsixaisleswithtwo-tiered shelvingunits.Aislesone,twoandthreewerelledwithsuppliesanddrygoodsthat wereorganizedfromheaviestandinthelargestquantitiestothelightestandindividualitems(Figure4.6).Heavyweightkitchencleaningsuppliesweretherstitems 36
(a) (b) Figure4.6:Thestartandendpointsofthedrygoodspickpath.(a)Thebeginning ofthedrygoodspickpathwithheavyitemsandcleaningchemicalsinaisleone.(b) Theendofthedrygoodspickpathwithindividualitemsinaislethree. inaisleonetopreventleakageintofooditemsifthepackagingwasdamaged.Items pickedintheaisleswereorganizedintotheground-levelbins.Binswerenumbered indescendingorder,withoddnumbersontheleftsideoftheaisleandevennumbers ontherightsideoftheaisle.Therstnumberinthebinnumbercorrespondedto theaisleitwaslocatedin.Binnumbersweredividedbyevenandoddnumbersto facilitateaz-pickpath.Thez-pickpathmadetheorderpickingprocessmoree"cient bypreventingorderllersfrombacktrackingbetweenitemsinanaisle.Aislesfour throughsixstoredoverstockitemsinlargequantities(Figure4.7).Thesecond-tier oftheshelvingunitsinaislesonethroughthreealsostoredoverstockitems,butin smallerquantities. Thecoolerandfreezerareaswereadjacenttothedrygoodsarea.Theentrance tothecoolerwaslinedwiththickplasticblindstokeepthetemperatureat40 F. Thecoolerwasalarge,openrectangularroom,withtwo-tieredshelvingunitslining twoopposingwalls(Figure4.8).Anautomateddoorservedastheentrancetothe freezerdirectlyoppositeoftheentrancetothecooler.Thefreezerwascomprisedof two-tieredracksthatalignedallofthewalls,inadditiontoacentralislandoftwotieredracks.Thisu-shapedesignfacilitatedyetanotherz-pickpathbeforeexciting 37
backintothecooler.Itemsinthecoolerandfreezerwereorganizedbyweight,with heavyanddurableitemsintherstbinandlightfragileitemsinthenalbin(Figure 4.9).Onceanorderwaslled,theemployeewrappedtheorderinplasticwrapto securetheitemsonthepallet,markedtheschoolnameinsharpieonallfoursidesof theorder,andplacedthepalletonthesideofthecoolerattheendofthepickpath (Figure4.10). Figure4.8:Acornerofthecoolerwithitemsforpickingstackedonpalletsand overstockitemsonthesecond-tieroftheshelvingracks. Atypicaldayforanemployeebeganateither6amor8am.Employeesarriving at6amwereresponsibleforthequalitycheckofincomingproducefortheday,as wellasstackingitemsinthecoolertoprepareforthedailycoolerpickorders.Once thecooleritemswerecheckedandplacedinthecorrectlayoutinthecooler,the orderpickingcommenced.Employeesattendedamorningmeetingandweregiven productivitysheetstocompleteastheylledorders.Thegoalwastocompleteallof thecoolerpickingby1:30-2:00pm.Employeesinthecoolerselectedpickticketsfrom acubbyunitlocatedinthecooler,whileemployeesinthedrygoodsareaselectedpick 39
(a) (b) Figure4.9:Thestartandendpointsofthecoolerpickpath.(a)Itemsinboxesat thebeginningofcoolerpickpath.(b)Fragileproduceattheendofcoolerpickpath. Figure4.10:Completedordersstoredinthecooler. 40
(a) (b) Figure4.11:Pickticketcubbies.(a)Drygoodspickticketarea.(b)Coolerpick ticketarea. ticketsinacubbyatthestartoftherstaisle(Figure4.11).Thereweretypically veemployeesorderpickinginthecoolerandoneortwoemployeespickinginthe drygoodsarea. Oncecoolerorderpickingwascomplete,employeeswhoarrivedat6:00amwere closetoleavingworkfortheday,buttheystillassistedwiththeend-of-the-daytasks ifcoolerpickingwasnishedearly.Theend-of-the-daytasksincludedcollectingall ofthetrash,sweepingtocollectallofthedebrisontheoor,andreorganizingitems thatremainedinthecoolerandfreezer.Oncecleanuptaskswerecomplete,items forthefollowingdaywererestocked. 4.1.3Site3:GoodwilleCommerce Thethirdsitewasa100,000squarefootwarehousethathousedtwoseparate programs;GoodwilleCommerceandGoodwillGoodElectronics(Figure4.12).The focusofthesitevisitswasintheeCommercesectionofthewarehouse.TheeCommercedepartmentwasestablishedatGoodwillafterrecognizingthevalueofcertain itemsthatweredonatedtotheorganization.Thewarehouseservesasacorporate o ceanddonationcenter,receivingitemsfrom28storesinthestateofColorado. WhenemployeesatGoodwillstoresanddonationcenterssortthroughdonateditems, anythingthatlookslikeitmaybeofgreatervalueisplacedasideanddeliveredto 41
(a) (b) Figure4.12:TheGoodwillwarehousespacesubdividedintotwosections.(a)The GoodElectronicssectionofthewarehouse.(b)TheeCommercesectionofthewarehouse. theeCommercewarehouse.OnceitemsarereceivedattheeCommercewarehouse, theyaresortedandeitherreturnedtothestoresiftheyareoflowvalue,orprepared foronlinelisting. TheeCommercesectioniscomprisedofeleven"hubs"intheopenspace;areceivingdockforbinsofitemsfromstores,asortingarea,aspecializedlistingand researcharea,agenerallistingarea,aphotographyarea,ajewelrysortingarea,a mediaandbookstoragearea,abooksortingarea,ashippingarea,anitemstorage area,andastagingareaforitemspickedupinperson(Figure4.13).Itemsarriveat thereceivingdockinlargebinsseparatedbetweenbooksandallotheritems.The largebinsofbooks,referredtoas"melons,"areplacednearthebook-sorterconveyer beltwheretheyaresortedusingabarcodescanningsystem(Figure4.14).Allother binsareplacednearthelistingareaandreviewedbyemployeeswholistitems.Ifan itemappearstobeunique,thespecializedlistersintheresearcharearesearchthe itemanddetermineitsvalue.Itemsthatdonothavesignicantvaluearereviewedby thegeneralitemlisterswhoarealsoresponsibleforlistingitemsonline.Onceitems havebeenaggedforlisting,theyareplacednearthephotographystationwhere photographerstakehighqualityphotosofindividualitems.Afterphotosaretaken, 42
(a) (b) (c) (d) Figure4.13:Fouroftheelevenseparate"hubs"withintheeCommercewarehouse. (a)Bookanditemstorageracks.(b)Itemlistingarea.(c)Photographystation.(d) Stagingareaforitempickup. theitemsareplacedonacartwiththeSDcardandthelisterliststheitemsonline withadescriptionandthecorrespondingphotos. Atypicalworkdayforanemployeebeginsat7amandendsbetween11am-5pm becausemanyemployeesarepart-timeworkers.Whilethereweredesignatedtasks inthe"hubs"withintheeCommercearea,itwasrareforanemployeetocomplete onlyacertainsetoftasksinthewarehouse.Employeesmovedaroundtocomplete di!erenttasksdependingonthelevelofactivityinthewarehouse.Booksortersoften assistedwithllingordersandlistershelpedwithllingordersandphotography. 43
(a) (b) Figure4.14:Thebooksortingareainthewarehouse.(a)Booksortingconveyorbelt. (b)Binsofunsortedbooks. 44
4.1.4Site4:OskarBluesBrewery Thefourthsitewasa100,000squarefootbreweryinLongmontthathoused equipmentforbrewing,packaging,andshippingbeer.Thefocusofthisworkwasin theshippingandstorageareaofthebrewery.Theshippingareawasalarge,open coolerthatkepttheproductatalowtemperaturetopreventavoralterations(Figure 4.15a).Palletswithavarietyofproductlineswereplacedinthecornerofthecooler. Theshippingareawasadjacenttoanother,muchlargercoolerthatstoredkegsand pallets(Figure4.15b). (a) (b) Figure4.15:Coolersintheshippingdockofthewarehouse.(a)Shippingareacooler. (b)Storagecooleradjacenttotheshippingareacooler. Atypicalworkdaybeganbetween7am-10am,withstaggered8hourshifts(7am3pm,8am-4pm,9am-5pm,10am-6pm)toensuretheshippingdockwascoveredbetween7amand6pm.Identicalpaperprintticketsweredistributedtoemployeesat thebeginningofthemorningshiftandlargeorderswerelledbythegrouponto deliverytrucks.Alloftheequipmentusedtocompleteorderllingwaspowered; everyemployeeusedaforklifttollorders,withtheexceptionofoneemployeewho wasresponsibleforbuildingpallets.Buildingpalletswasaprocessofdismantling palletsofasingleproductandcombiningindividualpacksonseparatepalletswith di!erentproducts.Thisprocesswasrequiredforcertainordersbecausenotallorders 45
(a) (b) Figure4.16:Palletsbuiltintheshippingdockareloadedontotrucks.(a)Dismantled palletsinthecoolerforbuildingorders.(b)Palletloadinginstructionsforload distributionrequirementsonthetrucks. requiredtheexactquantityofaproductlineononepallet(Figure4.16a).Completedorderswerewrappedwithanautomatedwrappingmachineandloadedonto thedeliverytrucksinaspecicorder(Figure4.16b).Theorderpalletswereloaded intothetruckswasextremelyimportantforweightdistributionandmaxloadlimits. Truckdriversalsohadmultipleordersonasingletruck,andtheordersneededto correspondtotheirdeliveryroute. 4.1.5TheFindings Atthemostbasiclevel,warehousesserveasanintermediarybetweeninbound andoutbounditems.Whilereceivingandshippingprocessesexistinallwarehousesto facilitatetheowofitems,theprocessesbetweenentryanddeparturevarydrastically. Naturally,thevariationisdependentupontheindustryawarehouseisassociated with;processeswithinafurniturewarehousewillvaryfromprocessesinagrocery chainwarehouse.Although,greatdi!erencesdoexistamongstwarehouseswithin thesameindustrygiventhewidevarietyofdecisionspertainingtologistics,storage structure,andworkowtasks. Figure4.17showsthedataabstractionfrominductivecontentanalysis.There arefourteensubcategories,fourmaincategories,andonedataabstractiondescrip46
tion.Thisabstractionidentiesadescriptionoftheresearchtopicandrelatesthe analysisbacktotheresearchquestions,andissummarizedasfactorsthata!ectthe implementationoftechnologyinwarehouses. Factors that may affect the implementation of new technology in warehouse environments Perception Negative Positive Neutral Organization Restructuring Structured Unstructured Environment Communication Work/Task related Social Equipment Manual Powered Layout Temperature Technology Time Quotas Tempo Day Month Week Year Schedules Figure4.17:Adataabstractiondiagramofdatacollectedfromtheethnographic study.Thereare4maincategories,14subcategories,and8furthersubcategories. Factorsthatmaya!ecttheimplementationofnewtechnologyinwarehouseenvironmentsistheoverarchingdescriptionofthecategoriesthatemergedfromthedata. 18.104.22.168Environment Theenvironmentofawarehousewasdenedaseverythingenclosedwithinthe wallsofthewarehouse,bothphysicalandabstract.Thiscategoryemergedfromthe di !erentoccurrencesthatdrovetaskcompletioninthefourwarehouses.Similarities anddi! erencesbetweentheenvironmentshighlighteduniqueformsofcommunication andvaryingequipment,layouts,temperature,andtechnology. 22.214.171.124Environment:Communication Thecompletionofdailytaskswasneverasoloendeavor;itrequiredcommunicationwithothersandtheenvironment.AtDPSandGoodwill,communicationbegan 47
beforethestartofthemorningworkshifts.Managersmetwithemployeestoeither discussdailyworktasks,presentupdatesonlogisticschangesinthewarehouse,and passoutrequiredinformationfortaskcompletion.Onesiteprovidedmoreofan updateonhappeningsinthewarehouse,aswellassomehumoroustriviabeforework o ciallycommenced: Onthatnote,today sleadingcelebritybirthdayisChickNorris.John HammfromMadMenis45.SharonStoneis58.Quickfunnystory, aboutayearintoeCommerce,IfoundanAARPMagazineandthecover girlwasSharonStone.That sthemomentIknewIwasold. Thehumorendedthemorningmeetingonalighternotebeforeworkbegan,asalmost everyemployeeleftthemeetingchucklinganddiscussingSharonStoneandtheother celebritybirthdays.Eventhoughthemeetingwasworkandtask-related,themanager madeane!orttoincludetriviatolightenthemoodandimprovethecommencement ofworkfortheday. Thisisincontrasttotheothersiteswithnodesignatedmorningmeetings.Employeesarrivedandknewtheorderllingtasksthathadtobecompletedforthe day,andbeganworkonthosetasks.However,atOskarBlues,oncetheworkshift began,itwasheavilydependentuponcommunication.Oneemployeesaidyelling wasanecessitytocommunicatetoothersastowhatorderwasbeinglled.While theemployeestatedthismethodofcommunicationwasnotverye" cient,hesaid itwasnecessarytoavoidduplicatingordersandaccommodatetheadditionoflastminuteitemstotheordersfromthesalesteam.Theprocessofllingordersatthis siterequiredbuildingpalletsandloadingthemontofreighttrucksinaspecicorder astoevenlydistributetheload.Theemployeesllingtheordersusedforkliftsand drovearoundquicklytoretrievepallets,rarelystoppingtodiscusswholledwhat butinsteadyellingitoutinreal-time. 48
Communicationwithotherstocompleteworktaskswasalsodependentuponthe employee sfamiliaritywiththewarehouseenvironmentandtaskcompletionprocesses. Onemanagerdescribedtheonboardingprocessas: Firstyou rejustbuildingstu!.4di!erenttypesofwayswepackageour beers.Onceyou recomfortablewiththatandthevariationofbrands wehave,wehave9brands,50something...onceyou recomfortablewith that,you'remoreofanassistant,you'renotontheforklift,you'rebuilding palletsandhelpingthemout,shoutingattheforkliftdrivers.Oncethey getpastthatstage,you'reprettymuchonyourown.Idon'tmicromanage, becauseIdon'thaveto.It'sreallysimpleonceyougetyourheadaround whateverythingis. Asfamiliaritygrows,comfortwithtaskcompletiongrowsandresultsinreduced relianceonaskingothersforhelp.Theleveloffamiliarityislargelydependentupon thestructureofthewarehouseandhowoftenprocesseschange.AtEP,amanager statedittookapproximatelythreeweeksforanemployeetobecomecomfortable withlocatingitemsinthewarehouse.Therewasanincreasedleveloftaskrelated communicationatthissiteduetotheneedtonditems.Evenemployeeswhoworked inthewarehouseforovertenyearsoccasionallyforgotwhereanitemwaslocated, andhadtoaskotherswhereitwaslocated. Whilesomeemployeeswereresponsibleforllingsingleordersindividually,they occasionallycommunicatedtootherstheneedforphysicalassistancetocompletea task.Atonesite,asanemployeewasllinganorder,shecameacrossalargeitemshe wasunabletopullfromtheshelf,"ThisiswhatImeanbyabiggeritem.Thereare itemsthisbigthatarereallyheavy.That'swhatAmandaisfor."Shealsodescribed oneorderasadoubleheader,andchangedthemannerinwhichshecompletedthe orderllingprocess: 49
Researcher: What sadoubleheaderrst? Employee: Twoitemsonasheetinsteadofone.Iusuallydothis.One... makesurethatbothitemsarethere.Sotheshipperwillknowthatthis istwoitems.Ijustliketheshippertoknowexactlywhat'sgoingon.I trytobeveryexactwithwhatIdo. Whilethisformofcommunicationwasnotdirectlywiththeemployee,herphysical methodwasstillawaytoeasilycommunicatewiththeshippingemployees. Communicationwiththeenvironmentwasjustascriticalascommunicationwith otherstocompleteworktasks.Communicationwiththeenvironmentincludeddirectionalsigns(Figure4.18a),informativesigns(Figure4.18c),andvisualcuesto guidetaskcompletion(Figure4.18b).Directionalsignsweredependentuponhow thewarehousewasorganized,althoughatonesite,theywerenotalwaysaccurate inregardtowhereitemswerelocated.Signswerebothhandwrittenandcomputer generatedandorganizedbytheabbreviationofproductlinesorlettersandnumbers toreectbinandshelvinglocations(Figure4.18d). Informativesignsweresignagethatincludedinstructionsonhowtocomplete tasks,whattoavoid,andcurrentupdatesfortheday.Visualcueswerenotwritten signage,butquickremindersforemployeesduringtaskcompletion.Forexample, atDPS,onedeliverytruck sheightlimitwasmuchlowerthantheotherdelivery trucksintheeet.Themaxheightlimitwasmarkedviaablacksharpieontheshelf unitwhereordersforthatroutewerestored(Figure4.19a).Therewasafail-safeif employeeswerenotremindedbythevisualcue,whichpresentedasawrittennotice pointingtoanotherblacklinedenotingthetruckheightlimitbeforethecoolerexit (Figure4.19b). 126.96.36.199Environment:Equipment Worktasksineachwarehouserequiredinterfacingwithsomeformofequipment. Orderswerelledusingcarts,palletjacks,andforklifts(Figure4.20).Thetypeof 50
(a) (b) (c) (d) Figure4.18:Communicationwiththeenvironmentincludedamixtureofdirectional signs,informativesigns,andvisualcues.(a)DirectionalsignsatGoodwilleCommerce.(b)PalletloadinginstructionsatOskarBluesBrewery.(c)Binnumbersat DPS.(d)HandwrittenlabelsatEngine&Performancewarehouse. (a) (b) Figure4.19:Visualcuesrequireknowledgeofwarehousetaskcompletiontobean e ectiveformofcommunication.(a)Blackmarkingsfortheheightlimitofordersfor aspecictruckrouteattheDPSwarehouse.(b)Aninformativesignattheexitof thecoolerincasethevisualcuewasmissed. 51
equipmentusedchangedthroughoutthedaydependingupontheworktasks.At DPS,othertasksoutsideoforderlling,likerestockingandorganizationattheend oftheday,requiredforkliftstomovelargerpallets.Nocerticationwasrequired formanualequipment,butcerticationwasrequiredforforkliftuse.Thelevelof powerwithintheequipmentalsoincreasedthenecessityofheightenedawarenessof immediatesurroundings.Employeesthatusedforkliftsandlargepalletjackswere constantlysurveyingtheirsurroundingsandbeepingtheequipmenthornstonotify othersoftheequipmentpresence.Equipmentcerticationwasalimitingfactorfor taskcompletionforsomeemployeesbecausethosewhowerenotcertiedwereunable tousetheforkliftstocompletetasks. 188.8.131.52Environment:Layout Taskcompletionwasgreatlyinuencedbythelayoutofthewarehouse.The layoutpertainstohowandwhereunitsthathouseproductsarearranged,aswellas thedividesbetweendi!erentareaswithinthewarehouse.AtDPS,thelayoutwas conducivetoaspecicpickpathtoenhanceproductivity.Theaislesprovidedazpickpathandthehorseshoeshapelayoutprovidedawaytoeasilycirclearounditems (Figure4.21a).AtEP,itemstorageonthreeseparateoorsrequiredemployeesto completeadditionalstepsfortaskcompletion.Thisincludedpullingitemsonseparate oorsandsendingthemdownslidestodi!erentlevels(Figure4.21b).Thelayoutat Goodwillwasthemostopenwiththeabsenceofphysicaldividesbetweenthe"hubs" ontheoor(Figure4.21d).Thislayoutmadeitmucheasierforemployeestoidentify whatworktasksneededtobecompleted.ThisissimilartoOskarBluesBrewery, wherenodividesexistedintheshippingareaandcooler(Figure4.21c).Thelayouts atallfoursitesremainedxed;however,atDPSandOskarBlues,layoutsrecently underwentatransformationtoimproveproductivityandstreamlineprocesses. 184.108.40.206Environment:Technology Technologyinthewarehouseenvironmentwasdenedastheinfrastructurethat supportedandmaintainedwarehouseoperationsandincludeddevicesthatintegrated 52
(a) (b) (c) (d) Figure4.20:ManualandpoweredequipmentatDPS,Engine&Performancewarehouse,OskarBluesBrewery,andGoodwilleCommerce.(a)Anelectricpalletjack atDPS.(b)PushcartsatEngine&Performancewarehouse.(c)AforkliftatOskar BluesBrewery.(d)ApushcartatGoodwilleCommerce. 53
(a) (b) (c) (d) Figure4.21:WarehouselayoutsatDPS,Engine&Performancewarehouse,Oskar BluesBrewery,andGoodwilleCommerce.(a)Beginningofz-pickpathinthedry goodssectionatDPS.(b)ThirdoorslideatEngine&Performancewarehouse. (c)OpencooleratOskarBluesBrewery.(d)Openspaceorganizedinto"hubs"at GoodwilleCommerce. 54
withthewarehousemanagementsystem(WMS).Employeesatallfoursitesperformedorderllingandrestockingtaskswithpaperpickingandrestockingsheets. Theabsenceofbarcodescannersrequiredincreasedattentiontodetail,astherewas noautomatedfeedbacktocheckiftaskswereperformedcorrectly.Employeesat DPS,EP,andOskarBluesBreweryrarelyinteractedwithtechnologywhilecompletingworktasks.TheexceptionwastheinventoryspecialistatEP,whowasresponsible forinputtingallincomingproductsintotheWMSonacomputer(Figure4.22). GoodwilleCommercewastheonlysitewhereemployeesinteracteddirectlywith technologyonadailybasis.ShopGoodwillwastheeCommercemarketplacewhere itemsinthewarehousewerelistedbyasubsetofemployees.Beforelistinganitem online,employeesperformeditemresearchtodetermineitsvalue.Theprocessof researchingandlistinganiteminvolvedadesktopcomputer.Fortheseemployees, technologywasanintegralpartoftheirjobresponsibilities. Figure4.22:TheinventoryareaatEngine&Performancewarehousewithacomputer connectedtothewarehousemanagementsystem. 55
220.127.116.11Time Warehouseprocessesareconstrainedbytime.Asonemanagerdescribed,"picking aretheonlydirecthours.That'swhatisgeneratingtheincome.Anythingelseis indirecthoursanditislosttimeprettymuch."Ine"cientprocesses,especiallywithin taskcompletion,wastesignicantamountsoftimeandcannegativelya!ectthe accuracyandtimelyllingoforders.Toensuretimeisspentinthemoste" cientway possible,managerseitherimplementquotas,setschedules,ordenetaskcompletion stepsforemployeestofollow.Quotasandschedulesoftensetthetempointhe warehouse,withtheexceptionofoutsideeventsthatmayaltertheinternalworkow. 18.104.22.168Time:Quotas Quotaswereameansformanagementtoensuretimewasspente"cientlyby employeesandtoenhanceworktaskproductivity.Therewasvariationinthedegree towhichquotaswereenforced,withonemanagerdescribingthequotasattheDPS warehouse: Researcher: Isthereaproductivitythreshold? Manager: Yeah.Sothat sontheproductivitysheet.Sowhenwepick inthefreezerandcooler,wewantthempickingat1,500piecesanhour. Freezerbyitself,onFriday,100casesanhour.Whenwedothedry,it's a104casesperhour.Whenwe rejustdoingthecooler,wedothecooler onit sown,orseparateonFridays,theyneedtobe1990. Thesexedquotaswerecloselytrackedandrequiredtobemetbyallemployees. Ifquotaswerenotfullled,employeeswereauditedbyamanager,whichentailed shadowingtheorderllerduringaworkshift.Thisisincontrasttolenientquotas withdi! erenttaskcompletionrequirements.OneemployeeatGoodwilldescribes: Researcher: Isthereaquotatheywantyoutoreacheachday? 56
Employee: Usuallythereis,althoughI'mgladwehavesomeleniency withthatbecauseifyoudon'thavetheitemsinthesystem,Ihavetogo andphotographthingsandthencomebackandlist.Theyareawareof that. Thedi! erenttaskcompletionrequirementswaskeyforthedegreeofleniencyfor quotas,asanotheremployeeatthesamesitebutwithadi!erentworktaskdescribed certainquotasasunobtainable: Whattheirgoalis,orwhatI'mtold,istheywantustoscan21,000books aday.Andoutofthat,thatwillgiveusmaybe5-7,000acceptablebooks. Theproblemwiththat,iswe'dneedthreemoreslidestoget21,000.On agoodday,weonlyget8-9,000.Forsomereasontheythoughtwecould getmorethroughhere. Thetwowarehouseswithoutquotasweredrivenbythetasksthatneededtobe completedbytheendoftheworkday.Withtheabsenceofquotas,therewasnoway totrackindividualemployeecontributions,whichcouldpotentiallyenhancetraining ifanemployeeneededadditionalhelporwasnotcontributingequally. 22.214.171.124Time:Tempo Tempodenedthepaceofworkowinthewarehouse.Itwaseitherseasonal, monthlyordailydependingupontheindustry.Goodwill seCommercetempowas dependentupontextbookseasonwhentheacademicyearbegan,theholidays,and springcleaning.Thetempoduringtheseseasonalrusheswasextremelyfast,with employeesquicklysortingthroughbooksanditemsdaily,andpullingitemsforshipping.EPalsohadaseasonaltempodrivenbyannualnationalcarshowscustomers preparedfor.ThetempoatDPSwasdependentontheschoolcalendar.Thebusiest timeoftheyearwaswhenschoolbeganinthefall: 57
It sgood,it sfun,itgetsalittlehecticandhightensionsometimes.The beginningoftheschoolyearisbad.Iwouldn tsayworkenvironment bad,butwe regettinghitsohardwithordersinthebeginningbecause theschoolsarestartingtorestockandeverythingisjusthuge.It squite, headdown,andwejustwanttogohome,forabout2-3months. Thisisthecompleteoppositeofthetempodescribedduringsummerbreak,when tasksaremuchmorerelaxedandhousekeepingandlightreorganizationhappensin thewarehouse. 126.96.36.199Time:Schedules Schedulesprovidedtheclosestresemblancetoroutineastheywereaguideline ofwhentasksshouldbecompleted.Whatdi!erentiatesschedulesfromtempoisthe calculatedandpredeterminedtimeframesthatbindaschedule.Whiletempowas moredependentuponexternalforcesinuencingawarehouse,scheduleswerethe timeframessetbymanagersfortaskcompletion. AtDPSandOskarBlues,themorningworkschedulesweresolelyorderlling related,andtheafternoonwasfocusedoncleanupreorganization.Orderpicking inthecooleratDPSwasscheduledforcompletionbetween1:30-2:00pmgiventhe numberofemployeesandpicksperemployeethatwascalculatedatthestartofeach day.AtOskarBlues,theschedulewastocompletelargetruckordersby12pm.After themorningschedulewascompleteatbothsites,theafternoonworktaskswereable tocommence.EmployeesatEPhadasimilarschedule,butitdeviatedaftermorning taskcompletion.EPorderllerscompletedlargetransferordersinthemorning, however,theafternoonschedulewastocompleteadditionalordersinsteadofclean upandrestocking.OnemanagerstatedafternoonsinthewarehouseatEPishectic withemployeesrunningaroundtocompleteremainingordersfortheday. TheGoodwilleCommerceschedulewascenteredaroundemployeeworkshiftsand completingquotas.Theschedulewasdependentuponwhatworkwasavailablethat 58
day.Outsideofthescheduledmorningmeeting,scheduleswerenotdominantand taskcompletionwasdrivenbyquotasandtheoveralltempowithinthewarehouse thatday. 188.8.131.52Organization Organizationfortaskcompletionwasdependentuponthecontinuedpresence orabsenceofmandatesandenvironmentalterationsfrommanagers.Thiscategory drewinspirationfromtheconceptofaconservativeforce,wheretheworkmeasured isindependentofthepathtakenbetweentwopoints(Figure4.23).Certaintasks, especiallythoserelatedtoorderlling,beganatastartpointofapickticketand concludedwithacompletedorder.Theoretically,ineverysituation,therewasa progressionofstepsanemployeecouldtaketocompletetheorderllingprocessinthe moste"cientwaypossible.Thestructureofthestepscompletedfromstarttonish werecontrolledbymanagementwhoeitherrequiredstep-by-steporenvironmental methodsforllingorders,orleftthemethodsfortheemployeestodecide.Thework ofllinganorderwasalwaysthesameandindependentofthemeansusedtocomplete thetask,nomatterhowe" cienttheprocesswas.Organizationwithinawarehouse istargetedatthesemeanstoenhanceproductivityasmuchaspossible. 184.108.40.206Organization:Restructuring Restructuringorganizationconsistsofprocessesthatundergoalterationsfrom managementasaconsequenceofneedanddesire.Optimizingtaskcompletionin awarehouseisoftenanecessitytomeetperformancestandardssetmanagersor supervisors. DPSunderwentintensiverestructuringoverveyearsagowhenamangerwith previousexperienceinhighlyautomatedwarehouseswashired.Themanagerdescribedthestateoftheorderpickingprocessashaving,"norhymeorreason...itwas notveryorganized,"withdailytaskcompletiondescribedas,"probablyonly2ofthe eighthourswerespentpickingforeachofthem.Everythingelsewassettingupand puttingaway." 59
Figure4.23:Aschematicofthevarietyofpossiblepathstollanorder.Themost e "cientpathwithnoerrorsisdenotedwiththebluearrow,andalesse"cientpath withsomeerrorsisdenotedwiththeredarrow.Regardlessofthepathtakentoll anorder,theendresultisalwaysacompletedorder. 60
Alterationstotheenvironmentincludedtheadditionofrackingandassigning everyitemalocationforapickpath.Thedrygoodssectionofthewarehousewasrestructuredtoaz-pickpathwithitemsorganizedaccordingtoweightandsizetoavoid reorganizinganorderafteritwaspicked.Inadditiontoenvironmentalchanges,the managerimplementedproductivitysheetstotrackthee!ectivenessofrestructuring decisionsandprovideinsightforwaystoimproveproductivity: Productivitysheetstracknotonlythedirectpicking,butalsotheindirect hoursthatarespent,tryingtogeteverythingreadybeforeyoucanstart picking.Pickingaretheonlydirecthours.That'swhatisgeneratingthe income.Anythingelseisindirecthoursanditislosttimeprettymuch. Thesesheetswerecarriedonaclipboardbyeveryemployeedailyandrequireddocumentationofwhenpickingforanorderbeganandconcluded.Activities,suchas restockingitemsorsearchingforitems,wasalsodocumentedfortotalingindirect hours.Afterproductivitysheetswereadopted,themanagernoticedrestockingaccountedforalargeamountoflosttimebecauseitwascompletedwhileemployees werellingorders,insteadofinadvanceofllingdailyorders: Theycouldjustpick,takeaslongastheywanttopick,aslongasthey nisheditbytheendoftheirshift,theydidn thaveanystandardsfor anything.Therewasreallynoorganizationinhowtheypicked...ifyou neededsomethinginapickbin,anditwasbackhereinareservebin,they wouldhavetojustdriveupanddowntheaislestondtheitemtoput itintothepickbin.Anditwasrightinthemiddleofthepickingprocess becausethepickersjustwentandgotwhatevertheyneededastheywere pickingit.Sotherewasnostandardsandtherewasnoorganizations, withinthedepartment. Asaresult,themanagerimplementedstockingsheets: 61
ThenextthingIputinplacewasthestockingsheets.Whichallowsusto beabletostockbeforewestartpicking,sooncethepickerstartspicking thereshouldbenoreasonthatthepickerhastostopinthemiddle. Thereorganizationofthelayout,productivitysheets,andstockingsheetsallserved asmajorvehiclestorestructuretheorganizationofthewarehouse.Thedecisionto alterthelayoutofthewarehousewassimilartotherestructuringthatoccurredat OskarBlues,andforsimilarreasons.AsthemanageratDPSdescribed,employees wererequiredtosetupandtakedownitemsforpickingeachdaybeforetheprocesses wererestructured.ThemanageratOskarBluesdescribesthetransition: Inthepast,it[thebeer]laidoutovertheoorandit dbesweating.The coolerwassosmallwehadtopulleverythingoutjusttogettoitems thatwerebehinditanditwouldstayoutfor10hourssweating.Wecan nowstore3timesasmuch.Rotatingstockwasafull2-3hoursoutof someonesshift! Thereorganizationatbothsitessignicantlyreducedtheamountoftime,"wasted," ortheamountoftimenotllingorders.However,eventhoughbothwarehousesunderwentrestructuring,theindividualprocesseswereeitherconformedtoadesignated structure,orremainedlargelyunstructured. 220.127.116.11Organization:Structured Structuredorganizationisarequiredsetofstepsthatmustbecompletedby employeeswhencompletingtasks.Inessence,thisistherightwayforemployees tocompleteworktasks.Astructuredorganizationwasoftenadirectresultofpast restructuring,asevidentattheDPSwarehousewiththez-pickpathandhorseshoe layout.TheOksarBluesBreweryshippingarea,whileitunderwentrestructuringto alterworktasks,didnotchangetheprocessofllingordersonforklifts.Thefreedom 62
tollordershowtheemployeeseestissimilartotheunstructuredapproachatEP, whichwillbediscussedinthenextsection. TheGoodwilleCommercewarehousewasasitethathadastructuredorganization,butitwasnotaresultofrecentrestructuring.Thestructureoftaskcompletion wasmandatedbymanagerstoensureitemswerelistedcorrectlyandforthecorrect price.Theowofitemsrarelydeviatedfromthefollowingworkow;receiving,researchorlisting,photography,storage,retrievalafterpurchase,andshipping.Even thoughemployeesatthewarehouseparticipatedinmorethanoneworktask,there wasalwaysanordertocompletingeachtask.Forexample,thephotographersinthe eCommerceareawererequiredtotakepicturesofitemsacertainway.Fourpictures wereneededperitem,oneoneachside,withtheappropriatelightingandhighlightingofimportantinformationontheitem.Photoqualitywasimportantbecause itemswithpoorqualityphotosresultedinincreasedquestionstocustomerserviceor potentialdisinterestinanitem. 18.104.22.168Organization:Unstructured Unstructuredorganizationenablesemployeesfullpowertodecidehowtheycompletetasksbetweenreceivingapickticketandcompletinganorderandtheenvironmentisconducivetotheabilitytoselectavarietyofmethods.ManagementatDPS andGoodwilleCommerceprovidedspecicrulesfortaskcompletion,whilemanagementatEPandOskarBluesbothgaveemployeesfreedomtochoosetaskcompletion methods. TheunstructuredorganizationatEPwasevidentonceanemployeeselecteda picksheetfromthepickticketstation.Theitemslistedwereinnospecicorder, sotheemployeehadtodetermineiftheitemsweresplitbyoor.Accordingtothe manager,amajorityofthepickticketswereforenginekitswithcomponentsstored ontherstoorofthewarehouse.Twoemployeesshadowedatthiswarehousehad beenemployedforapproximatelytwoweeks,andtheyalreadyhadvastlydi!erent 63
methodsandpaceforlocatingitems.Oneemployeeconstructedhisown"cheatsheet" bylistingproductlinesthatresidedoneachlevel.Healreadyhadconstructedaset pickpathforhimselftollenginekits,andwascompletingordersatafastrate.The otheremployeefeltmuchlesscomfortablewiththewarehouse,andwasoftenasking whereitemswerelocated,andhadnotdevelopedasetpickpathforllingcommon orders.Thiscontrastdemonstratesthestrikingdi! erencesincompletingworktasks andhowtheyaredependentupontheorganizationalstructure. TheorganizationatOskarBlueswassimilartothatofEP,buta! ectedworktasks inadi!erentmanner.Employeeslledlargeorderstogetherontofreighttrucksand thestructureofllingcomponentsoftheorderwasdependentuponwhatothershad completed.Whileemployeesweregiventhefreedomtodesignatehoworderswere lledfrommanagement,thatautonomywasnotpresentamongsttheorderllers themselves.Conformingtheunstructuredorganizationtothegroupobjectivelimited theoptionseachemployeehadforretrievingandloadingitems. 22.214.171.124Perception Perceptionwashowemployeesrelatedtotheirjobsandtaskcompletion,specificallywithpersonalfeelings.Thiscategorywasnotimmediatelyrecognizedinthe data,butemergedafteremployeescontinuedtoprovidehonestopinionsaboutwork. 126.96.36.199Perception:Positive Amixtureofemployeesandmanagersheldhighlypositiveperceptions.One manageratDPShadapositiveperceptiondespitethedi" cultieshemayfacedayto-day: Thereissomuch,especiallywiththecustomerservicepartbecausethere aresomanypeoplethatwehavetodealwith,thatIhavetodealwith, because,mydriversarethefaceofthecompany.AndImyself,thefaceof thecompany.Ihavetodealwithsecretaries,principles,custodians,area suprivisorshere,areasuprivisorsatourservicebuilding,suprivisorsfor 64
ouroverallcustodians,administrativedirectors,executivedirectors,it s just,thereisalotofpeoplethatyoucomeincontactwithandsometimes thatcanbedi" cultbecausepeopletendtobedi" cult.I mapretty handsontypeofpersonandit sachallenge,andIlookforwardtothe challenge.Itmakesitexcitingtoworkhere.Sometimesyouhavetogo aboveandbeyondtohelppeopleout,youknow,o"cesta!here,weget alotoflastminuterequestsforthingsandtotrytokeepourcustomers pleasedwiththeservicewe reproviding. Hisperceptionalsoinuencedhisworktasks,ashewasoftenthego-topersonfor trainingemployees,"ItrytodotrainingwiththeorderselectingbecauseI'mpatient withpeople." AtGoodwill,anemployeeenjoyedhisworktasksforadi!erentreasonoutsideof challenges,"Ilikethecustomerservicedeskbecauseit'salittlebitdi!erenteveryday. Ilikethedi!erentquestions." 188.8.131.52Perception:Negative Negativeperceptionsoftenarosefromintensiverestructuringthatalteredtask completion,especiallyifadditionalworkwasaddedtotasks.Therestructuringatthe DPSwarehousereceivedalotofpushbackfromemployeeswhenitwasimplemented: Itwaseasytocatchon[tothechanges]...butwiththeveterans...anywhereyougo,changeishard.Theyaresetintheirways.Itwasn t receivedwellatthetime,but[themanager]kindofhadmeforefrontthe campaignandsayit'sgood.They dcometomeandI dhavetoexplain it llbebetter,it llbeeasier,buttheyjustdidn tbelieveme.Changeis noteasy. Therestructuringprocessnotonlybroughtaboutchange,butaddedadditionalwork employeeswererequiredtofulllwhilepickingorders.Theseproductivitysheetswere notwellreceived: 65
So,againitwaslike,everytimeyoudidanorderyou dhavetostopand llitoutandeveryonethoughtitwaspointlessandwhyarewedoing this...itwasbadforawhile.Butitworks! ThisisincontrasttoOskarBlues,whereemployeeswereelatedwiththetimesaved llingordersaftertherestructuringprocess.Oneemployeesaiditwaslessstressful andmuchmorestraightforward. 184.108.40.206Perception:Neutral Aneutralperceptionwasindi!erencetothejobandassociatedtasks.With employeesatallsitesexpressingboththepositiveandnegativeaspectsoftheirjobs, thereweresomewiththeattitudeof,"thisjobisjustajob."Employeeswithaneutral perceptioncompletedworktasksontimeandcorrectly,butlackedthepositivitythat wasevidentwithothermanagersandemployees.Oneemployee'sneutralperception washisfeelingstowardsthelackofupwardmobilitywherehewasworking.Employed asanitemlisterfortwoyears,hefelttherewasnoopportunitytoprogressforward, butstillcompletedhisworktasksontimeandcorrectly.Thelackofupwardmobility wasnotanegativeperceptioneither,asitwassimplyacceptedbythisemployee.A negativeperceptioninregardtolackofupwardmobilitywouldlikelyresultinpoor taskcompletion,orleavingthejobentirely. 4.1.6Inter-raterreliability Inter-raterreliabilitymeasuredthelevelofagreementbetweentwocodersthat codedthedatawiththepreviouslydiscussedcodingschemeandndings.Theinterraterreliabilitywasintherangeofslightagreement,withakappavalueof0.14. Thepercentagreementanddisagreementwas91.7%and8.26%respectively.The percentageofthesubjectivecodercodingasourcethattheobjectivecoderdidnot codewas7%,andthepercentageoftheobjectivecodercodingasourcethatthe subjectivecoderdidnotcodewas1.1%.Thepercentchanceofrandomlycodingthe datawas90.4%. 66
4.2UseCases OverallSystemDescription:Create-A-TaskApplication TheCreate-A-Taskmobileapplication(CTA)willcollect,editandstoremediafora non-linearcontextawarepromptingsysteminwarehouseenvironments.Warehouse managersandemployeeswillusetheapplicationtoassistintheinstallmentand maintenanceofthesystemtoenablewarehouseemployeestoperformworktasks. Figure4.24:TheoverallsystemusecasefortheCreate-A-Taskapplication.The systemhassevenmainusecases;Login,CreatePrompt,PromptLibrary,Review Dashboard,DeletePrompt,EditPrompt,andUpdateStatus.Theactorsareeither anemployeeormanager.Thesystemusecasesinteractwiththenon-linearcontextawarepromptingsystem. Figure4.24showstheoverallsystemusecasefortheapplication.Thesevenusecases and14subordinateusecasesarelistedinAppendixG. 4.3InspiredDesign Themobileapplicationdesignincorporatedthe21developedusecases(Figure 4.24)andethnographicstudydata.Theusecasesprovidedguidancesolelyforthe functionalityoftheapplication,butthereremaineddesigndecisionsforhowthe applicationwouldpresentwithmobilefeatures.Thefeaturedecisionsweregrounded intheethnographicndings,withadrivingdesignconsiderationthattheapplication shouldemulatecommonmobileapplicationfeaturesfoundonsmartphones.The 67
Figure4.25:Alistoftheveapplicationfeaturesinformedbytheethnographiceldwork.Eachfeaturewasinspiredbythe specicndingslistedbelowthefeatures. 68
categoriesfromdataabstractionweregroupedtogethertoformdescriptionsofeach feature,whichwerethentransformedintoaphysicalfeature(Figure4.25). 220.127.116.11Feature:Uniqueloginandpasswordforeachuser Aspreviouslydiscussed,drivingfactorsforwarehousesuccessandsustainabilityarellingordersaccuratelyandshippingordersontime.Consequencesfora warehousethatisunabletofulllthesetworequirementsmaybediminishedprots, thelossofcustomers,highemployeeturnover,andahostofothernegativeissues. Themostnotablecharacteristicforeachemployeewasindividualperceptionforwork tasksandwarehouseemploymentasawhole.Relatingperceptiontoafeaturerequiredrevisitingthepurposeofthemobileapplicationdevelopment,whichwasto provideameansformanagersandemployeestoinstallandmaintainpromptmedia foremployeeswithcognitivedisabilities.Ifthepromptmediainthesystemisincorrect,incomplete,orotherwiseine!ectivefortheenduserwithacognitivedisability, thepromptingsystemwouldcontributetoviolationsofthetwocorerequirements ofawarehouse.Employeeswithcognitivedisabilitiesusingthepromptingsystem requirethecorrectmediaandinformation,ortheymaylltheorderincorrectlyand spendexcessiveamountsoftimecallingforassistance. Managersandemployeeswhoviewadditionalworkaspointlessandirritating,or whowouldlikelyusetheapplicationincorrectly,maynotbethemoste! ectiveusersof theapplication.Employeesandmanagerswithapositiveperceptionaboutworktasks maybemoremotivatedtouseapplicationandshouldhaveaccesstothesystemas wellasfeedbackregardingthequalityoftheiteminformationandpromptscollected. Alogininfeaturewillprovideameansofinclusionandfeedbackforemployeeswith positiveperceptions,andexcludeemployeeswithnegativeperceptionswhomaynot usethesystemcorrectly.Theuserinterfaceofthefeaturemimickedtraditionallogin screensfoundonmobiledevicesthatrequiretextinputforausernameandpassword. 18.104.22.168Feature:Fastalterationoptionsinthereviewdashboard 69
Thereviewdashboardfunctionwasoutlinedinthesystemlevelusecase;however,themannerinwhichthereviewdashboardwoulddisplayandupdateerrors wasundened.Iftheorganizationofthewarehouseisunstructuredorundergoing restructuring,thereareboundtobemistakeswithorderllingwhichrequiretheuse ofthereviewdashboard.Giventhepotentialforavarietyofthetotalnumberof errors,aswellastheurgencyinwhicherrorsneedtoberesolved,afastalteration featurewasincorporatedintothereviewdashboard.Thefastestoptionwouldallow userstoeditpromptsanditemsdirectlyinthereviewdashboardbyreducingnavigationtime.Aswipetodeletefeaturewasimplementedinthereviewdashboardto enableafastalterations.Itadditiontoquicklyalteringpromptanditemerrors,the reviewdashboardalterationfeatureallowsausertoquicklyresolvethestatusofan itemanywhereinthewarehouse.Thiswillensureorderllingcontinueswithoutany majordisruptions,andallowcorrectionstobemadeatthedirectsourceoftheerror inthepromptingsystem. 22.214.171.124Feature:Prompteditingwithintheapplication Theabilitytoedititeminformationandmediafornon-linearpromptsinthe applicationisimportantbecauseitallowstheusertoeditpromptsanywhereinthe warehouse.Initiallyaneditingprogramwasgoingtobecreatedinconjunctionwith thisapplication;however,aftertherevelationbyamanagerthatwalkingbackand forthfromalocationinthewarehousetotheo"cewherehiscomputerwaslocated wasasignicantwasteoftime.Theabilitytostandataniteminthewarehouse andcollectandeditmediasoitisreadyforthesystemwillsavetimeandremove thenecessityoflearningyetanotherprogramonthecomputerinadditiontothe application. 70
126.96.36.199Feature:Adaptabletoavarietyofsearchinputs Eachemployeeinthewarehouseisoperatingwithintheunderstandingoftheir own,personalworld.Whenrecallinganitemorattemptingtolocateanitem,the thoughtprocessisuniquetoeachemployee.Oneemployeemightrecallanitem bywhereitwaslocated,whileanotheremployeemayrecallitbytheSKUnumber becausetheyhaveseenitsomanytimes.Thisvariabilitywithmemoryandrecall needstobeaccountedforwithintheapplicationforemployeestonditeasytouse. Adaptingthesearchfeaturetoavarietyofinputswillenableemployeestosearchfor itemsbasedontheirpersonalrecollection.Insteadofonlyincludingtheitemname andSKU,theinformationwillincludethelocation.Ifanemployeedoesnotwantto walkacrossthewarehousetolocatetheitemandSKU,theycantypeinalocation andtheapplicationwilllisttheitemsinthatareaandtheemployeecanlocatethe itemthatway. 188.8.131.52Feature:NavigationBar Theamountoftimerequiredtolearnnewtechnologycandirectlya! ectanemployee sperceptionoftheirworktasks,especiallyifthetechnologyisforanewprocess. Afeaturethatpreservestimebydecreasingthenumberofnavigationalselectionsin applicationsisbottombarnavigation.Abottombarnavigationfeaturewillnotonly savetimefornavigation,butthetimerequiredtolearnthenewtechnology.The bottombarnavigationfeatureiscommonamongstsmartphonesandisusedinnative phone,emailandmusicapplications.Therearemanyotherthird-partyapplications withthefeatureaswell,includingSpotify,Instagram,andGoogleMaps. 4.4MobileApplicationDevelopment TheCreate-A-TaskmobileapplicationprototypewasdevelopedinAndroidStudiousingtherequirementsoutlinedintheusecasesanddesignfeaturesinspiredby theethnographiceldwork.Atotaloftwoopensourcelibrarieswereusedinthe application;BottomBarandSwipeMenuListView.Theselibrariesgreatlyenhanced 71
featuresintheapplication,andbothlicensesareincludedinthesourcecodeat: www.github.com/rfry5/CreateATask. LoginandHomeScreen TheloginscreenistherstscreendisplayedaftertheCreate-A-Taskapplication iconisselected(Figure4.26).Thescreenincludesthenameoftheapplication,a graphicofanindividualauthoringaprompt,andinputeldsforausernameand password.Theusernameandpasswordfortheuserwerepre-registeredusingthe registerbuttonatthebottomofthescreen. Figure4.26:TheCreate-A-TaskapplicationLoginandHomescreen. Thehomescreenisdisplayedaftertheusersuccessfullylogsintotheapplication (Figure4.26).Awelcomemessage,andaninstructiontoselectanoptionarelocated abovevebuttonselections:CreatePrompt,SearchPromptLibrary,ReviewDashboard,andLogout.Eachbuttonnavigatestheusertoanewscreencorresponding tothebuttondescription.Buttoniconswereaddedtocorrespondwithbottombar navigation,whichwillbediscussedinasubsequentsection.Theiconswereselected 72
fromtheAndroidgraphicslibrary,withtheobjectivetocloselyresemblethefunction ofthebuttondescription.Thelogoutbuttonlogstheuseroutofthehomescreen andnavigatesbacktotheloginscreen. CreateTask Thecreatetaskscreenpresentswithavarietyoffeaturesandoptions(Figure 4.27).Therearefourbuttons:"takephoto","takevideo","uploadmedia",and "save."The"takephoto"buttonaccessesthecameraonthedevicetocapturea picture.The"takevideo"buttonaccessesthecameraonthedevicetocapturea video.The"uploadmedia"buttonopensadialogbox,promptingtheusertoselect theoptionofuploadingaphotoorvideofromthegalleryonthedevice.The"save." buttonsavesalloftheiteminformation,media,andpromptcommandlinestothe SQLitedatabaseonthedevice. Inadditiontothefourbuttons,theuserinterfaceincludesmediareviewwindows toreviewthephotocapturedwiththecamera,andreplaythevideorecordedwith thecamera(Figure4.28).Iftheuserisunsatisedwiththephotoorvideocaptured, theyareabletotakeanewpictureorvideothatupdatesinthemediareviewwindow. TheitemnameandSKUeldswereincorporatedbasedoniteminformation usedtolabelwarehouseproducts.Thelocation,description,itemtype,itemsize, andspecicnoteswereaddedtoprovideaddtionalinformationthatmayberequired bythepromptingsystemforunusualitemsinawarehouse.Thepromptcommand linesaretextthatcanbereadintothepromptingsystemforaudiocues. SearchLibrary Thesearchlibraryscreendisplaysselectinformationforitemsandcorresponding mediainthedatabase(Figure4.29).Athumbnailoftheitemisdisplayed,aswellat theitemname,SKU,locationanddescription.Theuserisabletosearchitemsby itemname,SKU,andlocation.Thesearchfeatureusesasearchlter,soastheuser istypingintheitemname,SKU,orlocation,theresultsareautomaticallyupdated 73
Figure4.27:TheCreate-A-TaskapplicationCreatescreen.(a)Accesstothedevicecameratocaptureapictureafterselecting "takephoto."(b)Accesstothedevicecameratorecordavideoafterselecting"takevideo."(c)Drop-downmenustoselectthe itemtypeanditemsize.(d)Thedialogboxthatappearsafter"uploadmedia"isselected.The"uploadphoto"and"upload video"buttonsnavigatetothephotogalleryonthedevice.(e)Scrollingsectiontoaddmultiplepromptcommands. 74
Figure4.28:TheCreate-A-TaskapplicationCreatescreenwithiteminformationand media.Thephotoandvideocapturedwiththecameraappearinthemediareview windows.Thevideocanbeplayedbackinthewindowbyclickingtheplaybutton. (a)Akeyboardappearswhenthetexteldsareselected.(b)Thekeyboardappears forthescrollingpromptcommandlinestextelds. 75
accordingtothetextinput.Thelistdisplayscrollssmoothlythroughalloftheitems, nomatterhowmanyitemsareinthelist.Anyiteminthelistcanbeselected,which navigatestheusertotheedittaskscreen.Thenalfeatureinthesearchlibraryisthe abilityswipetodeleteanitemanditscorrespondingmedia.Oncethedeleteoption isselected,thelistofitemsautomaticallyupdatesandtheitemisremovedfromthe databaseonthedevice. Figure4.29:TheCreate-A-TaskapplicationSearchscreen.(a)Thesearchbaruses atextlterthatautomaticallyupdatesthesearchresultswitheachletterand/or numberenteredbytheuser.Theusercansearchbyitemname,SKU,orlocation. (b)Theswipetodeletefeatureinthesearchlibrary.(c)Thesearchlibrarycanbe viewedbyscrollingthroughitems,andscrollssmoothlyregardlessofthenumberof itemsinthelibrary. EditTask Theedittaskscreenisvirtuallyidenticaltothecreatetaskscreen,withthe exceptionofthesavebutton(Figure4.30).Anupdatebuttonisinitsplaceto prompttheusertoupdatetheexistingiteminformation,insteadofsavinganew item. 76
Figure4.30:TheCreate-A-TaskapplicationEditscreen.TheEditscreenisidentical totheCreatescreen,withtheexceptionofthe"update"buttoninplaceofthe"save" button. ReviewDashboard Thereviewdashboardscreendisplaystheitemscurrentlyaggedforreview(Figure4.31).Aaggeditemmaybeinthereviewdashboardbecausetheincorrectmedia wascollectedfortheitem,oriteminformationhaschangedinthewarehouseandwas notupdatedinthesystem.Thelistdisplayisidenticaltothesearchlibraryscreen, withathumbnaildisplayingapictureoftheitem,aswellasthecorrespondingitem name,SKU,locationanddescription.Anyitemcanbeselected,anditwillnavigate theusertotheeditpromptscreen.Iftheiteminformationandmediaareedited tothecorrectinformation,theusercanswipetomarktheitemas"resolved."This willremovetheitemfromthereviewdashboard.Iftheitemiscurrentlyoutofstock andrequiresremovalfromthesystemuntilrestocking,theusercanselect"outof 77
stock."Thiswillremoveremovetheitemandcorrespondinginformationfromthe searchlibraryandthereviewdashboard. Figure4.31:TheCreate-A-TaskapplicationReviewDashboardscreenwiththeswipe tomarkanitemas"resolved"or"outofstock"feature. BottomBarNavigation Anavigationbarispresentatthebottomofeveryscreen,excludingthelogin andhomescreen(Figure4.32).Thisenablestheusertonavigatequicklybetween screensbyminimizingbuttonclicks.Theiconsusedforthebottomnavigationbar correspondtotheiconsonthehomescreenbuttons,withtheexceptionofthehome button.Textwasaddedunderneatheachicontoassisttheuserwithnavigationif theiconswerenotrecognized. 78
Figure4.32:TheCreate-A-TaskapplicationBottomBar.Thebottombarincludes fourkeywordsandiconsthatmatchthefunctionofthewords;Create,Home,Search, andReview.Thebottombarispresentonallscreens,withtheexceptionoftheHome screen.TheHomescreenincludesthebottombariconsassociatedwiththebutton functionthatmatchthenavigationbuttons. 79
4.5UsabilityTesting Themetricsanalyzedintheusabilitystudyweretimeontask,numberoferrors, ande" ciency.Pre-testsurveydataof11questionswerealsocollectedtorevieweach testsubject'sfamiliaritywithmobiledevices.Theguresnotincludedforthesurvey datacanbefoundinAppendixH. Allvesubjectsweremalewarehousemanagers,ranginginagefrom35to57years old.Everysubjectownedasmartphone,withonesubjectowningbothasmartphone andmobilephone.Foursubjectsownedasmartphoneforoversixyears,whileone subjectonlyownedasmartphonebetween2-4years(Figure4.33a).Allvesubjects usedsmartphonestotext,makephonecallsandtakephotos(Figure4.33c).Onlyfour subjectsusedsmartphonesemail,entertainment,andtakingvideos(Figure4.33c). Threesubjectsusedsmartphonesover20timesperday,whiletheothertwosubjects usesmartphonesonly5-10timesperdayand10-15timesperday(Figure4.33b). Allsubjectsusedtextinputoptionsonsmartphonesmultipletimesperday.Three subjectscapturedphotoswithsmartphonesonceperweek.Onesubjectcaptured photoswithasmartphoneonceperday,andonesubjectcapturedphotoswitha smartphonemultipletimesperday.Subjectsusedsmartphonestorecordvideoless frequently,withthreesubjectsonlyrecordingvideolessthanonceamonth.One subjectrecordedvideosonceperday,andonesubjectrecordedvideosonceeverytwo weeks.Threesubjectsusedothertouchscreendevicesinadditiontosmartphones. Noneofthesubjectswerecolorblind,andallofthesubjectsworeglasses. 80
0-6 Months 6 Months2 Years 2 Years4 Years 4 Years6 Years 6+ Years Subject Frequency 0 1 2 3 4 5 6 How long have you owned a smartphone/mobile phone? (a) 0-5 times per day 5-10 times per day 10-15 times per day 15-20 times per day 20+ times per day Subject Frequency 0 1 2 3 4 5 6 How often do you use a smartphone/mobile phone? (b) TextingPhone CallsEmailEntertainment Taking pictures Taking videos Subject Frequency 0 1 2 3 4 5 6 What do you use your phone for? (c) Figure4.33:Responsesfromthreeoftheelevensurveyquestionsfromallvesubjects. (a)Surveyresponsefordurationofsmartphoneownership.(b)Surveyresponsesfor frequencyofsmartphoneuse.(c)Surveyresponsesforsmartphoneusage. 81
Figure4.34showstheaveragetimetocompleteeachtaskandthedistributionof completiontimesamongstthesubjects.Thelogintaskwascompletedatthefastest rate,withanaveragetimeof1minuteand4seconds.Thesearch,editanddelete prompt(SED)taskwascompletedattheslowestrate,withanaveragetimeof8 minutesand14seconds.Thistaskhadoneoutlier,withonesubjectcompletingthe taskinunder5minutes.Thereviewdashboardtaskwascompletedinanaverageof 2minutesand27seconds,andthecreateprompttaskwascompletedinanaverage of8minutesand11seconds.Thecreateprompttaskhadanoutlieraswell,withone subjectcompletingthetaskinapproximately14minutes. Task LoginCreateS/E/DReview Time (minutes) 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Time On Task Data points Average time Figure4.34:Boxplotoftimespentcompletingeachtaskwithasamplesizeofn=5. Individualdatapointsforeachsubjectarelistedinblue,andtheaveragetimeto completeeachtaskislistedinred. Asdiscussedinthemethods,criticalerrorsweredenedasincorrectlycompleting ataskwhilethinkingitwascompletedcorrectly,oraskingforverbalassistance.Noncriticalerrorswerenavigationalerrorsanddeviationsfromtheshortestcompletion path.Non-criticalerrorsdidnotincludetyposthatwerecorrectedbeforeexitingthe texteld.Theaveragenumberofcriticalerrorswashighestforthesearch,editand deletetask,withanaverageof3.6criticalerrors(Figure4.35).Thereviewdashboard taskhadthelowestaveragenumberofcriticalerrorsat0.4ofacriticalerror(Figure 82
Task LoginCreateS/E/DReview Number of Critical Errors 0 1 2 3 4 5 6 Number of Critical Errors for Each Task Data Points Average Number of Errors Figure4.35:Boxplotofthenumberofcriticalerrorsforeachtaskwithasamplesize ofn=5.Individualdatapointsforeachsubjectarelistedinblue,andtheaverage numberofcriticalerrorsislistedinred. 4.36).Theaveragenumberofcriticalerrorsforthecreateandlogintaskswere1.2 and0.8errorsrespectively.Thetaskwiththehighestaveragenumberofnon-critical errorswasthesearch,editanddeletetask,withanaverageof9.8errors.Thetask withthelowestaverageofnon-criticalerrorswasthelogintask,withanaverageof1.4 errors.Theaveragenumberofnon-criticalerrorsforthecreateandreviewdashboard taskswere2.8and6.2errorsrespectively. Theafter-scenarioquestionnaires(ASQ)werescoredonasevenpointscale,with 1correspondingto"stronglyagree"and7correspondingto"stronglydisagree."From theASQsurveyscompletedaftereachtask,usersatisfactionforeaseofcompletion wasgreatestacrosseachtask,withtheexceptionofthecreateprompttask(Figure 4.37).Thelogin,search,editanddelete,andreviewdashboardtaskhadanaverage overallsatisfactionforeaseofcompletionscoreof2.4,3.4and2.2respectively.Forthe createtask,easeofcompletiontiedwithtimetocompletethetaskwithanaverage scoreof3.8(Figure4.37b). Usersatisfactionwaslowestinthetimerequiredtocompletethetaskforthe loginandsearch,editanddeletetasks(Figure4.37).Theaveragescoresinuser satisfactionforthesetaskswere3.4and4.2respectively.Forthecreateandreview 83
Task LoginCreateS/E/DReview Number of Non-Critical Errors 0 5 10 15 20 Number of Non-Critical Errors for Each Task Data Points Average Number of Errors Figure4.36:Boxplotofthenumberofnon-criticalerrorsforeachtaskwithasample sizeofn=5.Individualdatapointsforeachsubjectarelistedinblue,andtheaverage numberofnon-criticalerrorsislistedinred. dashboardtasks,theuserinterfacehadthelowestusersatisfactionwithanaverage scoreof4.2and3.0respectively(Figure4.37b). AveragingallASQscoresacrossthefourtasks,usersatisfactionwasgreatestfor easeofcompletionwithanaveragescoreof3.0(Figure4.38).Usersatisfactionwas lowestfortheuserinterfacewithanaveragescoreof3.6.TheaverageASQscorefor thetimetocompletethetaskswas3.5. Theaveragesystemusabilityscale(SUS)scorewas67.5,withalowscoreof35 pointsandahighscoreof95points(Figure4.39).Thestandarddeviationofthe SUSscoreswas 24points,andthemedianSUSscorewas77.5. 84
Ease of CompletionTime to CompleteUser Interface Strongly Agree 2 3 4 5 6 Strongly Disagree Overall Satisfaction for Login Task ASQ Score Data Points Average ASQ Score (a) Ease of CompletionTime to CompleteUser Interface Strongly Agree 2 3 4 5 6 Strongly Disagree Overall Satisfaction for Create Task ASQ Score Data Points Average ASQ Score (b) Ease of CompletionTime to CompleteUser Interface Strongly Agree 2 3 4 5 6 Strongly Disagree Overall Satisfaction for Search, Edit and Delete Task ASQ Score Data Points Average ASQ Score (c) Ease of CompletionTime to CompleteUser Interface Strongly Agree 2 3 4 5 6 Strongly Disagree Overall Satisfaction for Review Dashboard Task ASQ Score Data Points Average ASQ Score (d) Figure4.37:ASQscoresforthelogin,create,search/edit/delete,andreviewdashboardtaskswithasamplesizeofn=5. Individualdatapointsforeachsubjectarelistedinblue,andtheaverageASQscoreislistedinred.(a)BoxplotofASQscores forlogintask.(b)BoxplotofASQscoresforcreatetask.(c)BoxplotofASQscoresforsearch,editanddeletetask.(d)Box plotofASQscoresforreviewdashboardtask. 85
Ease of CompletionTime to CompleteUser Interface Strongly Agree 2 3 4 5 6 Strongly Disagree Overall Satisfaction for All Tasks ASQ Score Data Points Average ASQ Score Figure4.38:BoxplotofASQscoresforalltaskswithasamplesizeofn=5.Individual datapointsforeachsubjectarelistedinblue,andtheaverageASQscoreislistedin red. Subject 1Subject 2Subject 3Subject 4Subject 5 SUS Score 0 20 40 60 80 100 Subject SUS Scores Mean SUS Score Standard Deviation Figure4.39:Systemusabilityscale(SUS)scoresfromallvesubjects. 86
5.DiscussionandFutureWork 5.1Ethnographically-InformedDesign Theethnographicstudynotonlyprovidedaplethoraofinformationtoinspire designfeaturesfortheCreate-A-Task(CTA)application,butalsoinformationon worktasksandemployeerelationswithinavarietyofwarehouses.Theinter-rater reliabilityoftheinductivecontentanalysisresultswasmeasuredwithCohen skappa, andtheinspireddesignresultswereincorporatedintothedevelopmentoftheapplicationforusabilitytesting.Whiletheseresultswerebothundertheumbrellaof theethnographicstudy,theyprovidedinsighttoentirelydi!erentcomponentsofthis research. Thelowkappavalueof0.14suggestsonlyslightagreementbetweenthetwo coders.Thislowagreementislikelytheresultofusinganobjectivecodertocode thedata.Theprimarycoder,whospent25hoursintheeld,codedmorenodes inthedatacomparedtotheobjectivecoder,suggestingexperiencefromtheeld providedinformationthatwasnotunderstoodbytheobjectivecoder.Futurework shouldincludetestinginter-raterreliabilitywithsubjectivecoderswhospendanequal amountoftimeintheeldanddevelopthecodingschemetogether. However,seekingtoimproveinter-raterreliabilitywillnotnecessarilya!ectthe ethnographically-informeddesignresults.Whileitispossibleforfutureworkto uncovernewndingsandrenetheexistingcodingcategories,thenewndingsmay notbeabletodirectlya!ectthedesignofthemobileapplication.Forexample,more informationaboutthecultureamongstwarehouseemployeesmaysurface,butits relationtoinformdesignmaybeslight.Inaddition,theheartoftheethnographicallyinformeddesignprocessistransformingeldworkresultsintodesignchoicesinthe absenceofaformalmethod(butonlyaugmentingtraditionalmethodsandneveras astandaloneapproach).Thatis,thedesignchoicesmaybemeticulouslyjustied andcloselymirrortheethnographicndings,butthereisnosurereformulathatall 87
researchersusetotranslatequalitativedataintomeaningfuldesignchoices.Therefore, giventhenuancesofethnographyintheareaofdesign,amixedmethodsapproach maybemoreusefulforelicitingadditionalinformationmovingforwardwiththe eldwork.Futureworkshouldincludeanincreasedfocusonthecompletionofwork taskswithintheenvironmentsthroughthelensofactivitytheoryandactivityanalysis, aswellasdistributedcognition. DespitethelowCohen'skappavalue,thedesignfeaturesinspiredbytheethnographicstudytranslatedwellintotheCreate-A-Taskapplication.TheSUSscore of67.5washighforarstprototypeandtherewaspositivefeedbackaboutfeature familiarityfromallvesubjects.Initially,subjectswereunabletondthe"swipeto delete"featureinthesearch,editanddeletetask,whichwasalsothetaskwiththe highestaveragecriticalerrorrateof3.6errors.Thiswaslikelyadirectresultofthe vaguetestinstructionsof"deletetheitem,"insteadof"swipetodeletetheitem." Thiswasacriticalerrorforallvesubjects,althougheachofthemexpressedtheir knowledgeofthefeatureandthatbrieystatinghowtodeletetheitemwouldenable themtolocatethedeleteoptionwithease.Thisissupportedbythenumberofcritical errorsinthereviewdashboardtask,withnosubjectrequiringassistancetolocate the"resolve"and"outofstock"swipefunctions.Onesubjectenjoyedthisfeature themostbecauseitwaslargeanddidnotrequirereadingsmallerbuttons.Eachsubjectalsoexpressedtheywouldfeelcomfortablewiththedeviceafterabrief,informal training.Futureworkshouldmeasurelearnabilityinadditiontoothermetrics,as theremaybeadecreaseincriticalerrorsandtimeontaskforfeaturesthatusersare familiarwith.Acontrolgroupofuserswhoarenotwarehousemanagersshouldalso beincludedinfutureworktoprovideabasisforcomparisonfortheusabilitydata. Thesearchfeatureandbottombaralsoneedfurthertesting,asusersdidnotuse themfrequentlyduringtaskcompletion.Thisislikelyduetothesegmentationof thetasksandthesizeofthepromptlibrarywhereuserscouldsimplyscrolltolocate 88
thecorrectitem.Thefeaturesintheapplicationthatwereinspiredbytimeshould alsobetestedundercircumstancesmoresimilartotaskscompletedinawarehouse. Thismightincludelocatingexistingitemsindi! erentaislesandbins,aswellasitems spreadthroughouttheentiretyofthewarehouse.Extendedandmorestrenuoustasks maypresentmoreofane!orttosavetimeusingtheapplication,whichmaytranslate intousingthosefeatures.Thesubjectswhousedthesearchfeatureonlysearched bytheitemnameinsteadofitsSKUorlocation,whichislikelyduetothelinear natureoftheinstructions.Totestthesearchfeature sabilitytosearchaccording tohowusersrecalliteminformation,itemsemployeesarefamiliarwithshouldbe includedinfuturestudiesandpresentedasgenericallyaspossibleintheusability testinstructions. Thesesuggestionsforfutureworkhighlightthedisconnectbetweensuccessfully fulllingtaskswiththeapplicationandoverallapplicationutility.Whileusersmay completetasksquicklyandwitheaseduringatestscenario,knowledgeofthetrue utilityoftheapplicationisabsentwithoutthecontextoftheactualwarehouseenvironment.Theethnographically-informeddesignapproachaimedtoaddressthis disconnectbyinformingdesignfeaturesonthebasisofthecontextofthewarehouse environment;however,itwasnotmeasureddirectlywithusabilitytesting.Future workshouldincludedeployingaprototypeinawarehousewithoutdisruptingor changinganycomponentofthewarehouseandhowitoperates.Aeldstudylike thiswouldnotonlypresentanopportunitytoelicitmoreinformationaboutthepreviouslymentionedfeatures,buttheuseoftheentireapplicationinthecontextofthe warehouseenvironment. Ausabilitytestintheeldmayalsohighlighthowtheapplicationcancontribute valuetoawarehousecustomerbase.Nomatterhoweasytheapplicationistouse, orhowrenedandenhancedthefeaturesbecome,itsadoptionwillsu! erwithout contributingtoastrongvalueproposition.Theintrinsicvalueoftheapplicationis 89
toenablewarehousestohireadditionalemployeeswithcognitivedisabilitiesthrough supportofanautomatedjobcoachsystem,butdoesnotincludespecicvalueforthe warehousemanagementcustomersegment.Thevaluecanbebolsteredbypresenting anapplicationthatintegratesandisbenecialtoothertechnologyinwarehouses, suchasinventorymanagementapplicationsandtechnologythatinterfaceswiththe warehousemanagementsystem.Strengtheningthevalueofthisindividualcomponent ofthenon-linearcontext-awaresystemwillinturnstrengthenthevalueproposition ofthesystemasawhole. 5.2MobileApplicationDevelopmentandUsabilityTesting Usabilitytestinginrelationtothefunctionalityoftheapplicationuncoveredtwo bugsthatrequirecodealterations.Overall,theapplicationfullledthefunctional requirementswithonlytwoinstancesoftheapplicationcrashing.Theusecases generatedinthisstudywerecorefunctionalitiesofnon-linearpromptstructuresthat includedinherentcharacteristicsofitemsinwarehouses.Thefunctionalitywilllikely changeafterdevelopmentandtestingoftheinteractivepromptingplatform(IPP) toincludeadditionalrequirements,suchasintegrationwithothersystems.Future requirementsmayalsochangetoreectthevaryinglevelsofautomationandthe di !erentprocessesemployeescomplete.Afewofthetaskcompletionstepsobservedin theeldforthisstudythatwerenotincludedintheCTAweretheasset-management methodof"rstinrstout"(FIFO),adjustingthequantityofitemsthatcanbe pulled,howtostackandorganizeordersonpalletsandcartsforshipping,securing specicitems,andhowtheorderwillbeprocessedintheshippingarea. Feedbackfromtheafter-scenarioquestionnaire(ASQ)showstheuserinterface shouldberevisitedandthenon-criticalerrorratesuggestsnavigationfeaturesshould morecloselymimicthoseonsmartphones.Themostcommonlyobservednavigation errorwasswipingonthedevicetohidekeyboard.Theabilityfortheinterfacetoadapt tousabilityfeaturesinsmartphonesshouldalsobeexplored,specicallyincreasing 90
fontandbuttonsize.Fourofthevesubjectsexpressedadesireforlargerbuttons, withonesayingheusedtheincreaseduserinterfacesizeontheiPhone. 5.3FutureIntegrationwithRERC-ACTIIItechnologies Apprehensionsforthisresearchmaybeoftheform,"whydevelopthecollection applicationbeforethenon-linearpromptingsystem?"IfthepromptmediaisdependentuponthestructureandneedsoftheIPP,itmayseemdevelopmentofthe mediacollectionmethodisplacingtheproverbialcartbeforethehorse.Developing theCreate-A-Taskapplicationwascriticalbecauseitelicitedfeedbackfromtheend userasearlyinthesystemdevelopmentphaseaspossible.Finaldeploymentofthe IPPsystemwouldlikelysu! erwithoutsignicante! ortstoincorporaterequirements fromendusersrunningthesystem.Thisstudysuggeststhatdevelopinganapplicationthatcloselymimicscurrentsimplefeaturesonmobiledeviceswilllikelyhavea positiveoutcomewiththeenduser.Continuationofthisworkwillhelpensurethe IPPisusedtoitsfullpotentialbythegreatestnumberofnon-technicaluserswhen developmentiscomplete. 91
6.Conclusion Amobilemediacollectionandeditingapplicationforwarehousemanagerstoinstallandmaintainnon-linearpromptinformationandmediawasdevelopedbycombiningtraditionalrequirementselicitationmethodsandethnographically-informed systemdesign.Thisresearchwasacomponentofanon-linear,context-awaresystemthatwillenableadultswithcognitivedisabilitiestocompetitivelyperformwork tasksinawarehouseenvironment.Twentyvehoursofethnographiceldworkwas completedinfourwarehousestoinspiredesignforveapplicationfeatures,andinterviewswithkeystakeholderswereusedtogeneratethefunctionalrequirements representedwith21usecases.Theinter-raterreliabilitycalculatedwiththequalitativedatafromtheeldworkwas0.14withanobjectivecoder,suggestingonlyslight agreement.ThemobileapplicationwasdevelopedinAndroidStudioanddeployed onanLGNexus5device.Theinformeddesignapproachwasmeasuredindirectly withusabilitytesting,whichshowedsuccessfulfunctionalityoftheapplicationand highusersatisfactionwithanSUSscoreof67.5. 92
REFERENCES  S.Stoddard,"2014DisabilityStatisticsAnnualReport,"2014.  L.Lindstrom,B.Doren,andJ.Miesch,"Wagingaliving:Careerdevelopment andlong-termemploymentoutcomesforyoungadultswithdisabilities," Exceptionalchildren ,vol.77,no.4,pp.423434,2011.  B.S.Barnow,"Theemploymentrateofpeoplewithdisabilities.," MonthlyLabor Review ,vol.131,pp.4450,2008.  R.KoberandI.R.C.Eggleton,"Thee! ectofdi! erenttypesofemployment onqualityoflife.," Journalofintellectualdisabilityresearch:JIDR ,vol.49, no.October,pp.756760,2005.  G.N.Siperstein,R.C.Parker,andM.Drascher,"Nationalsnapshotofadults withintellectualdisabilitiesinthelaborforce," JournalofVocationalRehabilitation ,vol.39,pp.157165,2013.  B.ofLaborStatistics,"BLSInformationGlossary."Available:http://www.bls. gov/bls/glossary.htm#E,2016.[Online].Accessed:Nov.132016.  D.Gilbride,J.S.Mitus,J.Coughlin,andV.Scott,"Theconsortiumforemploymentsuccess:Collaborationasastrategytooptimizeemploymentoutcomes forpeoplewithdisabilities," JournalofRehabilitation ,vol.73,no.3,pp.4555, 2007.  H.J.Bitterman,"StateandFederalGrants-in-Aid,"pp.116,2006.  R.Silverstein,"AnOverviewoftheEmergingDisabilityPolicyFramework:A GuidepostforAnalyzingPublicPolicy," IowaLawReview ,vol.85,pp.1757 1776,2000.  R.Silverstein,G.Julnes,andR.Nolan,"Whatpolicymakersneedandmustdemandfromresearchregardingtheemploymentrateofpersonswithdisabilities," BehavioralSciencesandtheLaw ,vol.23,pp.399448,2005.  D.Stapleton,B.O.Day,andD.Wittenburg,"TickettoWorkattheCrossroads :ASolidFoundationwithanUncertainFuture,"no.202,2008.  D.C.StapletonandR.V.Burkhauser,"HastheEmploymentRateofPeople PolicyBrief,"no.December,2004.  J.KregelandD.H.Dean,"Shelteredvs.SupportedEmployment:ADirect ComparisonofLong-TermEarningsOutcomesforIndividualswithCognitive Disabilities,"pp.6384,2000.  P.Wehman,W.G.Revell,andV.Brooke,"CompetitiveEmployment:HasIt Becomethe"FirstChoice"Yet?," JournalofDisabilityPolicyStudies ,vol.14, pp.163173,2003. 93
 A.Migliore,D.Manka,T.Grossia,andP.Rogan,"Integratedemploymentor shelteredworkshops:Preferencesofadultswithintellectualdisabilities,their families,andsta! ," JournalofVocationalRehabilitation ,vol.26,pp.519,2007.  B.HernandezandC.Keys,"EmployerAttitudesTowardWorkerswithDisabilitiesandtheirADAEmploymentRights:ALiteratureReview,"  P.Yeager,H.S.Kaye,M.Reed,andT.M.Doe,"Assistivetechnologyandemployment:experiencesofCalifornianswithdisabilities.," Work ,vol.27,pp.333 344,2006.  A.J.LangtonandH.Ramseur,"Enhancingemploymentoutcomesthrough jobaccommodationandassistivetechnologyresourcesandservices,"pp.2737, 2001.  M.L.Wehmeyer,S.B.Palmer,S.J.Smith,W.Parent,D.K.Davies,and S.Stock,"Technologyusebypeoplewithintellectualanddevelopmentaldisabilitiestosupportemploymentactivities:Asingle-subjectdesignmetaanalysis," JournalofVocationalRehabilitation ,vol.24,pp.8186,2006.  C.Sarmiento,"GoodwillWorkstoSupportEmploymentofPeople withDisabilities."Available:http://www.goodwill.org/press-releases/ goodwill-works-to-support-employment-of-people-with-disabilities,2013. [Online].Accessed:Nov.132016.  J.BartholdiandS.Hankman Available:http://www2.isye.gatech.edu/jjb/wh/book/editions/whsci-0.96.pdf  M.L.MarkusandC.Tanis,"TheEnterpriseSystemExperienceFromAdoption toSuccess," FramingtheDomainsofITManagement:ProjectingtheFuture ThroughthePast ,pp.173207,2000.  Y.Ghazizadeh,"TotalCostofOwnership:FactorstoConsider."Available:https://software.intel.com/en-us/articles/ total-cost-of-ownership-factors-to-consider,2009.[Online].Accessed:Nov. 132016.  Ieee, GuidetotheSoftwareEngineeringBodyofKnowledgeVersion3.0(SWEBOKGuideV3.0) .2014.  M.Berg,"ThePoliticsofTechnology:OnBringingSocialTheoryintoTechnologicalDesign," Science,Technology&HumanValues ,vol.23,no.4,pp.456490, 1998.  F.D.DavisandV.Venkatesh,"Towardpreprototypeuseracceptancetesting ofnewinformationsystems:implicationsforsoftwareprojectmanagement," EngineeringManagement,IEEETransactionson ,vol.51,no.1,pp.3146,2004. 94
 A.Taylor,"ITprojects:SinkorSwim," TheComputerBulletin ,vol.42,no.January,pp.2426,2000.  TheStandishGroup,"TheStandishGroup:TheChaosReport," ProjectSmart p.16,1995.  C.HeathandP.Lu!,"CollaborationandControl:CrisisManagementandMultimediaTechnologyinLondonUndergroundLineControlRooms," Computer SupportedCooperativeWork ,vol.1,no.1,pp.6994,1992.  I.Sommerville,T.Rodden,P.Sawyer,R.Bentley,andM.Twidale,"Integrating ethnographyintotherequirementsengineeringprocess," ProceedingsoftheIEEE InternationalSymposiumonRequirementsEngineering ,1993.  "Ethnographically-informedsystemsdesignforairtra"ccontrol," Cscw'92 pp.123129,1992.  S.Reddivari,A.Asaithambi,N.Niu,W.Wang,L.D.Xu,andJ.-R.C.Cheng, "Ethnographiceldworkinrequirementsengineering," EnterpriseInformation Systems ,no.July,pp.123,2015.  K.SchmidtandL.Bannon,"ConstructingCSCW:TheFirstQuarterCentury," ComputerSupportedCooperativeWork(CSCW) ,vol.22,pp.345372,2013.  J.BlombergandH.Karasti, Reectionson25YearsofEthnographyinCSCW vol.22.2013.  J.BlombergandM.Burell,"Anethnographicapproachtodesign,"in The Human-ComputerInteractionHandbook ,pp.7194,2012.  L.A.Suchman,"PlansandSituatedActions:TheProblemofHumanMachine Communication.," ContemporarySociology ,vol.18,p.414,1989.  D.Martin,J.Bowers,andD.Wastell,"Theinteractionala!ordancesoftechnology:anethnographyofhuman-computerinteractioninanambulancecontrolcentreDepartmentsof1Psychologyand2ComputerScienceUniversity ofManchester,UKCentreforUser-OrientedIT-Design(CiD)Department," Science ,pp.113,1997.  J.PycockandJ.Bowers,"Gettingotherstogetitright:anethnographyof designworkinthefashionindustry," ComputerSupportedCooperativeWork pp.219228,1996.  J.Hughes,V.King,T.Rodden,andH.Andersen,"MovingOutfromtheControl Room:EthnographyinSystemDesign," CSCW'94Proceedingsofthe1994 ACMconferenceonComputerSupportedCooperativeWork ,pp.429439,1994.  D.R.Millen,"RapidEthnography:TimeDeepeningStrategiesforHCIField Research," Proceedingsoftheconferenceondesigninginteractivesystems:Processes,practices,methods,andtechniques ,pp.280288,2000. 95
 G.FitzpatrickandG.Ellingsen, Areviewof25yearsofCSCWresearchin healthcare:Contributions,challengesandfutureagendas ,vol.22.2013.  V.Pile,"ArchitectingTacitInformationinConceptualDataModelsforRequirementsProcessImprovement,"2013.  L.Plowman,Y.Rogers,andM.Ramage,"Whatareworkplacestudiesfor," ProceedingsofECSCW ,1995.  B.NuseibehandS.Easterbrook,"Requirementsengineering:aroadmap," ICSE '00ProceedingsoftheConferenceonTheFutureofSoftwareEngineering ,vol.1, pp.3546,2000.  H.Robinson,J.Segal,andH.Sharp,"Ethnographically-informedempiricalstudiesofsoftwarepractice," InformationandSoftwareTechnology ,vol.49,pp.540 551,2007.  J.E.BardramandT.R.Hansen,"Context-BasedWorkplaceAwareness," ComputerSupportedCooperativeWork(CSCW) ,vol.19,pp.105138,2010.  J.E.BardramandT.R.Hansen,"WhythePlanDoesn'tHoldaStudyof SituatedPlanning,ArticulationandCoordinationWorkinaSurgicalWard," Proceedingsofthe2010ACMConferenceonComputerSupportedCooperative Work ,pp.331340,2010.  J.BardramandA.Doryab,"Activityanalysis," ProceedingsoftheACM2011 conferenceonComputersupportedcooperativework-CSCW'11 ,pp.455464, 2011.  S.Easterbrook,J.Singer,M.-A.Storey,andD.Damian,"SelectingEmpirical MethodsforSoftwareEngineeringResearch," GuidetoAdvancedEmpiricalSoftwareEngineering ,pp.285311,2008.  F.Shull,J.Singer,andD.I.K.Sjberg, Guidetoadvancedempiricalsoftware engineering .2008.  S.EloandH.Kyngas,"Thequalitativecontentanalysisprocess," Journalof AdvancedNursing ,vol.62,no.1,pp.107115,2008.  J.Lazar,J.H.J.Feng,andH.Hochheiser, Researchmethodsinhuman-computer interaction .2010.  A.J.VieraandJ.M.Garrett,"UnderstandingInterobserverAgreement:The KappaStatistic," FamilyMedicine ,vol.37,no.May,pp.360363,2005.  D.DeWalt,N.Rothrock,S.Yount,andA.Stone,"EvaluationofItemCandidates:ThePROMISQualitativeItemReview," MedCare ,no.May,2007.  Q.International,"WhatisNVivo?."Available:http://www.qsrinternational. com/what-is-nvivo,2016.[Online].Accessed:Nov.132016. 96
 G.SchneiderandJ.Winters,"ApplyingUseCases:APracticalGuide."Available:http://www.ibm.com/developerworks/rational/library/content/legacy/ parttwo/1000/0670/0670 Schneider Ch07.pdf,2001.[Online].Accessed:Nov. 132016. 97
Warehouse Interview Questions General Questions How long have you been working here? What's been the most challenging part of running the warehouse? What are your thoughts on warehouse automation? What does your job entail on a day to day basis? What do you love about your job? Technology related questions What warehouse management system are you currently using? What have you used in the past? How often does this change? What picking method are the order fillers using? (according to a warehouse repor t, there are many options for this. Some include: bar code scanning, voice directed picking, pick to light systems, cart based picking, conveyor based picking and more ) What has the prog ression of technol ogy been like at this warehouse? What kinds of skills would you say are required to use this technology? Do your employees have to use other kinds of technology in the warehouse? ( computers, printers, technology to punch in and out of work ) Are there certain technologies that you do not allow in the warehouse that might affect o ur system (headphones or audio output, tablets for visual cues, a specialized cart, etc.) Warehouse specific questions What is the general layout of your warehouse? Does this change often? How large is your warehouse (square feet)? Do you have seasonal products that affect the organization of the warehouse? When you have a large shipment or new product come in, do yo u ever place them in the walkways? Do the same products always go on the same shelves? How do your employees know where thi ngs are shelved? (memory, signs ) Has any activity profiling been performed? Are any forklifts or other vehicles used within your warehouse? What are some big safety requirements that everyone here is required to follow? How loud does the warehouse get? Employee related questions What is the hiring process like for order filler positions? In your experience, what is the educational level of people applying for order filler jobs? What do you look for in an employee? What are basic skills they need to be successful? What are some basic tasks that employee s perform on a day to day basis? Do employees receive training once they are hired? Do you notice that employees have difficulty using the technology? No difficulty at all? Do employees work independently or in groups? Has your company ever hired someone with a disability before? If so, what kinds of accommodations were made?
Code Definitions Main Category Factors that may affect the implementation of technology in warehouse environments Generic and Sub Categories Environment The environme nt consists of everything enclosed within the four walls of the warehouse, both physical and abstract. Communication Communication is a direct or indirect person to person interaction, or an environment to person interaction. Both interactions can eithe r be work and task related, or social. Equipment Equipment is the means to complete tasks and processes within a warehouse. This includes entire warehouse process, from receiving items from the loading dock, stocking items, moving items around the warehou se, picking items, and shipping items. Layout The floorplan of the warehouse how items are stored and organized within the warehouse. Temperature The temperature of the warehouse environment. Technology Technology is the infrastructure that maintains w arehouse operations. It includes the warehouse management system, and any other technology that track s inventory from start (receiving) to finish (shipping) Time Time is the measurement that marks efficiency and the overall workflow within the warehouse. Quotas Quotas are metrics set by warehouse management to promote employee productivity and increase overall warehouse efficiency. Quotas are set by management for empl oyees to fulfill. Tempo The tempo is t he pace in the warehouse down to and individual employee and up to all of the warehouse operation processes. Tempo ranges from relaxed and/or slow to hectic and/or extremely busy. Schedules Schedules are set by warehouse management for warehouse employees. The schedules dictate the flow of work tasks and responsibilities for the day. The schedules vary from across the board requirements that must be met at certain time points or simply comple ting all of the work that needs to be done until the day is over. The daily schedule may fluctuate due to extenuating circumstances, but these alterations are not the greatest cause for concern. The most stringent schedule is ensuring all of the necessary work is completed by the end of the day.
Organization O rganization is the rigor and detail of structure on the management level for the entire warehouse It includes warehouse processes on a large scale, as well as processes involving employees (training ) technology (pick sheets) and warehouse products (inventory) Restructuring Restructuring is born e out of both need and desire. It defines management that is consistently looking for ways to improve warehouse efficiency and performance. It also defines management that is looking to alter processes due to certain circumstances, such as an increase in size. Structured A structured organization is one where management has fixed expectations and processes that do not fluctuate. There are set pick p ath s and process instructions that each employee is expected to follow Unstructured With unstructured organization, there is great fluctuation with processes. While the basic premise of a warehouse remains unchanged (products in and products out), every thing else is subject to change. Perception Perception relates to a person's feelings and ideas about their job as a whole. This ranges from the finer details of their designated work tasks, to the overall larger picture of warehouse operations as a whol e Positive A positive perception reflects as taking pride in one's work and work ing hard. Positive perceptions are also n oticed by managers. Negative A negative perception reflects as i nefficiency and and a refusal to complete tasks in the designated manner Neutral A neutral perception reflects as just another job. A person may be willing to complete tasks outside of what they are expected, but they are indifferent in doing so.
Subject ID#__________________ 1) Do you own a smartphone? If so, write which model. Yes No ___________________________________ 2) If you do not own a smartphone, do you own a mobile phone? Yes No N/A 3) How long have you owned a mobile phone/smart phone? 0 6 months 6 months 2 years 2 years 4 years 4 years 6 years 6+ years N/A 4) How often do you use a mobile phone/smartphone? (include N/A) 0 5 times a day 5 10 times a day 10 15 times a day 15 20 times a day 20+ times a day N/A 5) How often do you capture photos with a mobile phone/smart phone? Less than once a month Once every 2 weeks Once a week Once a day Multiple times per day N/A
6) How often do you record videos with a mobile phone/smart phone? Less than once a month Once every 2 weeks Once a week Once a day Multiple times per day N/A 7) How often do you use text input options on your mobile phone/smart phone (email, text, notes, reminders, etc.)? Less than once a month Once every 2 weeks Once a week Once a day Multiple times per day N/A 8) Do you use any other touch screen devices? If so, list them below. Yes No ___________________________________ 9) What do you use your phone for? Circle all that apply and write in any additional uses Texting Phone calls Email Entertainment Taking pictures Taking videos
10) Have you owned an Apple or Android device? Apple Android Both Neither 11) Which did you prefer? Apple Android Both Neither 12) Are you colorblind? Yes No
Create A Task Usability Test Subject ID#____________ Your warehouse has recently decided to implement a new prompting system to help employees through daily work tasks. To ensure the prompting system is effective, it needs to remain up to date with specific photos, videos and item information for products in your warehouse. If some of the item information is incorrect, an order may be filled incorrectly or may not be filled at all. You are tasked with helping collect photos, video s and item informat ion for products that are frequently filled in your warehouse. The Create A Task mobile application was provided to you to complete this task. Login 1. Open the Create A Task application 2. Login with the following credentials : Username: User1 Password: pass
Creating Prompt 1. Locate the following item on the shelving unit : Item Name SKU Location Description NetGear ProSafe 5 Port 10/100 12.031.456 Shelf 10 2 Desktop switch 2. Open Create Prompt 3. Enter the item name, SKU, location, and description from the label into the application 4. Enter the item type and item size 5. Enter the following new command into the Prompt Command: 5. Go to next location 6. Take a photo of the item on the shelf 7. Take a video of you removing the item from the shel f 8. Playback the video you recorded 9. If you are satisfied with the video you recorded, move to the next step. If not, re record your video. 10. Save the prompt media You have now completed your first create prompt task. Now you are going to update prompt informa tion for another item.
Search and Edit Prompt 1. Enter the Prompt Library 2. Search for the following item: Item Name SKU Location Description Ikea Table Legs 57.212.011 Shelf 104 Table legs 3. Select the item 4. Edit the location to: Shelf 104 5. Upload a new photo from the photo gallery of the individual table leg 6. Upload a new video of a person removing the table leg from the shelf 7. Change the item size and type to: small, individual 8. A dd the special note: Box or indi vidual 9. Add the following command line: 5. Move to next location 10. Save changes Recently the warehouse stopped carrying computer monitors and the prompts need to be removed from the system. 1. Search for the following item in the prompt library : Item Name SKU Location Description Dell Desktop Monitor 16.032 .456 Shelf 104 Monitor 2. D elete the item
Review Dashboard Your final task is to look into some prompts that the system has been experiencing trouble with. Multiple employees using the prompting system today experienced difficulty with the following items: 1. Locate the Home Gym and Exercise Mat on s helf 10 3 on the shelving unit 2. Enter the review dashboard in the app 3. Locate Home Gym and Exercise Mat in the review dashboard 4. Check if the locations match 5. If the locations m atch, mark the item as resolved 6. L ocate the Closet Hooks o n Shelf 102 Bin 202 on the shelving unit 7. Locate the Closet Hooks in the review dashboard 8. If no items are present in the bin, mark the item as out of stock
Subject ID# ___________ TELL THE TEST MONTOR WHEN YOU FINISH THIS TASK. Then complete the following short questionnaire The After Task Questionnaire For each of the statements below, circle the rating of your choice. 1. Overall, I am satisfied with the ease of completing this task. STRONG LY AGREE 1 2 3 4 5 6 7 STRONGLY DISAGREE 2. O verall, I am satisfied with the amount of time it took to complete this task. STRONGLY AGREE 1 2 3 4 5 6 7 STRONGLY DISAGREE 3. O verall, I a m satisfied with the user interface (button size, colors, text size, layout) when completing this task. STRONGLY AGREE 1 2 3 4 5 6 7 STRONGLY DISAGREE
!"#$%&'(#)*+,+$"'!-),% !!!!!!!!! "!#$%$&'(!)*+$,-./&!012,12'&$1/3!45678 ! ! !!!!!!!!! 9&21/%(: ! !!!!!! 9&21/%(:! ! !!!!!!!!! ;$<'%2.. ! !!!!!!!! '%2.. 48!=!&>$/?!&>'&!=!@1+(;!($?.!&1 !!! +<.!&>$.!',,!+//.C.<<'2$(: !!! C1-,(.D ! E8!=!&>1+%>&!&>.!',,!@'$/?!&>'&!=!@1+(;!/..;!&>. !!! <+,,12&!1A!'!&.C>/$C'(!,.2<1/!&1 !!! G.!'G(.!&1!+<.!&>$.!I'2$1+$1+%>&!&>.2.!@' !!! $/C1/<$<&./C:!$/!&>$'&!-1<&!,.1,(. !!! @1+(;!(.'2/!&1!+<.!&>$.!',,!I.2: !!! C+-G.2<1-.!&1!+<. ! 58!=!A.(&!I.2:!C1/A$;./&!+<$/ %!&>. !!! ',, ! 4K8!=!/..;.;!&1!(.'2/!'!(1&!1A !!! &>$/%!&>$
Login Use Case This use case describes the process by which users log in to the application. Actors Manager Employee Used Use Cases Create Prompt Search Prompt Library Review Dashboard Flow of Events 1) The Use Case begins when the user opens the application 2) The system will display the Welcome and Login S creen 3) The user will enter a username and password 4) The system will verify the username and password 5) The system will display the home screen 6) The user will select an option 7) While the user does not sel ect E xit loop 8) If the user selects Create Prompt then a. Use Create Prompt 9) Else if the user selects Search Prompt Library a. Use Search Prompt Library 10) Else if the user selects Review Dashboard a. Use Review Dashboard End if 11) The user will select an option End loop 12) The Use C ase ends
Activity Diagram Scenarios Incorrect password Incorrect username A u sername and password do es not exist for user The user does not select an option Subordinate Use Cases System Access Login Manage Prompt Media Display Home Screen ! ! ! ! !
Create Prompt Use Case This use case de scribes the process by which media and item information is collected to create prompts for an item in the warehouse. Actors Manager Employee Used Use Cases None Flow of Events 1) The Use C ase begins when the user selects Create Prompt 2) The system will display the Create Prompt screen 3) The user will input the item name, SKU, description, specific notes, item type and item size 4) The user will input additional prompt command lines 5) The user will select a function 6) While the user does not select a function, Exit l oop 7) If the user selects Take Photo a. The system will access the camera for the user to capture a photo of the item 8) Else i f the user selects Take Video a. The system will access the camera for the user to capture a video 9) Else i f the user selects Upload Media a. The application will display a dialog box b. The user will select a function c. While the user does not select a function, exit loop d. If the user selects Upload Photo i. The application will access the photo gallery e. Else if the user selects Upload Video i. The application will access the video gallery f. Else if the user selects Exit i. The dialog box will close End if 10) Else i f the user selects Replay Media a. The system will play back the video that was collected End if 11) The user will select a function End loop 12) The Use C ase ends Alternative Flow of Events The user is able to click the navigation bar options during the collection of the prompt. This will ex it the create prompt screen without saving the media and information, and the use case will end
Activity Diagram Scenarios The item is too large for a photo The item is too large to record a video The correct media is not in the gallery Subordinate Use Cases Database Save Media Database Get Media Manage Prompt Media Create Prompt
Edit Prompt Use Case This use case describes the process by which item information and media is edited for items in a warehouse Actors Manager Employee Used Use Cases None Flow of Events 1. The use case begi ns when the user selects an item from the Search Prompt Library 2. The application will display the Edit Prompt Screen 3. The user will select a function 4. While the user does not select a function, Exit loop a. If the user selects Take Photo i. The application will access the camera for the user to capture a new photo b. Else if the user selects Take Video i. The application will access the camera for the user to capture a video c. Else i f the user selects Replay Media i. The application will access and replay the current video d. Else i f the user selects Upload Media i. The application will display a dialog box ii. The user will select a function iii. While the user does not select a function, exit loop iv. If the user selects Upload Photo 1. The application will access the photo gallery v. Else if the user selects Upload Video 1. The applicat ion will access the video gallery vi. Else if the user selects Exit 1. The dialog box will close End if e. Else if the user selects Item Name i. The system will enable the user to alter the item name f. Else if the user selects SKU i. The system will enable the user to alt er the item SKU g. Else if the user selects Location i. The system will enable the user to alter the item location h. Else if the user selects Description i. The system will enable the user to alter the item description i. Else if the user selects Specific Notes i. The system will enable the user to alter the item specific notes j. Else if the user selects Item Type i. The system will enable the user to alter the item type k. Else if the user selects Item Size
i. The system will enable the user to alter the item size l. Else if the user selects Prompt Command i. The system will enable the user to alter the item prompt command End if 5. The user will select a function End loop 6. The Use Case e nds Activity Diagram Scenarios The item does not exist A photo for the item does not exist A video for the item does not exist Item information is missing Subordinate Use Cases Manage Prompt Edit Prompt Database Get Media Database Update Media !
Prompt Library Use Case This use case describes the process by which prompt media and item information is accessed and viewed. Actors Manager Employee Used Use Cases Delete Prompt Edit Prompt Flow of Events 1. The Use C ase begins when the user selects Search Prompt Library 2. The system displays the Prompt Library S creen 3. The user will select a function 4. While the user does not select, Exit loop 5. If the user searches by item name a. The system will filter the results by each letter the user inputs 6. Else if the user searches by SKU a. The system will filter the results by each number and character the user inputs 7. Else i f the user searches by location a. The system will filter the results by each letter the user inputs 8. Else if the user selects Delete a. Use Delete Prompt 9. Else if the user selects an item a. Use Edit Prompt End if 10. The Use Case ends
Activity Diagram Scenarios The item does not exist The item is no longer carried in the warehouse The item does not require prompts The item no longer requires prompts The search me thod does not return the item, even though it is present in the database Subordinate Use Cases Manage Prompt Media Search Prompt Library Database Get Media Database Delete Media
Review Dashboard Use Case This use c ase describes the process by which prompt errors are reviewed. Actors Manager Used Use Cases Update Status Edit Prompt Flow of Events 1. The use case begins when the user selects Review Dashboard 2. The system will display the Review Dashboard screen 3. The user will select a function 4. While the user does not select a function, Exit loop 5. If the user selects Resolve a. Use Update Status 6. Else if the user Out of Stock a. Use Update Update Status 7. Else if the user selects an item a. Use Edit Prompt End if 8. The user will select a function End loop 9. The Use Case ends
Activity Diagram Scenarios The prompt is not located in the review dashboard Subordinate Use Cases Database Get Prompt Execution Data Manage Prompt Media Review Dashboard
Update Status Use Case This use c ase describes the interaction between the non linear, context aware prompting system and updates to prompt status. Actors Non linear, context aware prompting system Used Use Cases None Flow of Events 10. The use case begins when an update status request is received 11. The system will send the prompt status to the non linear, context aware prompting system 12. The non linear, context aware prompting system will send a status of OK. 13. The Use Case ends Activity Dia gram None User Interface None Scenarios The prompt does not exist Subordinate Use Cases Manage Pr ompt Media Update Prompt Media
Delete Prompt Use Case This use c ase describes the interaction between the non linear, context aware prompting system and prompt delete requests. Actors Non linear, context aware prompting system Used Use Cases None Flow of Events 14. The use case begins when delete request is received 15. The system will send the delete request to the non linear, context aware prompting system 16. The non linear, context aware prompting system will send a confirmation of deletion 17. The Use Case ends Activity Diagram None User Interface None Scenarios The prompt does not exist ! ! ! !
System Access Subsystem System Access Use Case Login Subordinate Use Case 1. The Use Case begins when the user opens the application 2. The system displays the login screen 3. The user enters a username and password 4. The system asks the database for the user record 5. The system validates the login 6. The system ask s Manage Prompts to display the home screen 7. The Use Case ends ! ! ! ! !
Manage Prompt Media Subsystem Manage Prompt Media Use Case Display Home Screen Subordinate Use Case 1. The Use Case begins when System Access requests the home screen to be displayed 2. The system will display the home screen 3. The user will select a function 4. If the user does not select a function, Exit the loop 5. If the user selects Create Prompt a. Use Create Prompt 6. Else if the user selects Prompt Library a. Use Prompt Library 7. Else if the user selects Review Dashboard a. Use Review Dashboard End if 8. The user will select a function End loop 9. The Use Case Ends Edit Prompt Subordinate Use Case 1. The Use Case starts when the request Edit Prompt is received 2. The system displays the Edit Prompt screen 3. The user will select a function 4. While the user does not select a function, Exit loop 5. If the user selects Take Photo a. The system will access the device camera to capture a photo b. The system asks the Database to store the new photo c. The system asks the Database to update the prompt 6. Else if the user selects Take V ideo
a. The system will access the device camera to record a video b. The system asks the Database to store the new video c. The system asks the Database to update the prompt 7. Else if the user selects Upload Media a. The system asks the Database to retrieve the new photo or video from the photo gallery b. The system asks the Database to update the item information 8. Else if the user selects Update a. The system asks the Database to update all of the new item information and media End if 9. The user will select a function End loop 10. The Use Case e nds Create Prompt Subordinate Use Case 1. The Use Case starts when the request Create Prompt is received 2. The system displays the Create Prompt screen 3. The user selects a function 4. While the user does not select a function, Exit loop 5. If the user selects item Item Name a. The system asks the Database to store the Item Name 6. If the user selects Take P hoto a. The system asks the Database to store a photo 7. Else if the user selects Take V ideo a. The system asks the Database to store a video 8. Else if the user selects Command Lines a. The system asks the Database to store the command line 9. Else if the user selects Upload Media a. The system asks the Database to retrieve a photo or video 10. Else if the user selects Save a. The system asks the Database to save the item information and media End if 11. The user will select a function End loop 12. The Use Case ends Search Prompt Library Subordinate Use Case 1. The Use Case begins when the the request Prompt Library is received 2. The system displays the Prompt Library screen 3. The user searches for a n item and prompt media 4. The system asks the database to retrieve the item information and media 5. The system will display the item information and media and the Use Case ends Review Dashboard Subordinate Use Case 1. The Use Case begins when the request Review Dashboard is received
2. The system display s the Review Dashboard screen 3. The user selects a function 4. While the user does not select a function, Exit loop 5. If the user selects Resolve a. The system will ask the Database to remove an error flag from the p rompt 6. Else if the user selects Out of Stock a. The system will ask the Database to flag the item out of stock End if 7. The user will select a func tion End loop 8. The use case ends ! ! ! ! ! ! !
Database Subsystem Database Use Case Get Us er Record Subordinate Use Case 1. The Use Case begins when the system requests a User Record 2. The system locates the user record using the username 3. The system returns the user record to System Access 4. The Use Case ends Save Media Subordinate Use Case 1. The Use Case begins when the user selects Save 2. The system requests the Datab ase to save the photo, video and item information 3. The Use C ase ends Update Media Subordinate Use Case 1. The Use Case begins when the user selects Edit Prompt 2. The user inputs the updated information and media 3. The user selects Update 4. The system requests the Database to update the media and information 5. The Use C ase ends
Get Media Subordinate Use Case 1. The Use Case begins when the user selects Prompt Library 2. The user will input product information 3. The system returns the prompt media for the item to Manage Prompt Media 4. The Use Case ends Get Prompt Status Subordinate Use Case 1. The Use Case begins when the user selects Review Dashboard 2. If the user selects resolve a. The system requests the Database to remove the item flag 3. Else if the user selects out of stock a. The system requests the Database to remove the item from the prompting system 4. The Use Case ends Delete Prompt Subordinate Use Case 1. The Use Case begins when the user selects delete prompt 2. The system requests the Database to delete the item information and prompt 3. The Use C ase ends
Only Smartphone Only Mobile Phone Both Subject Frequency 0 2 4 6 Do you own a smartphone or mobile phone? (a) YesNo Subject Frequency 0 1 2 3 4 5 6 Do you use other touch screen devices? (b) NoYes Subject Frequency 0 1 2 3 4 5 6 Are you colorblind? (c) FigureH.1:Responsesfromthreeoftheelevensurveyquestionsfromallvesubjects. (a)Surveyresponseforsmartphoneownership.(b)Surveyresponsesfortouchscreen deviceusage.(c)Surveyresponsesforcolorblindness. 136
Less than once a month Once every 2 weeksOnce per weekOnce per day Multiple times per day Subject Frequency 0 1 2 3 4 5 6 How often do you record videos with a smartphone/mobile phone? (a) Less than once a month Once every 2 weeksOnce per weekOnce per day Multiple times per day Subject Frequency 0 1 2 3 4 5 6 How often do you capture photos with a smartphone/mobile phone? (b) Less than once a month Once every 2 weeksOnce per weekOnce per day Multiple times per day Subject Frequency 0 1 2 3 4 5 6 How often do you use text input options on your smartphone/mobile phone? (c) FigureH.2:Responsesfromthreeoftheelevensurveyquestionsfromallvesubjects. (a)Surveyresponseforvideorecordingusage.(b)Surveyresponsesforcamerausage. (c)Surveyresponsesfortextinputusage. 137
Only Apple Only Android Both Subject Frequency 0 1 2 3 4 5 6 Have you owned an Apple or Android device? AppleAndroid No Preference Subject Frequency 0 1 2 3 4 5 6 Which did you prefer? FigureH.3:Responsesfromtwooftheelevensurveyquestionsfromallvesubjects. ThesesurveyresponseswereforAppleandAndroidphoneownership,andpreferences betweenAppleandAndroiddevices. 138