Citation
Colorado Children's Hospital mental health de-stigmatization evaluation

Material Information

Title:
Colorado Children's Hospital mental health de-stigmatization evaluation
Creator:
Aldrich, Sarah
Place of Publication:
Denver, CO
Publisher:
University of Colorado Denver
Publication Date:
Language:
English

Thesis/Dissertation Information

Degree:
Master's ( Master of public administration)
Degree Grantor:
University of Colorado Denver
Degree Divisions:
School of Public Affairs, CU Denver
Degree Disciplines:
Public administration
Committee Chair:
Boylard, Wendy

Notes

General Note:
Spring 2018

Record Information

Source Institution:
University of Colorado Denver
Holding Location:
Auraria Library
Rights Management:
Copyright Sarah Aldrich. Permission granted to University of Colorado Denver to digitize and display this item for non-profit research and educational purposes. Any reuse of this item in excess of fair use or other copyright exemptions requires permission of the copyright holder.

Downloads

This item has the following downloads:


Full Text
Running Head: CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
Colorado Children’s Hospital Mental Health De-Stigmatization Evaluation
Sarah Aldrich
University of Colorado Denver School of Public Affairs
This client-based project is submitted in partial fulfillment of the requirements for the degree of Master of Public Administration in the School of Public Affairs at the University of Colorado Denver Denver, Colorado
Spring
2018


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
2
Capstone Project Disclosure
This client-based project was completed on behalf of Colorado Children’s Hospital and supervised by PUAD 5361 Capstone course instructor Dr. Wendy Bolyard and second faculty reader Dr. Pamela Medina. This project does not necessarily reflect the views of the School of Public Affairs or the faculty readers. Raw data were not included in this document, rather relevant materials were provided directly to the client. Permissions to include this project in the Auraria Library Digital Repository are found in the final appendix. Questions about this capstone project should be directed to the student author.


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
3
Table of Contents
Executive Summary......................................................................3
Introduction...........................................................................4
Review of Literature...................................................................6
Methodology...........................................................................14
Results...............................................................................19
Discussion............................................................................23
Recomm endati ons.....................................................................25
Limitations and Conclusion............................................................27
References............................................................................29
Appendices
A - Mental Health Work Group Organization Outline.....................................34
B- Variable Descriptions..............................................................35
C- Interview Questions................................................................36
D- Hypothesis Table...................................................................37
E - Results Table.....................................................................38
F - Trainings Results Table...........................................................40
G - Employee Survey Responses.........................................................41
H - Content Analysis Themes...........................................................42
I - Content Analysis Process..........................................................43
J - MPA Core Competencies Supplement..................................................52


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
4
Executive Summary
Many organizations have a desire to change their organizational culture. One such organization is the Colorado Children’s Hospital, the client for this project, which requested an analysis of their mental health de-stigmatization efforts. This mental health de-stigmatization evaluation investigates this issue through analysis of secondary data from 2016 to 2017. Three major themes from the findings emerged. First, statistical analysis revealed that there has been a significant increase in mental health communications and media, both internally and externally. Second, employee participation has increased in all mental health efforts aside from the online mental health reviews. Third, employee responses to the mental health efforts yields overall positive results while stressing the importance of progressive improvement.
Three recommendations are offered. First, include online mental health resources that patients and families can engage in through mental wellness reviews and trainings. Second, develop and implement an organizational policy that incorporates requirements for mental health trainings and deters declines in mental health program participation. Finally, incorporate more contact-based approaches to target de-stigmatization. With this information, the client will be better able to assess the status of their mental health efforts including areas for improvement and the employee and family response to the 2017 initiatives.


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
5
Children’s Hospital Mental Health De-Stigmatization Evaluation
Mental health illnesses have permeated the nation, and the stigmatization of those illnesses presents barriers, negative experiences, and impairments to those seeking appropriate help. Research reveals that 8 of 10 workers with mental health illness reported that shame and stigma prevented them from seeking needed mental health care (National Alliance of Mental Illness, n.d.). The discrimination that those with mental illness face is rooted in various misconceptions and a lack of knowledge and understanding. Anti-stigma and mental health improvement programs have gained popularity in hopes of normalizing public perception, removing barriers, identifying mental illnesses, and increasing individuals’ understanding and knowledge.
In the field of public administration, organizational attempts have been made to create a culture that promotes an environment of openness and acceptance. Overall health and wellbeing infiltrate organizations as a top priority, with evidence pointing to the direct impact of organizational culture on performance outcomes (Lee & Yu, 2004).
Children’s Hospital Colorado
The Children’s Hospital of Colorado (Children’s Colorado), the client for this capstone project, is a nonprofit hospital established in 1908 to serve kids of all ages and stages of life (Children’s Hospital Colorado, 2018). Children’s Colorado’s provides a network of care through sixteen locations, including the main campus in Aurora, built in 2007. Children’s Colorado is devoted to the mission of improving the health of children through high quality patient care, education, research, and advocacy (Children’s Hospital Colorado, 2018). The hospital has nearly 750,000 patient visits per year (inpatient, outpatient, and emergency), positioning it as the leading treatment hospital throughout Colorado (Children’s Hospital Colorado, 2018).


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
6
Children’s Colorado acknowledges the role health care providers play in influencing the reduction of mental health stigma. Internal efforts at Children’s Colorado are centered on changes at the individual, organizational, and social levels, and positioning the hospital as a leading voice in the community for promoting youth mental health. Health care providers are uniquely positioned to reduce the stigma through their direct relationship with patients. The mental health stigma, defined as viewing someone negatively due to their condition, includes prejudice and discrimination that influences both the individual and the organization (Mayo Clinic, 2017). The hospital has implemented numerous mental health de-stigmatization approaches to benefit both health care professionals and patients.
De-Stigmatization Efforts
In 2016, Children’s Colorado implemented a multidisciplinary workgroup designed to address various evidence-based strategies to drive culture change. The workgroup illustrates their understanding of the value of forming a multidisciplinary team to effectively develop and implement an in-house initiative and promote mental wellness. The workgroup consists of various representatives across multiple departments. The de-stigmatization efforts target all interactions that occur within the hospital, including employees, patients, visitors, and families. The initiative focuses on education, exposure to persons with mental illness, communication and advocacy for mental health, and organization policies designed to provide support to promote mental wellbeing. An organizational model of the mental health multidisciplinary workgroup is outlined Appendix A.
The hospital recognizes mental health’s position as a health determinate and its relation to increased occurrence of chronic diseases and use of medical care (CDC, 2011). Mental wellness strategies were implemented on the foundational understanding that employees’


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
7
management of their own mental health needs directly impact the hospital. Children’s Colorado also acknowledges the opportunity to identify policies and practices that could better prevent patients and their families suffering from mental illness from feeling stigmatized when seeking treatment. This evaluation of the Colorado Children’s Hospital’s mental health de-stigmatization efforts allows the workgroup to identify the effectiveness of the various approaches, successes, and areas needing improvement.
This paper contains four sections. The first details literature on various mental health programs, the second outlines the methods of evaluation, followed by an examination of the results, and concludes with a discussion of the findings and recommendations.
Literature Review
Negative stigmatization of mental illness has created numerous barriers in the mental health field resulting in distress, a reluctance to seek help, and decreased opportunities, specifically for health care providers. The healthcare system serves as a key environment in which stigma and discrimination are experienced by individuals with mental illnesses (Knaak, Ungar & Szeto, 2015). To accurately gauge the effectiveness of the Children’s Colorado in-house mental health anti-stigma initiative, a literature review of similar programs is crucial.
The studies below outline controlled interventions, cross sectional surveys, and longitudinal panel studies used to identify program effectiveness. This review explores the varying solutions and effectiveness of programs in reducing stigma by looking at social contact, education, online media, and workplace-based approaches.
Contact Based Programs
Inclusion of personal narratives and experiences from persons with mental health illnesses serves as a crucial element that the Children’s Hospital has implemented among its


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
8
employees. Social contact based approaches for reducing mental health stigma consist of direct contact among individual participants, as well as the inclusion of persons with mental health illnesses. Social contact has been identified as the most effective approach in improving stigma related knowledge and attitudes for adults (Thomicroft et al., 2016). Contact based approaches employ educational techniques in efforts to increase knowledge and confidence in treating mental health patients among health care providers. Programs have resulted in positive correlations between increased levels of confidence, comfort, and improvement and contact based skill building approaches (Beaulieu et al., 2017).
Knaak and Patten (2016) identified strategies for effective de-stigmatization program planning to incorporate collaboration, partnerships, networks, participation, and involving individuals with lived mental health experience. Involvement of individuals with lived experience entails persons with mental illness presenting their testimonies and recovery stories. When targeting healthcare providers specifically, success has been identified through evaluation in incorporating pessimistic views towards recovery, seeing the illness of the person, identifying lack of competence and confidence, and a lack of awareness in personal prejudices (Knaak & Patten, 2016).
Social contact is most effective when there is equal status between participants, as well as a common goal for interaction and cooperation (Thomicroft et al., 2016). Studies on direct social contact between the population and persons with mental illness have revealed lower levels of agreement with stigmatizing statements, improved perception of dangerousness and disturbance attached to a person with mental illness, and a growing sense of integration in day to day life (Villani & Kovess-Masfety, 2017).


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
9
However, when looking at the long-term effects of social contact on retention of program design, evidence is weak (Thomicroft et al., 2016). This demonstrates the importance of developing programs that are adaptive and consistent in their provision of opportunities (Knaak & Patten, 2016). Contact based interventions demonstrate the need for long term follow-up as a result of many mental health stigma reduction programs being in their infancy.
Incorporating elements of the social contact based approach serves as a component in designing a strategy for an effective mental health program intervention. The inclusion of personal narratives and experiences helps develop a foundational understanding of what mental illness is, while simultaneously provoking an open environment for dialogue around mental health.
Education Based Programs
Educational aspects are present in many parts of interventions and serve as a central component in addressing mental illness stigmatization. Incorporating consumer contact and educational interventions among diverse populations has been identified as most effective in reducing public stigma (Griffiths, Carr on-Arthur, Parsons, & Reid, 2014). Contact based interventions were more effective than educational approaches for adults, while educational initiatives prevailed among younger populations (Griffiths et al., 2014). Li, Juan, Huang and Thomicroft (2014) determined that, overall, the educational training they implemented changed community health staff attitudes towards mental illness, decreased social distance between staff and individuals with mental illnesses, and revealed that it was possible to train community health staff with a short course in mental health stigma.
Educational interventions are developed to improve knowledge and understanding of signs, symptoms, treatments, and availability of mental health resources. The literacy component


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
10
implies that enhanced knowledge and awareness about mental health stigma and discrimination will arm individuals to seek appropriate action for treatment (Stuart, 2016). Programs that target specific groups resulted in significant improvements in enhancing knowledge about treatments, decreased social distance, and produced increased confidence in providing help to individuals with mental illnesses (Stuart, 2016).
Awareness raising is another key component involved in educational approaches, which encourage mental health workers to engage in a dialogue about mental health and recommend next steps to make healthcare services a reality for people with mental illness (Stuart, 2016). Educating, raising awareness, and opening a dialogue, studies predict higher social tolerance towards mental health issues among college students and adults (Stuart, 2016).
Organizational culture plays a significant role in the level of mental health stigmatization within an organization. Knaak, Mantlet, and Szeto (2017) explored the idea that solutions for stigma reduction need to be approached with the goal of culture change. Two key components for interventions were identified as including transformative learning targeted on bias and false beliefs, as well as emphasizing the roles that healthcare providers play in the recovery process (Knaak et al., 2017). Educational based interventions should incorporate workshops, skills-based approaches, and intensive social contact (Knaak et al., 2017). Research stresses the importance of education through findings that mental health knowledge predicts intentions to seek help, through its association with attitudes of tolerance and community support, underlining the importance of mental health literacy (Henderson, Evans-Lacko, & Thornicroft, 2013).
Educational approach to stigma reduction findings are important in narrowing down an effective in-house hospital approach. Application of the direct contact component to appease


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
11
adults while stressing the effectiveness of educational initiatives in targeting younger individuals is a valuable tool in implementing de-stigmatization efforts across a diverse population.
Online Programs
With technological advancement and internet-based tools, the exploration of online mental health programs and resources has increased. Numerous mental health programs have begun to implement various media components as part of their initiatives. The Children’s Hospital offers various online trainings and media communications to its employees and patients.
Mental health initiatives that consisted primarily of an online education programs were successful in increasing literacy, decreasing negative perception of mental illness, and providing advice on how to promote seeking help within the workplace (Griffiths, Bennett, Walker, & Goldsmid, 2016). The intervention revealed individuals who did not participate in the universal online program possessed higher depression and anxiety stigma scores, and lower literacy for the mental illnesses than participants who completed the assessment (Griffiths et al., 2016). Other online programs reveal similar results in producing more positive attitudes towards individuals with mental illness (Kaolis et al., 2017). Digital interventions serve as a powerful strategy in facilitating widespread behavioral and cultural change, while simultaneously reducing barriers to access, increasing participant engagement and adherence (Hanisch, Bimer, Oberhauser, Nowak, & Sabariego, 2017).
Interactive programs have been developed to increase engagement and participation with what could seem a cumbersome task of devoting additional staff, time, and resources. Leadership training in Mental Health Promotion (LMHP) was designed as a digital game to improve knowledge, attitudes, and self-efficacy with handling mental health situations at work (Hanisch et al., 2017). An evaluation of the digital mental health program revealed decreased perception of


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
12
stigmatizing attitudes such as avoidance, perceived dangerousness, and responsibility as a result of the program (Hanisch et al., 2017).
Online approaches can be utilized for not just interactive trainings, but also for the provision of mental health resources. Fact sheets, toolkits, guidelines, reviews, discussion forums, documentaries, and videos consisting of personal stories can be accessed to reduce stigma and discrimination (Cerully, Collins, Wong,, Roth, Marks,, & Yu, 2016). An evaluation of website traffic and activity revealed high numbers of website utilization, and 66,000 downloads of mental health resources (Cerully, Kase, Collins, Wong, & Yu, 2015). This further demonstrates that websites can potentially serve as a popular avenue for spreading information.
With the consistent increase in the use of online tools, media can serve as an important in communicating widespread information about mental health. When looking into implementation within the Children’s Colorado, the online options provide flexibility to the busy schedules of healthcare providers as well as opportunities for self-paced learning.
Workplace Initiatives
Workplace interventions consist of periodically implementing mental health programs to target the stigmatization that persists and allow for targeting of specific groups and tailoring programs to the organization’s culture and needs. The economic impact of mental health illnesses has become an issue for employers globally (Hanisch et al., 2017).
Hanisch et al. (2017) emphasizes the needs for organizations to focus on both the employee level of stress management as well as the organizational working conditions. Employee mental health is affected by the social environment that includes organizational culture, levels of support, and leadership styles (Hanisch et al., 2017). The role of leadership plays an important part in effective and efficient implementation of mental health stigma


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
13
reduction programs, as well as an understanding that more people could be reached by targeting interventions within organizations rather than to the general public (Hanisch, Twomey, Szeto, Birner, Nowak, & Sabriego, 2016). Workplace anti-stigma interventions have revealed positive impacts on employee’s knowledge, attitudes, and behavior toward individuals with mental health problems (Hanisch et al., 2017).
Social distance has been a reoccurring barrier within mental health. It is defined as a “persons desire to avoid contact with a person perceived to have mental health problems” (Cerully et al., 2016, p. 1). Workplace Stigma Discrimination Reduction Programs resulted in reduced desire for social distance and increased understanding of how to support individuals with mental illnesses (Cerully et al., 2016). With individuals being more willing to engage with persons with mental illnesses, a dialogue will be opened, and an associated reduction of barriers is expected to occur. Even further, Cerully et al. (2016) discovered workplace trainings produced positive beliefs in recovery, greater perceived ability to provide support, and heightened effectiveness in examining aspects of social contact.
Summary
The literature above alludes to various tools that measure effectiveness including software, surveys, and interviews. These tools have been applied to inform the methodology section of this paper. As implied from the literature, creating a positive mental health culture assumes higher utilization of resources and decreased stigmatization which is the goal of the Children’s Colorado de-stigmatization efforts.
Interventions targeting health care providers are uncommon. After reviewing a variety of programs, key components are identified that remained constant throughout. The effectiveness of social contact and educational approaches is evident in the literature as producing the most


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
14
significant outcome in reducing mental health stigma. The literature serves as a guide in designing a mental health program as well as making recommendations for existing ones. The literature supports that mental health programs decrease stigmatization, thus influencing increased utilization of available resources through increased knowledge and awareness.
The Colorado Children’s Hospital offers an assortment of educational trainings for hospital employees as well as guidance for families to promote their mental wellness. These workshops provide information about the importance of mental health, how to recognize when someone might be in need of mental health support, and how to recognize and respond to unconscious bias. Through the multidisciplinary workgroups prioritization of exposure, hospital team members witness personal mental illness testimonies from administrators and teammates through a speaking series outlet called Spark Talks. Through this outlet Children’s Colorado hopes to build an accepting environment oriented around support and awareness towards promoting mental wellness. Children’s Colorado also strives to be a consistent voice in improving mental health services for children throughout Colorado. Policies and programs that they advocate for include increased screening, early intervention services, integrated systems of behavioral and physical healthcare, and expanded access to prevention and high quality mental health care.
By evaluating the effectiveness of the hospital’s mental health efforts, Children’s Colorado will have a greater understanding of existing gaps, as well as the impact of the programs on utilization of mental health resources. The outcomes-based program evaluation will help determine if the de-stigmatization efforts are effective. Resource utilization and prevalence of mental health in hospital communications will serve as the primary measures of success,


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
15
indicating the program’s effectiveness in enhancing knowledge, utilization, and distribution of existing resources.
Methodology
The program evaluation for Children’s Colorado evaluates the effectiveness of the hospital’s mental health de-stigmatization efforts, their impact on utilization of resources, and change in organizational culture. In order to explore the effectiveness of the initiative, a mixed methods design included a comparison of secondary data and semi structured phone interviews.
Utilization of available resources was reviewed and compared to the previous year, and individual perceptions of the efforts from both team members and patients were examined. Incorporating both quantitative and qualitative methods offers an “informative, complete, balanced, and useful research result” (Johnson, Onwuegbuzie, & Turner, 2007, p. 18). The evaluation investigates the following research question: What are the impact of the mental health de-stigmatization efforts at Children’s Hospital Colorado?
Hypotheses
The literature strongly suggests that the implementation of contact, education, online, and workplace-based approaches will achieve success in reducing the mental health stigma. The destigmatization efforts implemented throughout Children’s Colorado include aspects of all four of the approaches to some extent. There are six hypotheses for the hospital’s de-stigmatization efforts.
1. Employee perceptions and knowledge of hospital mental health efforts will increase from 2016 to 2017.
2. There will be increased utilization of mental health resources by employees through the Vitality
program from 2016 to 2017.


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
16
3. There will be increased employee participation in educational trainings from 2016 to 2017.
4. Mental health hospital communications will increase internally and externally from 2016 to 2017.
5. Analysis of secondary data will indicate increased measures in mental health screenings by Children’s Colorado emergency departments and urgent care centers from 2016 to 2017.
6. There will be decreases in individuals who measure as low and at risk categories for mental wellbeing when looking at resilience, psychological factors, and stressors from 2016 to 2017. Measurement
The various de-stigmatization efforts educate employees and visitors on available resources and help them identify mental health illnesses within their personal lives, families, and peers. Resource utilization is examined to determine the impact of increased knowledge of mental health and availability of resources, as well as analysis of various indicators to measure impact.
An outcomes-based evaluation assesses the impact and extent to which the organization reaches its intended goals through the initiative (McNeil, 2011). In developing the mental health efforts, Children’s Colorado indicated three overarching objectives in evaluating the outcomes: enhanced knowledge of mental health and resources, increased use of resources, and increased comfort in identifying and discussing mental health issues. The overall objective is to decrease the negative mental health stigma throughout the hospital.
Numerous variables have been selected for analysis and are outlined in Appendix B with a description of their measurements and target groups. The outcomes fluctuate by variable, with some including a comparative component from previous years, and others only allowing quantification of use. The independent variable throughout the evaluation is the program de-


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
17
stigmatization efforts, with the utilization of resources, knowledge and comfort as the dependent variable. There were two different approaches applied for measurement; content analysis and analysis of secondary data.
Quantitative
The seven key variables defined in Appendix B identify potential increases or decreases between 2016 and 2017, as well as to quantify the utilization of available resources. In instances where there is comparison, percent change was calculated to provide percent increases and decreases while acknowledging the varying numbers included in each population studied. Descriptive statistics were also used to measure changes among the quantitative variables.
Secondary data analysis. Secondary data sources included external media communications, internal site hits, Vitality measures, and the employee survey responses. Analysis of secondary data assessed where the hospital’s current status was within the newer opportunities available. Media communication were retrieved from the PMHI marketing team, where the number of hits, reach, and posts were quantified, and mental health mentioned were removed for further analysis. Internal site hits were retrieved from the hospital’s software development team where hits and unique users were then quantified to reveal the level of utilization. The vitality program was analyzed, where all programs pertaining to mental health were selected, and compared to measures from the previous year. Access to employee response surveys was also granted where questions relating to mental health and wellness were selected, compared, and quantified.
The client asked to identify the level of utilization of resources by calculating the frequency of program usage compared to other popular efforts at the hospital. These indicators serve the hospital in determining what resources are deemed successful. Success here is defined as average to high use of resources compared to other available options.


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
18
The measures conducted throughout the evaluation focused primarily on the destigmatization efforts impacts on employees. The external communications were the only measures that examined the amount of interaction with patient families.
Content Analysis. Additionally, employee interviews and internal communications through weekly emails were assessed and compared with the previous year. Each of the 48 emails for 2016, and the 51 emails for 2017 were reviewed for any mentions and references to mental health and wellness.
Qualitative Measures
Semi-structured interviews conducted via email were used to determine program effectiveness among employees from various departments and positions. The interviews gathered participants’ opinions, perceptions, and attitudes through thorough questioning (Harrell & Bradley, 2009). Employees perceptions of the hospitals environment regarding mental health and de-stigmatization efforts was assessed.
A conventional approach to analysis was applied, where responses to mental health stigmatization status throughout the hospital were investigated. The process consisted of four stages. First, codes were derived from interview responses to capture key concepts in which labels for the codes emerged. The codes were then organized into categories based on their overall relationship throughout the responses. Definitions and descriptions for each of the codes were then constructed. Finally, clusters were derived from each category to indicate positive, negative or a neutral reflection of the hospital efforts, and results are reported out. A conventional approach was selected for analysis because of its processes revealing direct information from respondents, without inflicting preconceived categories (Hsieh & Shannon, 2005, p. 1281).


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
19
Sampling Plan
Interview participants were purposely selected from various hospital departments, with participants having varying positions ranging from clinical to administrative. Interviews assessed the respondent’s views of the hospital’s current mental health de-stigmatization efforts. The client passed on sixteen individuals contact information of whom were not a part of the mental health work group. Sixteen individuals were contacted to participate in the interview process, with only six being willing to respond.
Data was compared from 2016 to 2017 due to the mental health work group beginning their de-stigmatization efforts in early 2017. The comparison identified changes in utilization and perceptions of organizational culture in order to identify a possible correlation with workgroup efforts and increased/decreased utilization of resources and changes in individual perceptions. Validity and Reliability
To ensure face validity the interview questions were previously discussed with the client and explained to each participant. The questions asked in the interviews are included in Appendix C. Analysis of the responses followed the conventional coding process because of its ability to produce descriptive codes and themes (Saldana, 2009). The design of the interviews presents potential selection bias in its participants.
Data Analysis
Mental health de-stigmatization success is identified as any increase in level of participation, use of available resources, and positive feedback from both employees and patient families. The quantification of participation and utilization underwent analysis through the identification of percent increases and testing for statistical significance. The interview conventional content analysis framework elicits categories of information, where responses are


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
20
expanded into sub-categories based on responses, and themes are developed from the outcomes (Saldana, 2009). Appendix D indicates each hypothesis that was examined and whether or not it was supported by the evaluation results.
Results
Overall, analysis of the results from 2017 revealed increases in a majority of utilization variables, and positive mental wellbeing indicators. Appendix E outlines the overall results of the variables analyzed throughout the evaluation.
Communications and Media
Analysis of hospital internal and external communications yielded positive increases across all domains.
Internal communications. Weekly emails are distributed to hospital employees highlighting upcoming events, words of encouragement, and various articles linking information on important topics. A search for mental health topics, articles, and events was conducted in order to quantify how often mental health was referenced both prior to de-stigmatization efforts and after efforts had been initiated. Weekly internal emails were examined for 2016 and 2017, yielding 48 and 51 respectively. Each email was individually read in order to identify mental health mentions. For each mental health mention a tally was given, and the results calculated for comparison between each year. The reasoning for examining communications is that with more mental health references, there will be increased opportunity for discussion and dialogue, which decreases the negative stigmatization mental illness receives. Analysis of the mental health mentions from 2016 to 2017 reveals an increase of 34.6%(n=9-^26).
External Communications. External communications consist of emails and media
pushes sent out to patients, families and other members of the community. Media stories were


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
21
calculated for various media channels through secondary data. The number of mental health media stories increased by 82% from 2016 to 2017, accounting for 1,022 different stories.
Employee intranet. Children’s Colorado has an employee intranet where mental health information and resources can be accessed via the mental wellness page. The employee intranet shows the level of interaction that employees have engaged in with the mental health page since its creation in 2017. Unique clicks and visitors were analyzed for 2017 indicating increases in both unique users and intranet hits in September of 157 and 95 respectively.
Online Mental Health Assessment
Various online mental health assessments were offered through Vitality, the hospitals platform that houses and tracks employee wellness. The mental wellbeing review experienced a slight decrease of 0.8% (n=4,155-^4,335) in total participants, with the Vitality review program eliciting a slight 0.9 % (n=2,863 ->2,877) increase in total participants.
Through the Vitality program the employee participants level of stress is measured as low risk, at risk, and high risk. The goal of the hospital is to have a continued increase and higher percentages of employees in the low risk category, while steadily decreasing the number in the high and at risk categories for stress. Stressed measured via the Vitality platform indicated a 1.15% increase in low risk, with a .88% decrease in at risk and .28% decrease in high risk (n=4,146-^4,322). When looking more specifically at psychological well-being, stressors (-1.29%, -1.4%, n=2,873 ->2,847) and resilience (-.1%, -1.4%, n=2,712-^2,662) decreased in high and at-risk percentages at the population level. Results also indicated decreases in social support, which is the physical and emotional comfort and help received from family, friends, coworkers and others. Further Vitality measures revealed a 10.5% (n=3,213) decrease in individuals who reported being stressed.


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
22
Employee Trainings
Various employee trainings were offered that incorporated aspects of mental health and wellbeing. Comparison of trained employees revealed a 78% (n=3 5-^64) increase in employees who participated in the Youth Mental Health First Aid training and a 73% (n=62-> 107) increase in employees who participated in Unconscious Bias. From the Spring of 2017 to Fall there was a 15.2% (n=33->38) increase in employees registered in the mindfulness course, and a 60%
(n= 15-^24) increase in average session attendance.
Appendix F depicts the change in Children’s Colorado employees trained in Youth Mental Health First Aid and Unconscious Bias from 2016 to 2017.
Surveys
Numerous surveys were administered by the evaluator among hospital employees to gauge their perception and knowledge of mental illness, hospital resources, and perceptions of barriers to accessing mental health resources. The annual wellness survey indicates a 7.87% (n=32->86) decrease in stigma as being recorded as a barrier to accessing mental health resources at Children’s Colorado. The 2017 employee wellness survey further indicated that of the respondents, 35.11% (n= 581) described their health and wellness goals as focusing on mental health. Stress management and mental health seminars were selected as being of greatest interest to 61.3% and 39.96% of 553 participants respectively. However, results also revealed that of the participants who reported utilizing the Employee Assistance Program (EAP) in 2017, only 13.64%(n=484) claimed to use mental health resources. An overall employee population decrease of 16.6% (n= 1 00->564) was identified in participants claiming to understand how to
access EAP resources.


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
23
The 2017 employee opinion survey measured responses to mental health, stress, and resiliency. Responses were compared from 2016 to 2017 and revealed a 3% increase in participants viewing the amount of stress they feel as reasonable as well as 62% (n=556) of respondents reporting that their supervisor provides an accepting environment to discuss mental health related issues in 2017. The employee surveys were analyzed by picking out questions related to mental health and comparing and reporting the findings to previous year when applicable.
Survey response charts are included in Appendix G.
Screenings
Data collected on Children’s Colorado mental health screenings were only compiled in 2017. However, the 2017 data reveal a screening rate of 78.2% in all emergency department and urgent care locations for patients ages 10 and up, and an overall screening rate of 80% for children not previously screened in urgent care centers. The hospital also engineered efforts to screen caregivers, where 8,000 screenings were conducted on parents to assess their mental health in 2017.
Employee Interviews
In late March, 16 emails that included seven questions assessing the hospital’s culture and acceptance of mental health were distributed among a variety of employees. A response rate of 38% (n=6) allowed further investigation into the perceptions from clinical and administrative staff of hospital efforts. Of the respondents there was a project engineer, a clinical assistant, a nurse, and three administrative employees. Responses were coded and categorized, yielding overall positive, negative, and neutral results. The overall theme reflects the majority of the respondent’s perceptions of their experiences and views of mental health culture. Appendix H


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
24
outlines the results per category and cluster for each question. Question seven was used as a tool to guide recommendations and therefore wasn’t included in the content analysis.
Overall, the data reflects positive results of an improved mental health culture from previous years, employee experience, as well as a positive mental health culture. However, negative results emerged when participants were asked about Children’s Colorado culture to discuss mental health. All of the interview participants were found to be only somewhat aware of the hospitals mental health efforts, resources, and programs. Additionally, mixed results on acceptance by team members with mental illness were reported. It is important to note that only six hospital employees were interviewed among the 7,000 benefits eligible employees that work for Children’s Colorado. Appendix I includes the content analysis coding process and interview responses for the conventional content analysis.
Discussion
The results highlight the success of the mental health de-stigmatization efforts achieved after one year of initiation. Simultaneously, the results illustrate that organizational destigmatization is a continuing process that elicits room for improvement. Program success has been rooted in the cooperation and commitment of various hospital departments to address mental health stigma from all directions.
The findings parallel the educational and online program approaches. The literature largely identifies success in incorporating educational components to combat stigma change, enhance organizational culture, and address bias and false beliefs (Knaak, S. et al., 2017). The success that Children’s Colorado experienced in their online mental health forums further reflects Hanisch et al.’s (2017) perception of online forums being a powerful strategy to facilitate behavior and culture change. Further, the results reflect Cerully et al.’s (2015) findings that


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
25
online approaches can be utilized to reduce stigma and discrimination while increasing the utilization of available resources.
Through various communication channels, internet programs, and trainings it is evident that the workgroup has been successful in increasing the prevalence of and interaction with mental health and wellness resources among hospital employees. The initiatives success can be contributed to the workgroup incorporating all four of the approaches deemed effective by the literature. Analysis of employee mental wellness also noted overall positive results in decreasing the percentages of stress among high and at-risk employees.
Only four of the variables measured proposed areas where Children’s Colorado did not achieve its goals. Measured through Vitality, these areas were a decrease in social support for the low risk population, no change in high risk population’s measures of social support, a decrease in utilization of the mental wellbeing review, and a decrease in survey participants knowing how to utilize EAP. The interviews also revealed negative results in respondents indicating an overall inadequate environment to discuss mental health openly and only “some awareness” of mental health de-stigmatization efforts and resources.
Social support was defined by the work group as the physical and emotional comfort received from family, friends and coworkers. A number of factors can play into an individual’s level of social support; however, the results emphasize the importance of organizational culture that the mental health de-stigmatization workgroup was trying to improve. Further efforts to enhance the programs offered through vitality and EAP may counter the declines related to mental health.
Recommendations


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
26
For 2017 a majority of the mental health de-stigmatization efforts targeted hospital employees, with only minimal programs addressing patients and families. The intended design behind the de-stigmatization efforts stemmed from the hope that the efforts would have a trickle-down effect from providers to patients. Responses from hospital employees highlight existing gaps in this belief and inform recommendations to further incorporate patients and families into the de-stigmatization program. Overall, the Children’s Colorado mental health de-stigmatization evaluation revealed success in the workgroups efforts. Further focus should be undergone to increase awareness of mental health resources and de-stigmatization efforts through continued communications and marketing.
Three recommendations emerge. These could better help the client promote organizational culture change and further address mental health de-stigmatization across all individual interactions.
Recommendation 1: Include additional mental health resources for patients and families.
To address the issue of the lack of mental health specific tools for patients and families, the hospital needs to enhance opportunities for engagement. Through implementing similar tools that are currently offered to employees, such as the mental wellness trainings and online tools, patients and families can also participate in enhancing their knowledge and understanding of mental health. By allowing patients and families to engage in the online assessments such as the psychological wellbeing, social support, and resilience mental wellbeing reviews they will be better equipped to engage in mental wellness treatments and services. As affirmed in the literature, the development of online interactive resources would serve as an effective tool for promoting mental wellness, especially among the adolescent population (Hanisch, S. et al., 2017). This will not only elicit increased understanding but provoke a similar change in attitudes


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
27
and beliefs about mental illness, thus attributing to culture change and de-stigmatization. Future efforts should investigate targeting specific groups throughout the community that are more prone to mental health risk factors, such as gender, age, and socio-economic status. Recommendation 2: Develop and implement an organizational policy that requires employees to be trained in mental health/wellness.
In order to address the declines in participation and further encourage continued interaction with existing programs and resources, the client needs to implement an organization wide policy. An ideal organizational policy would incorporate efforts to prevent and reduce stress, as well as continually generate a dialogue for mental health. Developing a policy that requires participation in mental health efforts would ensure that individuals are receiving continued education. The requirement could be implemented in one of two ways:
1. Requiring all new employees to complete a mental health review and/or training during orientation.
2. Require mental health reviews/trainings to be completed by all employees that wish to receive points through Vitality.
This approach presents ease in implementation due to the hospital and its employees already having embraced the programs and efforts that were implemented in 2017. Simultaneously this approach incorporates the success Li et al. (2014) indicated from online approaches and workplace trainings that Cerully et al. (2015) presented as producing positive outcomes in reducing stigma and promoting mental wellness. Additionally, the approach highlights the sustainability of the de-stigmatization efforts in implementing an organizational policy.


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
28
Recommendation 3: Incorporate more contact based approaches to target destigmatization
In order to target mental health stigmatization, it is important to continually incorporate and enhance the social contact component that was shown in the literature to be extremely effective in the literature. The popular Spark Talks that the hospital organizes on a monthly basis should deliberately incorporate mental health into the discussions. Hospital employees can attend these talks, as well as listen to them online. This furthers the workgroups ability to reach a large portion of their target population. The topics do not need to be solely on mental health and stigma, but addressing various mental health components such as stress, anxiety, and depression presents a very feasible approach to encourage a dialogue for mental health. This approach further aids hospital efforts to decrease negative perceptions and stigma of mental illness (Villani & Kovess-Masfety, 2017).
Limitations
Various limitations of the mental health de-stigmatization efforts evaluation stem from its infancy as a program. The high percentage increases across a majority of the measures can be contributed to the fact that prior to 2017 there was an extreme lack of mental health efforts throughout the hospital. Further analysis should be sought after multiple years of implementation in order to analyze the continued impact over time.
Other limitations are a result of the hospitals incomplete documentation from 2016 which limits effective comparison. Throughout the time period evaluated Children’s Colorado switched EAP providers due to inadequate reporting, which hindered analysis of EAP and employee interaction. Varying levels in sample sizes for survey responses, as well as individual participants could contribute and directly affect the results.


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
29
Limitations were also present in the interviews conducted with the hospitals employees. The results indicated very mixed responses regarding Children’s Colorado and the mental health environment, with responses being contradictory. The format of the interviews did not allow further investigation into participants answers and were limited to only six responses. Not every department was represented in the sample, as well as one of the respondents not being employed the previous year, thus negating the comparison question.
Conclusion
The intent behind implementing a grassroot mental health de-stigmatization effort was that before the hospital can seek culture change at the community level, they must first address the problems with the internal culture at Children’s Colorado. Through continued collaboration, involvement, and commitment of all hospital employees, mental health stigmatization will effectively be addressed.
Based on analysis of Children’s Colorado operations, implementing an organizational wide initiative will reap the most success when it entails tasks and objectives that departments and employees are already doing. Initiatives will be the most technically feasible when the addition of tasks are limited. The importance of a multifaceted approach (education, online, workplace, and contact based) is also a crucial aspect recognized for producing success within reducing mental health stigma throughout the organization.


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
30
References
Bakolis, I., Evans-Lacko, S., Henderson, C., Robinson, E., Sampogna, G., & Thomicroft, G.
(February 2017). The impact of social marketing campaigns on reducing mental health stigma: Results from the 2009-2014 Time to Change programme. European Psychiatry 40, 116-122. https://doi.Org/10.1016/j.eurpsy.2016.08.008
Beaulieu, T., Patten, S., Knaak, S., Weinerman, R., Campbell, H., & Lauria-Horner, B. (2017). Impact of Skill-Based Approaches in Reducing Stigma in Primary Care Physicians: Results from a Double-Blind, Parallel-Cluster, Randomized Controlled Trial. Canadian Journal of Psychiatry. Revue Canadienne de Psychiatrie, 62(5), 327-335. http://doi.org.aurarialibrary.idm.oclc.Org/10.l 177/0706743716686919
Cerully, J. L., Kase, C. A., Collins, R. L., Wong, E. C., & Yu, J. (2015). CalMHSA Stigma and Discrimination Reduction Online Resources: Highlights from an Evaluation of Web Analytic Data. Rand Health Quarterly, 5(3), 4.
Cerully, J. L., Collins, R. L., Wong, E. C., Roth, E., Marks, J., & Yu, J. (2016). Effects of Stigma and Discrimination Reduction Trainings Conducted Under the California Mental Health Services Authority: An Evaluation of Disability Rights California and Mental Health America of California Trainings. Rand Health Quarterly, 5(3), 5.
Children’s Hospital of Colorado. (2018). Welcome to children’s hospital Colorado. Retrieved from: https://www.childrenscolorado.org/about/


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
31
Griffiths, K., Bennett, K., Walker, J., Goldsmid, S. (2016) Anthony Bennett, Effectiveness of MH-Guru, a brief online mental health program for the workplace: A randomized controlled trial, In Internet Interventions, 6, 29-39. Retrieved from: https://doi.Org/10.1016/j.invent.2016.09.004
Griffiths, K., Carron-Arthur, B., Parsons, A. Reid, R. (June 2, 2014). Effectiveness of programs for reducing the stigma associated with mental disorders. A meta-analysis of randomized controlled trials. World Psychiatry, 13(2), 161-165. doi: 10.1002/wps.20129
Hanisch, S., Birner, U., Oberhauser, C., Nowak, D., & Sabariego, C. (2017). Development and Evaluation of Digital Game-Based Training for Managers to Promote Employee Mental Health and Reduce Mental Illness Stigma at Work: Quasi-Experimental Study of Program Effectiveness. JMIR Mental Health, 4(3), e31. http://doi.org.aurarialibrary.idm.oclc.org/10.2196/mental.7600
Hanisch, S., Twomey, D., Szeto, A., Birner, U., Nowak, D., & Sabariego, C. (2016). The
effectiveness of interventions targeting the stigma of mental illness at the workplace: a systematic review. BMC Psychiatry, 16:1. https://doi.org/10.1186/sl2888-015-0706-4
Harrell, M., & Bradley, M. (2009). Data collection methods: Semi-structured interviews and focus groups. RAND Corporation. Retrieved from http s: //www. rand. org/ pub s/techni cal_reports/TR718. html.
Henderson, C., Evans-Lacko, S., & Thomicroft, G. (2013). Mental Illness Stigma, Help Seeking, and Public Health Programs. American Journal of Public Health, 103(5), 777-780. Retrieved from http://doi.org.aurarialibrary.idm.oclc.org/10.2105/AJPH.2012.301056


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
32
Hsieh, H., Shannon, S. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277-1288. Retrieved from http://journals.sagepub.com/doi/pdf/10.1177/1049732305276687
Johnson, B., Onwuegbuzie, A., Turner, L. (2007). Toward a definition of mixed method research. Journal of mixed method research, 1(2), 112-133. Retrieved from http://journals.sagepub.com.aurarialibrary.idm.oclc.org/doi/pdf/10.1177/1558689806298 224
Knaak, S., Patten, S. (July 17, 2016). A grounded theory model for reducing stigma in health professionals in Canada. Acta Psychiatrica Scandinavica, 134(5446), 53-62. doi: 10.1111/acps.l2612
Knaak, S., Mantlet, E., & Szeto, A. (2017) Mental illness-related stigma in healthcare: barriers to access and care and evidence-based solutions. Healthcare Management Forum. 30:2,
111-116. https://doi.org/10.1177%2F0840470416679413
Lee, S. K. J., & Yu, K. (2004). Corporate culture and organizational performance. Journal of
Managerial Psychology, 19(4), 340-359. http://dx.doi.org/10.1108/02683940410537927
Li, J., Li, Juan., Huang, Y. & Thronicroft, G. (December 4, 2014). Mental health training
program for community mental health staff in Guangzhou, China: Effects on knowledge of mental illness and stigma. International Journal of Mental Health systems, 8(49). https://doi.org/10.1186/1752-4458-8-49


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
33
McNeil, R. C. (2011). A program evaluation model: Using Bloom's taxonomy to identify
outcome indicators in outcomes- based program evaluations. MPAEA Journal Of Adult Education, 40(2), 24-29. Retrieved from:
http://aurarialibrary.idm.oclc.Org/login7urUhttp://search.ebscohost.com/login.aspx7direct =true&db=eft& AN=8 5 651355
Mayo Clinic. (2017). Mental health: Overcoming the stigma of mental illness Retrieved from: https://www.mayoclinic.org/diseases-conditions/mental-illness/in-depth/mental-health/ART-20046477
National Alliance of Mental Illness, (n.d.). Stigma free company. Retrieved from: https://www.nami.org/stigmafreeco
Saldana, J. (2009). An introduction to codes and coding. In The coding manual for qualitative researchers (Chapter 1). Retrieved from
http://stevescollection.weebly.com/uploads/l/3/8/6/13866629/saldana_2009_the-coding-
manual-for-qualitative-researchers.pdf
Stuart, H. (2016). Reducing the stigma of mental illness. Global Mental Health, 3, el7. http://doi.org.aurarialibrary.idm.oclc.org/10.1017/gmh.2016.ll
Thomicroft, G., Mehta, N., Clement, S., Evans-Lacko, S., Doherty, M., Rose, D., Koschorke, M., Shidhaye, R., O'Reilly, C., & Henderson, C. (2016). Evidence for effective interventions to reduce mental-health-related stigma and discrimination, In The Lancet, Volume 387, Issue 10023, 1123-1132. Retrieved from: https://doi.org/10.1016/S0140-6736(15)00298-
6.


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
34
Ungar, T., Knaak, S., & Szeto, A. C. (2016). Theoretical and Practical Considerations for
Combating Mental Illness Stigma in Health Care. Community Mental Health Journal, 52, 262-271. http://doi.org.aurarialibrary.idm.oclc.org/10.1007/sl0597-015-9910-4
Villani, M., & Kovess - Masfety, V. (2017). Could a short training intervention modify opinions about mental illness? A case study on French health professionals. BMC Psychiatry, 17, 133. http://doi.org.aurarialibrary.idm.oclc.Org/10.l 186/sl2888-017-1296-0


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
35
Appendix A: Organizational Model of the Mental Health Workgroup
Colorado Children's Hospital
MH Multi-disciplary Workgroup
Employees, patients &
visitors
Employees
Employees
Employees


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
36
Appendix B: Variable Descriptions
Variable Description Target Measure
Internal mental wellness site The employee intranet contains a mental wellness page. Visitors and clicks were analyzed for 2017. Employees ID employee interaction
Employee opinion & Annual wellness survey Both survey’s asked various mental health questions. Responses are compared to the previous year’s responses. Surveys given with entry into prize incentives upon completion. Employees Comparison
Vitality points earned for MH activities, includes mental wellness review The hospital offers various opportunities for employees to garner vitality points where various discounts are offered upon completion of a certain number of points. Use of mental health activities and results will be compared to previous year. Employees Comparative analysis, identifies percent change from 2016 to 2017
Educational Trainings Identify number of staff who completed educational trainings through various outlets such as Youth mental health first aid and unconscious bias. Employees Statistical analysis, identifies percent change from 2016 to 2017
Mental Health (MH) hospital communications Internal and external communications. Internal consists of weekly emails that are sent to employees, analysis will include identifying the amount of references for mental health resources/articles/activities. External are communications sent out to the patients, families, and other members of the community. Clicks and site visits will be calculated and compared to previous years. Employees, patients, & families Comparison from 2016 to 2017
Screening for mental health The number of mental health screenings conducted at Children’s hospital clinics will be calculated and compared to previous years. Number of individuals screened, and number of clinics doing the screenings will be identified. Patients Quantify the amount of MH screenings
Employee Perceptions The mental health family advisory council meets monthly to discuss, promote, and improve family centered care. The council provides direct input of hospital initiatives. Individual interviews serve as an outlet to analyze the de-stigmatization efforts from the perceptions of patient families and members of the council. A majority of the members have been there before and after the push towards de-stigmatization began. Employees Email interviews and content analysis (coding framework)


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
37
Appendix C: Interview Questions
A series of 6 questions were asked in via email, where 6 out of the 16 employees contacted responded. Conventional coding framework was applied to the interview responses to quantify similarities and themes. Positive and negative themes were identified, with a majority of responses yielding positive and mixed results.
1. What has your experience at Children’s been like?
2. What mental wellness efforts are you aware of through Children’s Hospital of Colorado? (programs, opportunities, resources, changes, updates)
3. How would you describe the current mental health culture and atmosphere of Children’s Colorado?
4. How would you describe Children’s Colorado’s environment to discuss mental health related issues?
5. How would you compare the current mental health culture to years past?
6. How would you describe the acceptance of mental illnesses by team members? (clarification: sensitive, judgmental)
7. What does success in promoting mental wellness look like to you?


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
38
Appendix D: Hypothesis table
Hypothesis Result
HI: Employee perceptions and knowledge of hospital mental health efforts will increase from 2016 to 2017. Partially supported
H2: Utilization of mental health resources by employees through the Vitality program will increase. Partially supported
H3: Employee participation in educational trainings will increase. Supported
H4: Mental health hospital communications will increase internally and externally. Supported
H5: Mental health screenings by hospital emergency departments and urgent care centers will increase. Supported
H6: There will be decreases in high and at risk categories for mental wellbeing measures for resilience, psychological factors, and stressors, and increases in low risk categories. Partially supported


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
39
Appendix E: Data Collection Results
Variable Results
Internal Mental Wellness Site hits • A total of 545 hits and 229 unique users in 2017
Employee Survey • 16.60% Decrease in participants knowledge of how to access EAP • Of the participants that utilized EAP in 2017, only 13.64% utilized mental health resources • 7.87 % Decrease in stigma as a barrier for accessing Mental Wellness Resources • 2016 survey: 87% claim that Children’s Colorado has a culture that supports mental and physical health • 2017: 35.11% say they would describe their health and wellness goals as focusing on mental health • 2017: Among seminar topics offered, stress management (61.3%) and mental health (39.96%) were in the top four • Employee opinion survey: o 3% increase in respondents reporting that the amount of stress they feel is reasonable o 3.93 (out of 5) respondents reporting that Children’s Colorado provides resources that support mental wellbeing in 2017 o 3% Increase in EOS respondents reporting the level of stress they feel is reasonable
Vitality points earned for MH activities, includes mental wellness review • Decrease in high & at risk individuals indicated in psychological wellbeing (-.1%, -2.1%), stressors (-1.2%, -1.4%), and resilience (-.1%, -1.4%) • Increase in social support at risk populations (1.15%) • Decrease in social support for low risk population (-1.5%) • No % change in high risk population social support • Decrease in high (-.28%) and at risk (-.88%) populations for stress measures • Increase in low risk population for stress measures (1.15%) • Increase in utilization of vitality health review (.9%) • Decrease in utilization of mental wellbeing review (-.8%) • Decrease in individuals who reported being stressed in 2016 to 2017(10.5%)
Educational Trainings • Increase in employees who participated in Youth Mental Health First Aid (78%) • Increase in employees who participated in Unconscious Bias (73%)


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
40
• Increase/Decrease in employees who completed LDI • Increase in employees who completed the mindfulness course from Fall 2017 to Spring 2017 o 15.15% increase in registered o 60% increase in avg. per session
Mental Health (MH) hospital communications Internal • Increase in mental health references (34.6%) External • Increase in mental health media stories (82%)
Screening for mental health • Screening rate of 78.2% in Emergency Departments & Urgent Care, 80% screening rate in other hospital areas. • Increase from 1 clinic screening in 2016 to 15 clinics screening in 2017
Employee Interviews • Mixed themes, overall, they revealed positive employee perceptions of the Children’s Colorado’s environment to discuss mental health.


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
41
Appendix F: Mental Health Training Participation
120
100
80
60
40
20
0
Participation Growth in MH Trainings
2016 2017
â–  Youth Mental Health First Aid â–  Unconscious Bias


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
42
Appendix G: Employee Survey Responses
• Do you know how to access the Employee Assistance Program (EAP)
o 2016: n=100
â–  83% yes
â–  17% no o 2017: n=564
â–  66.4% yes
â–  33.16% no
o -16.6% change in yes responses
• If yes did you utilize EAP?
o 2016: n=538
â–  13.01% yes
â–  77.7% no
o If yes, did you utilize mental health resources last year? o N=484
â–  13.64% yes
â–  86.36% no
• Which of the following describes your health and wellness goals?
o N=581
o 35.11% focus on my mental health
• Among the seminar topics offered, select those that are of greatest interest to you
o N=553
o 61.3% said stress management o 39.96% said mental health
• Does Children’s have a culture that equally supports mental and physical health?
o N=100 o 87% yes o 13% no
What are the barriers to accessing resources to mental wellness at CHCO? Select all that apply
2016 Count 2017 Count % Change
Time 72% 72 75.79% 263 3.79%
Technology 2.04% 2 3.75% 13 1.71%
Stigma 32.65% 32 24.78% 86 -7.87%
Fear 23.47% 23 17.87% 61 -5.60%


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
43
Appendix H: Content Analysis Chart
Question Answer (category) Cluster (overall)
1. Experience (n=6) Inadequate: 11 Positive:llll Positive
2. Awareness (n=6) Some awareness: 111111 Neutral
3. MH Culture (n=6) Poor: 11 Priority: 1 Supportive: 11 Advanced: 1 Positive
4. Environ. To discuss MH (n=6) Poor: 1111 Open: 11 Negative
5. Culture compared to previous years (n=5) Improved: 1111 Inadequate: 1 Positive
6. Acceptance by team members (n=6) Sensitive: 11 Uninformed: 1 Supportive: 111 Judgemental:! Neutral


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
44
Appendix I: Content Analysis Coding Process
Content Analysis Steps:
• Codes
• Categories
• Definitions and descriptions for each of the codes
• Cluster: Good/Bad Respondent 1:
1. What has your experience at Children’s been like?
I work with many of the psychologists here at Children’s and I see and hear how hard they have to work to promote mental wellness and the importance of integration with physical health. Mental health is not elevated to where it needs to be as a priority in the Children’s workplace culture.
• Codes: A lot of effort, hard work
• Category: Inadequate experience
• Definition: Experience at Children’s Colorado does not meet expectations and needs to be improved.
• Negative
2. What mental wellness efforts are you aware of through Children’s Hospital of Colorado? (programs, opportunities, resources, changes, updates)
The stigma reduction campaign.
• Codes: Somewhat aware of efforts
• Category: Some Awareness
• Definition: The individual is aware of some of the resources available at Children’s Colorado, but does not reveal knowledge of a majority of any of the resources and opportunities available. This reflects that Children’s Colorado is failing to convey the available resources to employees.
• Nuetral
3. How would you describe the current mental health culture and atmosphere of Children’s Colorado?
Our team is very high performing and with that comes an expectation and a drive to take on more and more work. Most of our team is stressed or stretched thin at least 90% of the time. Mental health PTO days are not commonplace on our team particularly because it would mean you come back to work buried by emails, so it wouldn’t be worth taking a day off for that reason. The leadership team in our department all work well over 40 hours and are answering emails late at night because of the demands of the job. However, my mangers have always been good at checking in on my work life balance but it is not modeled well by leadership which then makes it trickle down to everyone else.
• Codes: Stressed, high expectation, need MH days off, need enhanced work/life balance, need improvement in leadership, overworked
• Category: Poor MH culture
• Definition: The current mental health culture and atmosphere is inadequate to the needs of its employees and needs to be improved.
• Negative


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
45
4. How would you describe Children’s Colorado’s environment to discuss mental health related issues?
Since our team works on mental health, it is a common topic of discussion. However, Children’s as a whole I would say still has a lot of work to do to make mental health a priority for employees to discuss it and have more awareness of mental health issues.
• Codes: a lot of work to be done, not adequate, needs improvement
• Category: Poor MH culture
• Definition: The current environment to discuss mental health needs to be improved.
• Negative
5. How would you compare the current mental health culture to years past?
I would say awareness is getting better from past years and stigma is being reduced.
• Codes: More awareness, reduced stigma
• Category: Improved
• Definition: The mental health culture has/is improving from past years.
• Positive
6. How would you describe the acceptance of mental illnesses by team members? (clarification: sensitive, judgmental)
Sensitive.
• Codes: Sensitive
• Category: Sensitive
• Definition: Respectful of others feelings/situation
• Positive
7. What does success in promoting mental wellness look like to you?
Recognizing the importance of not just physical health but also mental health and making that a topic of discussion in hospital and department policies to support employees and their families with mental wellness and supporting better work life balance.
• Codes: Increase discussion, work life balance, develop organizational policies
• Category: Organizational policies and increased discussion
• Definition: Develop organizational policies to meet the needs of employees. As well as increase opportunity to discuss mental wellness.
Respondent 2:
1. What has your experience at Children’s been like?
Overall it has been positive. I have had great managers who have supported my growth and who want to make work a positive environment.
• Codes: Great managers, support growth, positive environment
• Category: Positive/supportive environment
• Definition: Experience at Children’s Colorado has been a positive work environment.
• Positive
2. What mental wellness efforts are you aware of through Children’s Hospital of Colorado? (programs, opportunities, resources, changes, updates)
EAP, Break the Stigma campaign


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
46
• Codes: Awareness of two programs
• Category: Some Awareness
• Definition: The individual is aware of some of the resources available at Children’s Colorado, but does not reveal knowledge of a majority of any of the resources and opportunities available. This reflects that Children’s Colorado is failing to convey the available resources to employees.
• Nuetral
3. How would you describe the current mental health culture and atmosphere of Children’s Colorado?
I’m not sure, I feel like I am hearing more about the importance of mental health but that could be because I work in PMHI where we are all about mental health.
• Codes: Increase discussion of MH importance in department
• Category: Priority
• Definition: Mental health has become a higher priority.
• Positive
4. How would you describe Children’s Colorado’s environment to discuss mental health related issues?
I think it’s becoming more open, but it doesn’t feel like people outside of the mental health profession discuss mental health related issues much unless they are discussing patient care. The psychologists I work with are very open to discussing mental health related issues in their own lives and in mine when appropriate.
• Codes: Open in PMHI, not as common throughout the hospital in general
• Category: Poor (overall, supportive in PMHI)
• Definition: The overall hospital does not reflect an environment that is discussing mental health. PMHI does a good job being open in discussion mental health and wellness.
• Negative
5. How would you compare the current mental health culture to years past?
Hard to say since I changed positions, but it feels more open in PMHI than it did in Ambulatory.
• Codes: Enhanced, more open
• Category: Improved
• Definition: The mental health culture has/is improving from past years.
• Positive
6. How would you describe the acceptance of mental illnesses by team members? (clarification: sensitive, judgmental)
I think people are sensitive to it, but unsure how to handle mental illnesses.
• Codes: Sensitive, confused, not qualified/informed
• Category: Sensitive & uninformed
• Definition: Respectful of others feelings/situation. Lacking knowledge of mental health.
• Nuetral (both)
7. What does success in promoting mental wellness look like to you?


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
47
More normalization of mental wellness and mental health first-aid (preventative daily mental wellness activities). Also, ability/acceptance for taking mental health days similar to taking sick days.
• Codes: Normalization, increased discussion, acceptance, prevention
• Category: Increased Discussion
• Definition:
Respondent 3:
1. What has your experience at Children’s been like?
I have loved working at Children's. I have the ability to work closely with patients, my coworkers are excellent, and our floor administrators are great communicators and consistently look for ways to improve patient care.
• Codes: Positive work experience, good communication and relationships
• Category: Positive environment
• Definition: Experience at Children’s Colorado has been a positive work environment.
• Positive
2. What mental wellness efforts are you aware of through Children’s Hospital of Colorado? (programs, opportunities, resources, changes, updates)
I have attended resiliency rounds and I know that there is counseling available to staff.
• Codes: Some awareness
• Category: Some Awareness
• Definition: The individual is aware of some of the resources available at Children’s Colorado, but does not reveal knowledge of a majority of any of the resources and opportunities available. This reflects that Children’s Colorado is failing to convey the available resources to employees.
• Nuetral
3. How would you describe the current mental health culture and atmosphere of Children’s Colorado?
I work in the CCBD and I believe the mental health culture on our floor is especially tolerant and transparent. The CCBD team is especially close with the patients and their families during their extended stays, so we engage in respite time, REST sessions, room blessings, etc. to allow us space to grieve deaths and process the emotional difficulty of dealing with cancer on a day to day basis. Maintaining mental wellness and allowing space to process sorrow seems to be a priority on our floor.
• Codes: tolerant and transparent, good environment, resources/opportunities to address mental wellness, relational
• Category: Supportive
• Definition: The culture of the hospital is supportive in addressing mental health.
• Positive
4. How would you describe Children’s Colorado’s environment to discuss mental health related issues?
Thinking back on the year, I have received numerous emails from Children's regarding mental health initiatives for parents and patients. It seems that mental health is as much a priority at Children's as other physical ailments, and there is an atmosphere of openness surrounding these


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
48
issues. Likewise, while sitting with patients with mental health issues, I have noticed that the staff is decidedly nonjudgmental.
• Codes: equal focus/attention, open, nonjudgmental
• Category: Open
• Definition: The culture of the hospital is open and accepting to discussing mental health.
• Positive
5. How would you compare the current mental health culture to years past?
I have just completed my first year at Children's, so I have nothing to compare.
• N/A
6. How would you describe the acceptance of mental illness by team members? (clarification: sensitive, judgmental)
Regarding patients with mental illness, team members are nonjudgmental and strive to be positive encouragers. Similarly, in the CCBD the staff is understanding that we each have to go through the grieving process for different patients and need time to heal. However, I have seen frustration with team members who struggle with anxiety or other mental illnesses that affect their ability to work. So there is sensitivity and acceptance until the mental illness diminishes job performance or becomes a negative influence.
• Codes: nonjudgmental, positive, encouraging, understanding, sensitivity, acceptance, frustration
• Category: Supportive
• Definition: Hospital team members are supportive of mental illness among patients and employees.
• Positive
7. What does success in promoting mental wellness look like to you?
It seems to me that Children's is already relatively successful at promoting mental wellness. I receive hospital-wide newsletters with information on mental health initiatives, the CCBD unit managers consistently check in to see how we are coping, there are numerous traditions in the CCBD that allow us to process grief and celebrate life, and overall it seems there is an understanding that mental illness is common and should not be hidden.
• Codes: Children’s is successful, good leadership, awareness, communication
• Category: Awareness, Leadership
• Definition: Awareness of mental health and involvement of leadership.
Respondent 4:
1. What has your experience at Children’s been like?
Stressful. Somewhat rewarding but disappointing overall.
• Codes: Stressful, disappointing
• Category: Inadequate experience
• Definition: Experience at Children’s Colorado does not meet expectations and needs to be improved.
• Negative
2. What mental wellness efforts are you aware of through Children’s Hospital of Colorado? (programs, opportunities, resources, changes, updates)
Yoga/meditation classes, mindfulness classes.
• Codes: Some awareness


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
49
• Category: Some Awareness
• Definition: The individual is aware of some of the resources available at Children’s Colorado, but does not reveal knowledge of a majority of any of the resources and opportunities available. This reflects that Children’s Colorado is failing to convey the available resources to employees.
• Nuetral
3. How would you describe the current mental health culture and atmosphere of Children’s Colorado?
I don’t know, I’m not sure that one exists.
• Codes: Lacking, no culture/support
• Category: Poor MH culture
• Definition: The current mental health culture and atmosphere is inadequate to the needs of its employees and needs to be improved.
• Negative
4. How would you describe Children’s Colorado’s environment to discuss mental health related issues?
Supportive but in a superficial way.
• Codes: superficial
• Category: Poor MH culture
• Definition: The current environment to discuss mental health needs to be improved.
• Negative
5. How would you compare the current mental health culture to years past?
No change.
• Codes: No change in culture
• Category: Inadequate change in culture.
• Definition: The mental health culture has not enhanced from previous years.
• Negative
6. How would you describe the acceptance of mental illnesses by team members? (clarification: sensitive, judgmental)
Judgmental, fearful, lacking knowledge.
• Codes: Judgmental, fearful, lacking knowledge, bad
• Category: Judgmental
• Definition: Critical and not accepting of mental illness.
• Negative
7. What does success in promoting mental wellness look like to you?
Promotion at the same intensity as other health issues.
• Codes: Equal emphasis on MH
• Category: Awareness
• Definition: Awareness through equal promotion and focus of mental health to that of other illnesses throughout the hospital.
Respondent 5:
1. What has your experience at Children’s been like?


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
50
I have enjoyed my time here at Children’s. I worked in radiology for the first two years and have been in Apheresis for the remaining years. I do miss the smaller feel of the hospital but understand that with success growth will follow.
• Codes: enjoyed, positive experience
• Category: Positive
• Definition: Experience at Children’s Colorado has been a positive work environment.
• Positive
2. What mental wellness efforts are you aware of through Children’s Hospital of Colorado? (programs, opportunities, resources, changes, updates):
I know that there is an employee assistance program as well as some classes through HR that help with stress.
• Codes: Some awareness
• Category: Some Awareness
• Definition: The individual is aware of some of the resources available at Children’s Colorado, but does not reveal knowledge of a majority of any of the resources and opportunities available. This reflects that Children’s Colorado is failing to convey the available resources to employees.
• Nuetral
3. How would you describe the current mental health culture and atmosphere of Children’s Colorado?
I think Children’s is ahead of most workplaces.
• Codes: ahead, succeeding
• Category: Advanced
• Definition: Children’s Colorado is successful at promoting mental health throughout the hospital.
• Positive
4. How would you describe Children’s Colorado’s environment to discuss mental health related issues?
I don’t feel like there is a robust program that addresses mental health for employees in a proactive manner. I think managers could do a better job of doing mental health check in with staff: Something simple like: rate your stress level this week or your level of enjoyment with current job.
• Codes: needs improvement, lacking leadership
• Category: Poor MH culture
• Definition: The current environment to discuss mental health needs to be improved.
• Negative
5. How would you compare the current mental health culture to years past?
I think there is more awareness of these problems
• Codes: Increased awareness
• Category: Improved
• Definition: The mental health culture has/is improving from past years.
• Positive
6. How would you describe the acceptance of mental illnesses by team members?
(clarification: sensitive, judgmental):
I would say that overall people are very supportive of other staff that have any type of illness.


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
51
• Codes: Supportive
• Category: Supportive
• Definition: Hospital team members are supportive of mental illness among patients and employees.
• Positive
7. What does success in promoting mental wellness look like to you?
Respondent 6:
1. What has your experience at Children’s been like?
Everyone is willing to help each other, and we have a great work environment
• Codes: Supportive, positive work environment
• Category: Positive
• Definition: Experience at Children’s Colorado has been a positive work environment.
• Positive
2. What mental wellness efforts are you aware of through Children’s Hospital of Colorado? (programs, opportunities, resources, changes, updates)
Vitality, The Gary and Spark Talks
• Codes: Some awareness
• Category: Some Awareness
• Definition: The individual is aware of some of the resources available at Children’s Colorado but does not reveal knowledge of a majority of any of the resources and opportunities available. This reflects that Children’s Colorado is failing to convey the available resources to employees.
• Nuetral
3. How would you describe the current mental health culture and atmosphere of Children’s Colorado?
I think it’s out in the open and everyone takes it very seriously.
• Codes: Open, aware
• Category: Supportive
• Definition: The culture of the hospital is supportive in addressing mental health.
• Positive
4. How would you describe Children’s Colorado’s environment to discuss mental health related issues?
We have a very accepting environment of metal heath issues.
• Codes: Accepting, positive environment
• Category: Open
• Definition: The culture of the hospital is open and accepting to discussing mental health.
• Positive
5. How would you compare the current mental health culture to years past?
I think it’s really getting light shed on it in the last year or so.
• Codes: Increased awareness, discussion/focus
• Category: Improved culture
• Definition: The mental health culture has/is improving from past years.
• Positive


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
52
6. How would you describe the acceptance of mental illnesses by team members? (clarification: sensitive, judgmental)
I think everyone is very open and supportive of mental health.
• Codes: Open and supportive
• Category: Supportive
• Definition: Hospital team members are supportive of mental illness among patients and employees
• Positive
7. What does success in promoting mental wellness look like to you?
I think if people felt comfortable talking about their mental health issues without feeling like they were getting judged, that would be a success.
• Codes: nonjudgmental, comfort, openness
• Category: Nonjudgmental
• Definition: Being an organization that has an environment that is accepting and free of judgment.


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
53
Appendix J: MPA Core Competencies Supplement
The core competencies that this capstone project most relied on are described below, as well an explanation of the knowledge and skills gained from each of the courses in relation to the project.
To lead and manage in public governance
Throughout the MPA program at the University of Colorado, efficiency is consistently stressed as a priority for both government and non-profit organizations. To ensure an efficient workforce, the Children’s Hospital of Colorado needed to identify and evaluate its status regarding mental health and direct de-stigmatization efforts towards continued effectiveness.
Working with a nonprofit entity to promote mental health without stressing the current infrastructure of the organization was deemed an efficient and effective approach. Skills and understanding behind the importance of implementing initiatives that highlight cooperation of multiple departments to ensure success were also garnered. The collaboration of the multidisciplinary workgroup was crucial part in developing a successful de-stigmatization approach to mental health. Recommendations for further collaboration were based on the workgroups success as well as their ability to reach multiple facets of the hospital. The findings that emerged from the scholarly literature present the client with evidence that paralleled the findings from the evaluation. Application of the four approaches represented by the literature are the key avenues to provoking organizational wide behavior change and incremental changes, specifically with mental health stigma.
Further, when proposing an organizational policy or initiative it is crucial that the proposed approaches success is alluded to in its ease of implementation, organizational acceptance, and technical feasibility. PUAD 5002 Organizational Management, PUAD 5008,


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
54
Evidence Based Decision Making, and PUAD 5004, Economics served as the primary outlets for application of knowledge and skills garnered from the courses to conduct the research and recommendations for the project.
To analyze, synthesize, think critically, solve problems and make decisions
This competency was referenced throughout multiple aspects of the capstone project. The research methods selected, as well as analysis of the findings served as a crucial component to the client. The information was distributed among the mental health de-stigmatization work group, as well as at the Association of Community Health Improvement (ACHI) conference to express the organizations success. As alluded to in the previous section, the recommendations stemmed not only from the scholarly literature, but also as a result of analysis of the data. Measures that indicated success were recommended for continuation, with slight improvements to address levels of participation.
Throughout the analysis of the variables for this project numerous limitations arose that required problem solving in assessing which data to include to tell the story of the hospitals efforts. This competency relied heavily on knowledge and skills garnered from PUAD 5008 Evidence Based Decision Making, and PUAD 5003 Research and analytical methods. Communicate and interact productively with a diverse and changing workforce and citizenry
This competency was a crucial component throughout the project process due to the time sensitivity that arose soon after the project was initiated. This aspect of the project not only effected the researcher but other departments that served as major contributors to distributing the secondary data. Due to the diversity of the multi-disciplinary workgroup, interactions and communications with each directly reflected the diverse backgrounds and viewpoints that are outlined in the competency. This competency is further illustrated in the program evaluation


CHILDREN'S HOSPITAL MENTAL HEALTH DE-STIGMATIZATION EVALUATION
55
paper being a communication tool to the diverse audiences. Knowledge and skills from all MPA courses were applied, PUAD 5001 Introduction to Public Administration, and PUAD 5005 Policy process played an essential role in developing the project to be understood by various audiences.


School of Public Affairs
UNIVERSITY OF COLORADO DENVER
Form Name: Submission Time: Browser:
IP Address: Unique ID: Location:
capstone repository permission May 14, 2018 9:04 am IE 11.0/Windows 7 66.128.217.192 407759652
39.729000091553,-104.95279693604
Description Area
SCHOOL OF PUBLIC AFFAIRS ELECTRONIC CAPSTONE REPOSITORY
Description Area Dear Capstone Author and Capstone Client:The Auraria Library
Digital Library Program is a nonprofit center responsible for the collection and preservation of digital resources for education.The capstone project, protected by your copyright, and/or created under the supervision of the client has been identified as important to the educational mission of the University of Colorado Denver and Auraria Library.The University of Colorado Denver and Auraria Library respectfully requests non-exclusive rights to digitize the capstone project for Internet distribution in image and text formats for an unlimited term. Digitized versions will be made available via the Internet, for on- and off-line educational use, with a statement identifying your rights as copyright holder and the terms of the grant of permissions.Please review, sign and return the follow Grant of Permissions. Please do not hesitate to call me or email your questions.Sincerely,Matthew C. MarinerAuraria LibraryDigital Collections ManagerMatthew.mariner@ucdenver.edu303.556.5817
Grant of Permissions
Description Area In reference to the following title(s):
Author (Student Name) Sarah Aldrich
Title (Capstone Project Title) Colorado Children's Hospital Mental Health De-Stigmatization Evaluation
Publication Date 5-2-18
I am the: Client
Description Area As client of the copyright holder affirm that the content submitted is identical to that which was originally supervised and that the content is suitable for publication in the Auraria Library Digital Collections.


Description Area
Description Area This is a non-exclusive grant of permissions for on-line and off-line use for an indefinite term. Off-line uses shall be consistent either for educational uses, with the terms of U.S. copyright legislation's "fair use" provisions or, by the University of Colorado Denver and/or Auraria Library, with the maintenance and preservation of an archival copy. Digitization allows the University of Colorado Denver and/or Auraria Library to generate image-and text-based versions as appropriate and to provide and enhance access using search software. This grant of permissions prohibits use of the digitized versions for commercial use or profit.
Signature
~P 7
Your Name Katie O'Connor
Date 5-14-18
Email Address
ATTENTION
Description Area Grant of Permissions is provided to:
Auraria Digital Library Program / Matthew C. MarinerAuraria Libraryl 100 Lawrence | Denver, CO 80204matthew.mariner@ucdenver.edu303-556-5817


School of Public Affairs
UNIVERSITY OF COLORADO DENVER
Form Name: Submission Time: Browser:
IP Address: Unique ID: Location:
capstone repository permission
May 14,2018 4:32 pm
Chrome 58.0.3029.110 / Windows
71.196.235.242
407910214
39.731300354004, -104.96920013428
Description Area
SCHOOL OF PUBLIC AFFAIRS ELECTRONIC CAPSTONE REPOSITORY
Description Area Dear Capstone Author and Capstone Client:The Auraria Library
Digital Library Program is a nonprofit center responsible for the collection and preservation of digital resources for education.The capstone project, protected by your copyright, and/or created under the supervision of the client has been identified as important to the educational mission of the University of Colorado Denver and Auraria Library.The University of Colorado Denver and Auraria Library respectfully requests non-exclusive rights to digitize the capstone project for Internet distribution in image and text formats for an unlimited term. Digitized versions will be made available via the Internet, for on- and off-line educational use, with a statement identifying your rights as copyright holder and the terms of the grant of permissions.Please review, sign and return the follow Grant of Permissions. Please do not hesitate to call me or email your questions.Sincerely,Matthew C. MarinerAuraria LibraryDigital Collections ManagerMatthew.mariner@ucdenver.edu303.556.5817
Grant of Permissions
Description Area In reference to the following title(s):
Author (Student Name) Sarah Aldrich
Title (Capstone Project Title) Children's Hospital of Colorado Mental Health De-Stigmatization Evaluation
Publication Date May 2, 2018
I am the: Author (student)
Description Area As copyright holder or licensee with the authority to grant copyright permissions for the aforementioned title(s), I hereby authorize Auraria Library and University of Colorado Denver to digitize, distribute, and archive the title(s) for nonprofit, educational purposes via the Internet or successive technologies.


Description Area
This is a non-exclusive grant of permissions for on-line and off-line use for an indefinite term. Off-line uses shall be consistent either for educational uses, with the terms of U.S. copyright legislation's "fair use" provisions or, by the University of Colorado Denver and/or Auraria Library, with the maintenance and preservation of an archival copy. Digitization allows the University of Colorado Denver and/or Auraria Library to generate image-and text-based versions as appropriate and to provide and enhance access using search software.
This grant of permissions prohibits use of the digitized versions for commercial use or profit.
Signature
Your Name Sarah Aldrich
Date May 14, 2018
Address
ATTENTION
Description Area Grant of Permissions is provided to:
Auraria Digital Library Program / Matthew C. MarinerAuraria Libraryl 100 Lawrence | Denver, CO 80204matthew.mariner@ucdenver.edu303-556-5817


Full Text

PAGE 1

STIGMATIZATION EVALUATION -Stigmatization Evaluation Sarah Aldrich University of Colorado Denver School of Public Affair Spring 201 8

PAGE 2

MENTAL HEALTH DE STIGMATIZATION EVALUATION 2 Capstone Project Disclosure This client-based project was completed on behalf of Hospital and supervised by PUAD 5361 Capstone course instructor Dr. Wendy Bolyard and second faculty reader Dr. Pamela Medina . This project does not necessarily reflect the views of the School of Public Affairs or the faculty readers. Raw data were not inc luded in this document, rather relevant materials were provided directly to the client. Permissions to include this project in the Auraria Library Digital Repository are found in the final appendix . Questions about this capstone project should be directed to the student author.

PAGE 3

MENTAL HEALTH DE STIGMATIZATION EVALUATION 3 Table of Content s 3 4 6 Method ology 14 1 9 Discussio n . 2 3 5 Limitations .. 2 7 9 Appendices A Mental Health Work Group Organization Outline .. 3 4 B Variable Descriptions 5 C Interview Questions... . 3 6 D 7 E Results Table ... .. . ........... . 3 8 F Trainings Results Table .. . . 40 G Employee Survey ... .. . 4 1 H Content Analysis Themes ... 4 2 I . 3 J .. 5 2

PAGE 4

MENTAL HEALTH DE STIGMATIZATION EVALUATION 4 Executive Summary Many organizations have a desire to change their organizational culture. One such the client for this project, which requested an analysis of their mental health de stigmatization efforts. This mental health de stigmatization evaluation investigates this issue thr ough analysis of secondary data from 2016 to 2017 . Three major themes from the findings emerged. First, statistical analysis revealed that there has been a significant increase in mental healt h communications and media, both internally and externally. Second, employee participation has increased in all mental health efforts aside from the online mental health reviews. Third, employee responses to the mental health efforts yields overall positiv e results while stressing the importance of progressive improvement. T hree recommendations are offered. First, includ e online mental health resources that patients and families can engage in through mental wellness reviews and trainings . Second, develop and implement an organizational policy that incorporates requirement s for mental health trainings and deter s declines in mental health program participation. Finally, incorporate more contact based approaches to target de stigmatization . With this informat ion, the client will be better able to assess the status of their mental health efforts including areas for improvement and the employee and family response to the 2017 initiatives.

PAGE 5

MENTAL HEALTH DE STIGMATIZATION EVALUATION 5 Stigmatization Evaluation Mental health illnesses have permeated the nation, and the stigmatization of those illnesses presents barriers, negative experiences, and impairments to those seeking appropriate help. Research reveals that 8 of 10 workers with mental health illness report ed that shame and stigma prevented them from seeking needed mental health care (National Alliance of Mental Illness, n.d.). The discrimination that those with mental illness face is rooted in various misconceptions and a lack of knowledge and understanding . Anti stigma and mental health improvement programs have gained popularity in hopes of normalizing public perception, removing barriers, identifying mental illnesses, and increasing understanding and knowledge . In the field of public admini stration, organizational attempts have been made to create a culture that promotes a n environment of openness and acceptanc e . Overall health and wellbeing infiltrate organizations as a top priority, with evidence pointing to the direct impact of organizati onal culture on performance outcomes (Lee & Yu, 200 4 ). of Colorado , the client for this capstone project, is a nonprofit hospital established in 1908 to serve kids of all ages and stages of life provides a network of care through devoted to the mission of improving the health of c hildren through high quality patient care, 750,000 patient visits per year (inpatient, outpatient, and emergency), positioning it as the leading treatment hospi

PAGE 6

MENTAL HEALTH DE STIGMATIZATION EVALUATION 6 changes at the individual, organizational, and social levels, and positioning the hospital as a leading voice in the community for promoting youth mental health. Health care providers are uniquely positioned to reduce the stigma through their direct relationship with pa tients. The mental health stigma, defined as viewing someone negatively due to their condition, includes prejudice and discrimination that influences both the individual and the organization (Mayo Clinic, 2017) . The hospital has implemented numerous mental health de stigmatization approaches to benefit both health care professionals and patients. De Stigmatization Efforts address various evidence based strategies to drive cu lture change. The workgroup illustrates their understanding of the value of forming a multidisciplinary team to effectively develop and implement an in house initiative and promote mental wellness. The workgroup consists of various representatives across m ultiple departments. The de stigmatization efforts target all interactions that occur within the hospital, including employees, patients, visitors, and families. The initiative focuses on education , exposure to persons with mental illness, communication and advocacy for mental health, and organization policies designed to provide suppor t to promote mental wellbeing . An organizational model of the mental health multidisciplinary workgroup is outlined Appendix A. The hospital recognizes ition as a health determinate and its relation to increased occurrence of chronic diseases and use of medical care (CDC , 2011). Mental wellness strategies

PAGE 7

MENTAL HEALTH DE STIGMATIZATION EVALUATION 7 management of their own mental hea Colorado also acknowledges the opportunity to identify policies and practices that could better prevent patients and their families suffering from mental illness from feeling stigmatized when seeking treat ment. stigmatization efforts allows the workgroup to identify the effectiveness of the various approaches, successes, and areas needing improvement. This p aper contains four sections. The first details literature on various mental health programs, the second outlines the methods of evaluation, followed by an examination of the results, and conclud es with a discussion of the findings and recommendations. Literature Review Negative stigmatization of mental illness has created numerous barriers in the mental health field resulting in distress, a reluctance to seek help, and decreased opportunities, specifically for health care providers. The healthcare system serves as a key environme nt in which stigma and discrimination are experienced by individuals with mental illnesses (Knaak, house mental health anti stigma initiative, a literature review of similar programs is crucial. The studies below outline controlled interventions, cross sectional surveys, and longitudinal panel studies used to identify program effectiveness. This review explores the varying solutions and effectiveness of programs in red ucing stigma by looking at social contact, education, online media, and workplace based approaches. Contact Based Programs Inclusion of personal narratives and experiences from persons with mental health

PAGE 8

MENTAL HEALTH DE STIGMATIZATION EVALUATION 8 employees. Social contact based approaches for reducing mental health stigma consi st of direct contact among individual participants, as well as the inclusion of persons with mental health illnesses. Social contact has been identified as the most effective approach in improving stigma related knowledge and attitudes for adults ( Thornicr oft et al., 2016). Contact based approaches employ educational techniques in efforts to increase knowledge and confidence in treating mental health patients among health care providers. Programs have resulted in positive correlations between increased leve ls of confidence, comfort, and improvement and contact based skill building approaches ( Beaulieu et al., 2017). Knaak and Patten (2016) identified strategies for effective de stigmatization program planning to incorporate collaboration, partnerships, networks, participation, and involving individuals with lived mental health experience. Involvement of individuals with lived experience entails persons with mental illness presenting their testimonies and recovery stories. When targeting healthcare provid ers specifically, success has been identified through evaluation in incorporating pessimistic views towards recovery, seeing the illness of the person, identifying lack of competence and confidence, and a lack of awareness in person al prejudices (Knaak & P atten, 2016). Social contact is most effective when there is equal status between participants, as well as a common goal for interaction and cooperation ( Thornicroft et al., 2016) . Studies on direct social contact between the population and persons with m ental illness have revealed lower levels of agreement with stigmatizing statements, improved perception of dangerousness and disturbance attached to a person with mental illness, and a growing sense of integration in day to day life ( Villani & Kovess Masfe ty, 2017).

PAGE 9

MENTAL HEALTH DE STIGMATIZATION EVALUATION 9 However, when looking at the long term effects of social contact on retention of program design, evidence is weak ( Thornicroft et al., 2016). This demonstrates the importance of developing programs that are adaptive and consistent in their provision of opportunities ( Knaak & Patten, 2016). Contact based interventions demonstrate the need for long term follow up as a result of many mental health stigma reduction programs being in their infancy. Incorporating elements of the social contact based approach serves as a component in designing a strategy for an effective mental health program intervention. The inclusion of personal narratives and experiences helps develop a foundational understanding of what mental illness is , w hile simultaneously provoking an open environment for dialog ue around mental health. Education Based Programs Educational aspects are present in many parts of interventions and serve as a c entral component in addressing mental illness stigmatization. Incorporating consumer contact and educational interventions among diverse populations has been identified as most effective in reducing public stigma (Griffiths, Carron Arthur, Parsons, & Reid, 2014). Contact bas ed interventions were more effective than educational approaches for adults, while educational initiatives prevailed among younger populations (Griffiths et al . , 2014). Li, Juan, Huang and Thornicroft (2014) determined that, overall, the educational traini ng they implemented changed community health staff attitudes towards mental illness, decreased social distance between staff and individuals with mental illnesses, and revealed that it was possible to train community health staff with a short course in men tal health stigma. Educational interventions are developed to improve knowledge and understanding of signs, symptoms, treatments, and availability of mental health resources. The literacy component

PAGE 10

MENTAL HEALTH DE STIGMATIZATION EVALUATION 10 implies that enhanced knowledge and awareness about ment al health stigma and discrimination will arm individuals to seek appropriate action for treatment (Stuart, 2016). Programs that target specific groups resulted in significant improvements in enhancing knowledge about treatments, decreased social distance, and produced increased confidence in providing help to individuals with mental illnesses (Stuart, 2016). Awareness raising is another key component involved in educational approaches, which encourage mental health workers to engage in a dialogue about me ntal health and recommend next steps to make healthcare services a reality for people with mental illness (Stuart, 2016). E ducating, raising awareness, and opening a dialogue, studies predict higher social tolerance towards mental health issues among colle ge students and adults (Stuart, 2016). Organizational culture plays a significant role in the level of mental health stigmatization within an organization. Knaak, Mantlet, and Szeto (2017) explored the idea that solutions for stigma reduction need to be approached with the goal of culture change. Two key components for interventions were identified as including transformative learning targeted on bias and false beliefs, as well as emphasizing the roles that healthcare providers play in the recovery proces s (Knaak et al., 2017). Educational based intervention s should incorporate workshops, skills based approaches, and intensive social contact (Knaak et al., 2017). Research stresses the importance of education through findings that mental health knowledge predicts intentions to seek help, through its association with attitudes of tolerance and community support, underlining the importance of mental health literacy (Henderson, Evans Lacko, & Thornicr oft, 2013). E ducational approach to stigma reduction findings are important in narrowing down an effective in house hospital approach. Application of the direct contact component to appease

PAGE 11

MENTAL HEALTH DE STIGMATIZATION EVALUATION 11 adults while stressing the effectiveness of educational initiativ es in targeting younger individuals is a valuable tool in implementing de stigmatization efforts across a diverse population . Online Programs With technological advancement and internet based tools, the exploration of online mental health programs and re sources has increased. Numerous mental health programs have Hospital offers various online trainings and media communications to its employees and patients. M ental he alth initiatives that consisted primarily of an online education program s w ere success ful in increasing literacy, decreasing negative perception of mental illness, and providing advice on how to promote seeking help within the workplace ( Griffiths, Bennett, Walker, & Goldsmid, 2016). The intervention revealed individuals who did not participate in the universal online program possessed higher depression and anxiety stigma score s , and lower literacy for the mental illnesses th a n participant s who completed the assessment (Griffiths et al., 2016). Other online programs reveal similar results in producing more positive attitudes towards individuals with mental illness (Kaolis et al., 2017). Digital interventions serve as a powerful strategy in facilitating widespread behavioral and cultural change, while simultaneously reducing barriers to access, increasing participant engagement and adherence ( Hanisch, Birner, Oberhauser, Nowak, & Sabariego, 2017). Interactive programs have been developed to increase engagement and participation with what could seem a cumbersome task of devoting additional staff, time, and resources . Leadership training in Mental Health Promotion (LMHP) was designed as a digital game to improve knowledge, attitudes, and self efficacy with handling mental health situations at work ( Hanisch et al., 2017 ). An evaluation of the digital mental health program revealed decreased perception of

PAGE 12

MENTAL HEALTH DE STIGMATIZATION EVALUATION 12 stigmatizing attitudes such as avoidance, perceived dangerousness, and responsibility as a result of the program (Hanisch et al., 2017). Online approaches can be utilized for not just interactive trainings, but also for the provision of mental health resources. Fact sheets, toolkits, guidelines, reviews, discussion forums, documentaries, and vid eos consisting of personal stories can be accessed to reduce stigma and discrimination ( Cerully, Collins, Wong,, Roth, Marks,, & Yu, 2016 ). An evaluation of website traffic and activity revealed high numbers of website utilization, and 66,000 downloads of mental health resources ( Cerully, Kase, Collins, Wong, & Yu, 201 5 ). This further demonstrates that websites can potentially serve as a popular avenue for spreading information. With the consistent increase in the use of online tools, media can serve as an important in communicating widespread information about mental health. When looking into implementation within the , the online options provide flexibility to the busy schedules of healthcare providers as well as opportunities for self paced learning. Workplace Initiatives Workplace interventions consist of periodically implementing mental health programs to target the stigmatization that persists and allow for targeting of specific groups and tailoring programs to the organization illnesses has become an issue for employers globally ( Hanisch et al., 2017). Hanisch et al. (2017) emphasizes the needs for organizations to focus on both the employee level of stress management a s well as the organizational working conditions. Employee mental health is affected by the social environment that includes organizational culture, levels of support, and leadership styles (Hanisch et al., 2017). The role of leadership plays an important p art in effective and efficient implementation of mental health stigma

PAGE 13

MENTAL HEALTH DE STIGMATIZATION EVALUATION 13 reduction programs , as well as an understanding that m ore people could be reached by targeting interventions within organizations rather than to the general public ( Hanisch , Twomey, Szet o, Birner, Nowak, & Sabriego , 2016). Workplace anti stigma interventions have revealed positive problems ( Hanisch et al., 2017). Social distance has been a reoccurring barrier within mental health. It is defined as a (Cerully et al., 2016, p. 1 ). Workplace Stigma Discrimination Reduction P rograms resulted in reduced desire for social distance and increased understanding of how to support individuals with mental illnesses (Cerully et al., 2016). With individuals being more willing to engage with persons with mental illnesses, a dialogue will be opened, and an associated reduction of barriers is expected to occur. Even further, Cerully et al. (2016) discovered workplace trainings produced positive beliefs in recovery, greater perceived ability to provide support, and heightened effectiveness i n examining aspects of social contact. Summary The literature above alludes to various tools that measure effectiveness including software, surveys, and interviews. These tools have been applied to inform the methodology section of this paper. As implied from the literature, creating a positive mental health culture assumes higher utilization of resources and decreased stigmatization which is the goal of the stigmatization efforts. Interventions targeting health care providers are un common. After reviewing a variety of programs, key components are identified that remained constant throughout. The effectiveness of social contact and educational approaches is evident in the literature as producing the most

PAGE 14

MENTAL HEALTH DE STIGMATIZATION EVALUATION 14 significant outcome in reducin g mental health stigma. The literature serve s as a guide in designing a mental health program as well as making recommendations for existing ones. The literature supports that mental health programs decrease stigmatization, thus influencing increased utilization of available resources through increas ed knowledge and awar eness. H ospital offer s an assortment of educational trainings for hospital employees as well as guidance for families to promote their mental wellness. These workshops provide information about the importance o f mental health, how to recognize when someone might be in need of mental health support, and how to recognize and respond to unconscious bias. Through the multidisciplinary workgroups prioritization of exposure, hospital team members witness personal ment al illness testimonies from administrators and teammates through a speaking series outlet called Spark Talks. Through this outlet hope s to build an accepting environment oriented around support and awareness towards promoting mental wel improving mental health services for children throughout Colorado. Policies and programs that they advocate for include increased screening, early intervention services, integrated systems of behavioral and physical healthcare, and expanded access to prevention and high quality mental health care. By evaluating the effectiveness of the hospital Colorado will have a greater understanding of existing gaps, as well as the impact of the programs on utilization of mental health resources. The outcomes based program evaluation will help determine if the de stigmatiza tion efforts are effective. Resource utilization and prevalence of mental health in hospital communications will serve as the primary measures of success ,

PAGE 15

MENTAL HEALTH DE STIGMATIZATION EVALUATION 15 i exist ing resources. Method ology The program evaluation evaluates mental health de stigmatization efforts, their impact on utilization of resources, and change in organizational culture. In order to ex plore the effectiveness of the initiative, a mixed methods design i ncluded a comparison of secondary data and semi structured phone interviews . Utilization of available resources was reviewed and compared to the previous year, and individual perceptions of the efforts from both team members and patients were examined . Incorporating both quantitative and qualitative methods informative, complete, balanced, and useful research result p.18). The evaluation investigates the following research question: What are the impact of the mental health de Hypotheses The literature strongly suggests that the implementation of contact, educat ion, online, and workplace based approaches will achieve success in reducing the mental health stigma. The de Colorado include aspects of all four of the approaches to some extent . There are s ix hypotheses stigmatization efforts. 1. Employee perceptions and knowledge of hospital mental health efforts will increase from 2016 to 2017. 2. There will be increased u tilization of mental health resources by employees through the Vitalit y program from 2016 to 2017 .

PAGE 16

MENTAL HEALTH DE STIGMATIZATION EVALUATION 16 3. There will be increased e mployee participation in educational trainings from 2016 to 2017 . 4. Mental health hospital communications will increase internally and externally from 2016 to 2017. 5. Analysis of secondary data will indic ate increase d measures in m ental health screenings by from 2016 to 2017. 6. There will be decreases in individuals who measure as low and at risk categories for mental wellbeing when looking a t resilience, psychological factors, and stressors from 2016 to 2017. Measurement The various de stigmatization efforts educate employees and visitors on available resources and help them identify mental health illnesses within their personal lives, famil ies , and peers. Resource utilization is examined to determine the impact of increased knowledge of mental health and availability of resources , a s well as analysis of various indicators to measure impact. An outcomes based evaluation assess es the impact and extent to which the organization reaches its intended goals through the initiative (McNeil, 2011). In developing the mental health enhanced knowledge of mental health and resources, increased use of resources, and increased comfort in identifying and discussing mental health issues. The overall objective is to decrease the negative mental health stigma throughout the hospital. Numerous variables have been selected for analysis and are outlined in Appendix B with a description of their measurements and target groups . The outcomes fluctuate by variable, with some including a comparative component from previous years, and others only allowing quantification of use. The independent variable throughout the evaluation is the program de -

PAGE 17

MENTAL HEALTH DE STIGMATIZATION EVALUATION 17 stigmatization efforts, with the utilization of resources , knowledge and comfort as the depen dent variable. There were two different approaches applied for measurement ; content analysis and analysis of secondary data. Quantitative The seven key variables defined in Appendix B identify potential increases or decreases between 2016 and 2017, as w ell as to quantify the utilization of available resources. In instances where there is comparison, percent change was calculated to provide percent increases and decreases while acknowledging the varying numbers included in each population studied. Descriptive statistics we re also used to measure changes among the quantitative variables. Secondary d ata a nalysis. Secondary data sources included external media communications, internal site hits, Vitality measures, and the employee survey responses. An alysis of secondary data assess ed was within the newer opportunities available. Media communication were retrieved from the PMHI marketing team, where the number of hits, reach, and posts were quantified, and mental heal th mentioned were removed for further analysis. hits and unique users were then quantified to reveal the level of utilization. The vitality program was analyzed, where al l programs pertaining to mental health were selected, and compared to measures from the previous year. Access to employee response surveys was also granted where questions relating to mental health and wellness were selected, compared, and quantified. The client asked to identify the level of utilization of resources by calculating the frequency of program usage compared to other popular efforts at the hospital. These indicators serve the hospital in determining what resources are deemed successful. Suc cess here is defined as average to high use of resources compared to other available options.

PAGE 18

MENTAL HEALTH DE STIGMATIZATION EVALUATION 18 The measures conducted throughout the evaluation focused primarily on the de stigmatization efforts impacts on employees. The external communications were the on ly measures that examined the amount of interaction with patient families. Content Analysis . Additionally, employee interviews and internal communications through weekly emails were assessed and compared with the previous year. Each of the 48 emails for 2 016, and the 51 emails for 2017 were reviewed for any mentions and references to mental health and wellness. Qualitative Measures Semi structure d interviews conducted via email were used to determine program effectiveness among employees from various departments and positions . The interviews gathered participants opinions, perception s, and attitudes through thorough questioning ( Harrell & Bradley , 2009). Employees perceptions of the hospitals environment regarding mental health and de stigmatization e fforts was assessed. A conventional approach to analysis was applied, where responses to mental health stigmatization status throughout the hospital were investigated. The process consisted of four stages. First, c odes were derived from interview responses to capture key concepts in which labels for the codes emerged. The codes were then organized into categories based on their overall relationship throughout the responses . D efinitions and descriptions for each of t he c odes were then constructed . Finally, clusters were derived from each category to indicate positive, negative or a n eu tral reflection of the hospital efforts , and results are reported out. A conventional approach was selected for analysis because of its processes revealing direct information from respondents, without inflicting preconceived categories (Hsieh & Shannon, 2005, p. 1281).

PAGE 19

MENTAL HEALTH DE STIGMATIZATION EVALUATION 19 Sampling Plan Interview participants were purposely selected from various hospital departments, with participants having varying positions ranging from clinical to administrative . Interviews assessed the stigmatization efforts . The client passed on sixteen individuals contact information of whom were not a part o f the mental health work group. S ixteen individuals were contacted to participate in the interview process, with only six being willing to respond. Data was compared from 2016 to 2017 due to the mental health work group beginning their de stigmatization e fforts in early 2017. The comparison identified changes in utilization and perceptions of organizational culture in order to identify a possible correlation with workgroup efforts and increased/decreased utilization of resources and changes in individual p erceptions. Validity and Reliability To ensure face validity the interview questions were previously discussed with the client and explained to each participant . The questions asked in the interviews are included in Appendix C . Analysis of the responses followed the conventional coding process because of its ability to produce descriptive codes and themes ( Saldana, 2009 ). The design of the interviews presents potential selection bias in its participants. Data Analysis Mental health de stigmatization success is identified as any increase in level of participation, use of available resources, and positive feedback from both employees and patient families. The quantification of participation and utilization underwent analy sis through the identification of percent increases and testing for statistical significance. The interview conventional content analysis framework elicits categories of information , where responses are

PAGE 20

MENTAL HEALTH DE STIGMATIZATION EVALUATION 20 expanded into sub categori es based on responses , and themes are developed from the outcomes ( Saldana, 2009). Appendix D indicate s each hypothesis that was examined and whether or not it was supported by the evaluation results. Results Overall, analysis of the results from 2017 revealed increases in a majority of utilization variables, and positive mental wellbeing indicators. Appendix E outlines the overall results of the variables analyzed throughout the evaluation. Communications and Media A nalysis of hospital internal and external communications yielded positive increases across all domains. Internal communications. Weekly emails are distributed to hospital employees highlighting upcoming events, words of encouragement, and various articles linking information on important topics. A search for mental health topics, articles, and events was conducted in order to quantify how often mental health was referenced both prior to de stigmatization efforts and after efforts had been initiated. Weekly internal emails were examined for 2016 and 2017, yielding 48 and 51 respectively. Each email was individually read in order to identify mental health mentions. For each mental health mention a tally was given, and the results calculated for comparison bet ween each year. The reasoning for examining communications is that with more mental health references, there will be increased opportunity for discussion and dialog ue , which decreases the negative stigmatization mental illness receives. Analysis of the men tal health mentions from 2016 to 2017 reveals a n increase of 34.6% (n=9 26) . External Communications. External communications consist of emails and media pushes sent out to patients, families and other members of the community. Media stories were

PAGE 21

MENTAL HEALTH DE STIGMATIZATION EVALUATION 21 calculated for various media channels through secondary data . The number of mental health media stor i es increased by 82% from 2016 to 2017, accounting for 1,022 different stories . Employee i ntranet . l health information and resources can be accessed via the mental wellness page. The employee intranet shows the level of interaction that employees have engaged in with the mental health page since its creation in 2017. Unique clicks and visitors were ana lyzed for 2017 indicating increases in both unique users and intranet hits in September of 157 and 95 respectively. Online Mental Health Assessment Various online mental health assessments were offered through V itality, the hospitals platform that houses and tracks employee wellness. The mental wellbeing review experienced a slight decrease of 0 .8% ( n= 4,155 4,335 ) in total participants, with the V itality review program eliciting a slight 0 .9 % ( n= 2,863 2,877 ) increase in total participants. Through the V itality program the employee participants level of stress is measured as low risk, at risk, and high risk. The goal of the hospital is to have a continued increase and higher percentages of employees in the lo w risk category , while steadily decreasing the number in the high and at risk categories for stress. Stressed measure d via the V itality platform indicated a 1.15% increase in low risk, with a .88% decrease in at risk and .28% decrease in high risk (n=4,146 4,322) . When looking more specifically at psychological well being, stressors ( 1.29%, 1.4% , n=2,87 3 2,847 ) and resilience ( .1%, 1.4% , n=2,712 2,662 ) decrease d in high and at risk percentages at the population level. Results also indicated decreases in social support, which is the physical and emotional comfort and help received from family, friends, coworkers and others. Further V itality measures revealed a 10.5% (n= 3,213 ) decrease in individuals who reported being stressed.

PAGE 22

MENTAL HEALTH DE STIGMATIZATION EVALUATION 22 Employee Trainings Var ious employee trainings were offered that incorporated aspects of mental health and wellbeing. Comparison of trained employees revealed a 78% ( n =35 64) increase in employees who participated in the Youth Mental Health First Aid training and a 73% ( n =62 107) increase in employees who participated in Unconscious Bias . From the Spring of 2017 to Fall there was a 15.2% ( n =33 38) increase in employees registered in the mindfulness course, and a 60% ( n =15 24) increase in average session attendance. Appendix F Mental Health First Aid and Unconscious Bias from 2016 to 2017. Surveys Numerous surveys were administered by the evaluator among hospital employees to gauge their perception and knowl edge of mental illness, hospital resources, and perception s of barriers to accessing mental health resources . The annual wellness survey indicate s a 7.87% ( n =32 86) decrease in stigma as being recorded as a barrier to accessing mental health resources at C . The 2017 employee wellness survey further indicated that of the respondents, 35.11% ( n = 581) described their health and wellness goals as focusing on mental health. S tress management and mental health seminars were selected as being of greatest interest to 61.3% and 39.96% of 553 participants respectively. However, results also revealed that of the participants who reported utilizing the Employee Assistance Program (EAP) in 2017, only 13.64% ( n =484) claimed to use mental health resources. A n overall employee population decrease of 16.6% (n= 100 564 ) was identified in participants claiming to understand how to access EAP resources.

PAGE 23

MENTAL HEALTH DE STIGMATIZATION EVALUATION 23 The 2017 employee opinion survey measured responses to mental health, stress, and resiliency. Responses were compared from 2016 to 2017 and revealed a 3% increase in participants viewing the amount of stress the y feel as reasonable a s well as 62% ( n =556) of respondents reporting that their supervisor provides an accepting en vironment to discuss mental health related issues in 2017. The employee surveys were analyzed by picking out questions related to mental health and comparing and reporting the findings to previous year when applicable. Survey response charts are included in Appendix G . Screenings ere only compiled in 2017. However, the 2017 data reveal a screening rate of 78.2% in all e mergency department and urgent care lo cations for patients ages 10 and up, and an overall screening rate of 80% for children not previously screened in urgent care centers. The hospital also engineered efforts to screen caregivers, where 8 , 000 screenings were conducted on parents to assess the ir mental health in 2017. Employee Interviews In late March, 16 emails that included seven questions assessing the hospital s culture and acceptance of mental health were distributed among a variety of employees. A response rate of 38 % (n=6) allowed further investigation into the perceptions from clinical and administrative staff of hospital efforts . Of the respondents there was a project engineer, a clinical assistant, a nurse, a nd three administrative employees. Responses were coded and categorized, yielding overall positive, negative , and n eu tral results . The overall theme reflects the majority of the s of their experience s and views of mental health culture. Appendix H

PAGE 24

MENTAL HEALTH DE STIGMATIZATION EVALUATION 24 outlines the result s per category and cluster for each question. Question seven was used as a tool Overall, the data reflects positive results of an improve d mental health culture from previous years, employee experience, as well as a positive mental health culture. However, discuss mental health. All of the interview participants were found to be only some what aware of the hospitals mental health efforts, resources, and programs. Additionally, mixed results o n acceptance by team members with mental illness were reported . It is important to note that only six hospital employees were interviewed among the 7 ,000 benefits eligible employees that work Appendix I includes the content analysis coding process and interview responses for the conventional content analysis. Discussion The results highlight the success of the mental health de stigmatization efforts achieved after one year of initiation. S imultaneously , the results illustrate tha t organizational de stigmatization is a continuing process that elicits room for improvement. Program success has been rooted in the cooperation and commitment of various hospital departments to a ddress mental health stigma from all directions. The findings parallel the educational and online program approaches . The literature largely identifies success in incorporating educational components to combat stigma change , enhance organizational culture, and address bias and false beliefs ( Knaak, S. et al., 2017). The forums further reflects online forums being a powerful strategy to facilitate 5 ) findings that

PAGE 25

MENTAL HEALTH DE STIGMATIZATION EVALUATION 25 online approaches can be utilized to reduce stigma and discrimination while increasing the utilization of available resources. Through various communication channels, internet programs , and trainings it is evident that the workgroup has been successful in increasing the prevalence of and interaction with mental health and wellness resources among hospital emp loyees. The initiatives success can be contributed to the workgroup incorporating all four of the approaches deemed effective by the literature. Analysis of employee mental wellness also noted overall positive results in decreasing the percentages of stress among high and at risk employees . Only four achieve its goals. Measured through V itality, these areas were a decrease in social support for the low risk population, no chang utilization of the mental wellbeing review , and a decrease in survey participants knowing how to utilize EAP . The interviews also revealed negative results in respondents indicating an o verall health de stigmatization efforts and resources. Social support was defined by the work group as the physical and emotional comfort received from family, fr iends and coworkers. A number of factors can play into an level of social support; however, the results emphasize the importance of organizational culture that the mental health de stigmatization workgroup was trying to improve . Further effort s to enhance the programs offered through vitality and EAP may co unter the declines related to mental health. Recommendations

PAGE 26

MENTAL HEALTH DE STIGMATIZATION EVALUATION 26 For 2017 a majority of the mental health de stigmatization efforts target ed hospital employees, with only minimal programs addressing patients and families. The inten ded design behind the de stigmatization efforts stemmed from the hope that the efforts would have a trickle down effect from providers to patients. Responses from ho spital employees highlight existing gaps in this belief and inform recommendations to further incorporate patients and families into the de stigmatization program . stigmatization evaluation revealed success in the workgroups efforts. Further focus should be undergone to increase awareness of mental health resources and de stigmatization efforts through continued communications and marketing. T hree recommendations emerge. These could better help the client promote organizational culture change and further address mental health de stigmatization across all individual interactions. Recommendation 1: Include additional mental health resources for patients and families . To address the issue of the lack of mental health specific tools for patients and families, the hospital needs to enhance opportunities for engagement. Through implementing similar tools that are currently offered to employees, such as the mental wellness trainings and online tools , patients and families can also participate in enhancing their knowledge and understanding of mental health. By a llowing patients and families to engage in the online assessments such as the psychological wellbeing, social support, and resilience mental wellbeing r eviews they will be better equipped to engage in mental wellness treatments and services. As affirmed in the literature, the development of online interactive resources would serve as an effective tool for promoting mental wellness, especially among the ad olescent population ( Hanisch, S. et al., 2017). This will not only elicit increased understanding but provoke a similar change in attitudes

PAGE 27

MENTAL HEALTH DE STIGMATIZATION EVALUATION 27 and beliefs about mental illness, thus attributing to culture change and de stigmatization. Future efforts should in vestigate targeting specific groups throughout the community that are more prone to mental health risk factors, such as gender, age, and socio economic status. Recommendation 2: Develop and implement an o rganizational p olicy that requires employees to be trained in mental health/wellness. In order to address the declines in participation and further encourage continued interaction with existing programs and resources, the client needs to implement an organization wide policy . An ideal organizational policy would incorporate efforts to prevent and reduce stress, as well as continually generate a dialog ue for mental health. Developing a policy that requires participation in mental health efforts would ensure that individuals a re receiving continued education. The requirement could be implemented in one of two ways : 1. Requiring all new employees to complete a mental health review and/or training during orientation. 2. Require mental health reviews/trainings to be completed by all e mployees that wish to receive points through V itality. This approach presents ease in implementation due to the hospital and its employees already having embraced the programs and efforts that were implemented in 2017. S imultaneously this approach incorporat es the success Li et al. ( 2014 ) indicated from online approaches and workplace trainings that Cerully et al. (201 5 ) presented as producing positive outcomes in reducing stigma and promoting mental wellness. Additionally, the approach highlight s t he sustainability of the de stigmatization efforts in implementing an organizational policy.

PAGE 28

MENTAL HEALTH DE STIGMATIZATION EVALUATION 28 Recommendation 3: Incorporate more contact based approaches to target de stigmatization In order to target mental health stigmatization, it is important to conti nually incorporate and enhance the social contact component that was shown in the literature to be extremely effective in the literature. The popular S park T alks that the hospital organizes on a monthly basis should deliberately incorporate mental health into the discussions. Hospital employees can attend these talks, as well as listen to them online. This furthers the workgroups ability to reach a large portion of their target population. The topics do not need to be solely on mental health and stigma, but addressing various mental health components such as stress, anxiety, and depression presents a very feasible approach to encourage a dialog ue for mental health. This approach further aid s hospital eff orts to decreas e negative perceptions and stigma of mental illness ( Villani & Kovess Masfety, 2017). Limitation s Various limitations of the mental health de stigmatization efforts evaluation stem from its infancy as a program. The high percentage increases across a majority of the measures can be contributed to the fact that prior to 2017 there was an extreme lack of mental health efforts throughout the hospital. Further analysis should be sought after multiple years of implementation in order to analyze the continued impact over time. Other limitations are a result of the hospitals incomplete documentation from 2016 which limits effective comparison. EAP providers due to inadequate reporting , which hindered analysis of EAP and employee interaction. Varying levels in sample sizes for survey responses, a s well as individual participants could contribute and directly affect the results.

PAGE 29

MENTAL HEALTH DE STIGMATIZATION EVALUATION 29 Limitations were also present in the interviews conducted with t he hospitals employees . ntal health environment, with responses being contradictory. The format of the interviews did not allow further investigation into participants answers and were limited to only six responses. Not every department was represented in the sample, as well as o ne of the respondents not being employed the previous year, thus negating the comparison question. Conclusion The intent behind implementing a grassroot mental health de stigmatization effort was that before the hospital can seek culture change at the community level, they must first address the problems . Through continued collaboration, involvement, and commitment of all hospital employees , mental health stigmatization will effectiv ely be addressed. initiative will reap the most success when it entails tasks and objectives that departments and employees are already doing. Initiatives will be the most technically feasible when the addition of tasks are limited. The importance of a multifaceted approach (education, online, workplace, and contact based) is also a crucial aspect recognized for producing success within reducing mental health stigma th roughout the organization.

PAGE 30

MENTAL HEALTH DE STIGMATIZATION EVALUATION 30 References Bakolis, I., Evans Lacko, S., Henderson, C., Robinson, E., Sampogna, G., & Thornicroft, G. (February 2017). The impact of social marketing campaigns on reducing mental health stigma: Results from the 2009 2014 Time to Change programme. European Psychiatry 40, 116 122. https://doi.org/10.1016/j.eurpsy.2016.08.008 Beaulieu, T., Patten, S., Knaak, S., Weinerman, R., Campbell, H., & Lauria Horner, B. (2017). Impact of Skill Based Approaches in Reducing Stigma in Primary Care Physicians: Results from a Double B lind, Parallel Cluster, Randomized Controlled Trial. Canadian Journal of Psychiatry. Revue Canadienne de Psychiatrie , 62 (5), 327 335. http://doi.org.aurarialibrary.idm.oclc.org/10.1177/0706743716686919 Cerully, J. L., Kase, C. A., Collins, R. L., Wong, E. C., & Yu, J. (201 5 ). CalMHSA Stigma and Discrimination Reduction Online Resources: Highlights from an Evaluation of Web Analytic Data. Rand Health Quarterly , 5 (3), 4. Cerully, J. L., Collins, R. L., Wong, E. C., Roth, E., Marks, J., & Yu, J. (2016). Effect s of Stigma and Discrimination Reduction Trainings Conducted Under the California Mental Health Services Authority: An Evaluation of Disability Rights California and Mental Health America of California Trainings. Rand Health Quarterly , 5 (3), 5. . Retrieved from: https://www.childrenscolorado.org/about/

PAGE 31

MENTAL HEALTH DE STIGMATIZATION EVALUATION 31 Griffiths , K. , Bennett, K., Walker, J., Goldsmid , S. (2016) Anthony Bennett, Effectiveness of MH Guru, a brief online mental health program for the workplace: A randomi z ed controlled trial, In Internet Interventions, 6, 29 39 . Retrieved from: https://doi.org/10.1016/j.invent.2016.09.004 Griffiths, K., Carron Arthur, B., Parsons, A. Reid, R. (June 2, 2014). Effectiveness of programs for reducing the stigma associated with mental disorders. A meta analysis of randomized controlled trials. World Psychiatry , 13(2), 161 165. doi: 10.1002/wps.2 0129 Hanisch, S., Birner, U., Oberhauser, C., Nowak, D., & Sabariego, C. (2017). Development and Evaluation of Digital Game Based Training for Managers to Promote Employee Mental Health and Reduce Mental Illness Stigma at Work: Quasi Experimental Study of Progr am Effectiveness. JMIR Mental Health , 4 (3), e31. http://doi.org.aurarialibrary.idm.oclc.org/10.2196/mental.7600 Hanisch, S., Twomey, D., Szeto, A., Birner, U., Nowak, D., & Sabariego, C. (2016). The effectiveness of interventions targeting the stigma of me ntal illness at the workplace: a systematic review. BMC Psychiatry, 16:1. https://doi.org/10.1186/s12888 015 0706 4 Harrell, M., & Bradley, M. (2009). Data collection methods: Semi structured interviews and focus groups . RAND Corporation. Retrieved from https://www.rand.org/pubs/technical_reports/TR718.html. Henderson, C., Evans Lacko, S., & Thornicroft, G. (2013). Mental Illness Stigma, Help Seeking, and Public Health Programs. American Journal of Public Health , 103 (5), 777 780. Retrieved from http://doi.org.aurarialibrary.idm.oclc.org/10.2105/AJPH.2012.301056

PAGE 32

MENTAL HEALTH DE STIGMATIZATION EVALUATION 32 Hsieh, H., Shan n on, S. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277 1288. Retrieved from http://journals.sagepub.com/doi/pdf/10. 1177/1049732305276687 Johnson, B., Onwuegbuzie, A., Turner, L. (2007). Toward a definition of mixed method research. Journal of mixed method research, 1(2), 112 133. Retrieved from http://journals.sagepub.com.aurarialibrary.idm.oclc.org/doi/pdf/10.1177/155 8689806298 224 Knaak, S., Patten, S. (July 17, 2016). A grounded theory model for reducing stigma in health professionals in Canada. Acta Psychiatrica Scandinavica , 134(5446), 53 62. doi: 10.1111/acps.12612 Knaak, S., Mantlet, E., & Szeto, A. (2017) Mental illness related stigma in healthcare: barriers to access and care and evidence based solutions. Healthcare Management Forum. 30:2, 111 116. https:/ /doi.org/10.1177%2F0840470416679413 Lee, S. K. J., & Yu, K. (2004). Corporate culture and organizational performance. Journal of Managerial Psychology, 19 (4), 340 359. http://dx.doi.org/10.1108/02683940410537927 Li, J., Li, Juan., Huang, Y. & Thronicroft, G. (December 4, 2014). Mental health training program for community mental health staff in Guangzhou, China: Effects on knowledge of mental illness and stigma. Internat ional Journal of Mental Health systems , 8(49). https://doi.org/10.1186/1752 4458 8 49

PAGE 33

MENTAL HEALTH DE STIGMATIZATION EVALUATION 33 McNeil, R. C. (2011). A program evaluation model: Using Bloom's taxonomy to identify outcome indicators in outcomes based program evaluations. MPAEA Journal Of Adult Edu cation , 40 (2), 24 29. Retrieved from: http://aurarialibrary.idm.oclc.org/login?url=http://search.ebscohost.com/login.aspx?direct =true&db=eft&AN=85651355 Mayo Clinic. (2017). Mental health: Overcoming the stigma of mental illness Retrieved from: https://www .mayoclinic.org/diseases conditions/mental illness/in depth/mental health/ART 20046477 National Alliance of Mental Illness. (n.d.). Stigma free company . Retrieved from: https://www.nami.org/stigmafreeco Saldana, J. (2009). An introduction to codes and coding. In The coding manual for qualitative researchers (Chapter 1). Retrieved from http://stevescollection.weebly.com/uploads/1/3/8/6/13866629/saldana_2009_the coding manual for qualitative researchers.pdf Stuart, H. (2016). Reducing the stigma of mental illness. Global Mental Health , 3 , e17. http://doi.org.aurarialibrar y.idm.oclc.org/10.1017/gmh.2016.11 Thornicroft, G., Mehta, N., Clement, S., Evans Lacko, S., Doherty, M., Rose, D., Koschorke, M., Shidhaye, R., O'Reilly, C., & Henderson, C. (2016). Evidence for effective interventions to reduce mental health related stigma and discrimination, In The Lancet, Volume 387, Issue 10023, 1123 1132 . Retrieved from: https://doi.org/10.1016/S0140 6736(15)00298 6.

PAGE 34

MENTAL HEALTH DE STIGMATIZATION EVALUATION 34 Ungar, T., Knaak, S., & Szeto, A. C. (2016). Theoretical and Practical Considerations for Combating Mental Illness Stigma in Health Care. Community Mental Health Journal , 52 , 262 271. http://doi.org.aurarialibrary.idm.oclc.org/10.1007/s10597 015 9910 4 Villani, M., & Kovess Masfety, V. (2017). Could a short training intervention modify opinions about mental illness? A case study on French health professionals. BMC Psychiatry , 17 , 133. http://doi.org.aurarialibrary.idm.oclc.org/10.1186/s12888 017 1296 0

PAGE 35

MENTAL HEALTH DE STIGMATIZATION EVALUATION 35 Appendix A: Organizational Model of the Mental Health Workgroup

PAGE 36

MENTAL HEALTH DE STIGMATIZATION EVALUATION 36 Appendix B: Variable Descriptions Variable Description Target Measure Internal mental wellness site The employee intranet contains a mental wellness page. Visitors and clicks were analyzed for 2017 . Employees I D employee interaction Employee opinion & Annual wellness survey Both survey asked various mental health questions. Responses are compared to the Surveys given with entry into prize incentives upon completion. Employees Comparison Vitality points earned for MH activities, includes mental welln ess review The hospital offers various opportunities for employees to garner vitality points where various discounts are offered upon completion of a certain number of points. Use of mental health activities and results will be compared to previous year. Employees Comparative a nalysis, identifies percent change from 2016 to 2017 Educational Trainings Identify number of staff who completed educational trainings through various outlets such as Youth mental health first aid and unconscious bias . Employees Statistical analysis, identifies percent change from 2016 to 2017 Mental Health (MH) hospital communications Internal and external communications. Internal consists of weekly emails that are sent to employees, analysis will include identifying the amount of references for mental health resources/articles/activities. External are communications sent out to the patients, families, and other members of the community. Clicks and site visits will be calculated and compared to previous years. Employees, patient s, & families Comparison from 2016 to 2017 Screening for mental health The number of mental health screenings be calculated and compared to previous years. Number of individuals screened, and number of clini cs doing the screenings will be identified. Patients Quantify the amount of MH screenings Employee Perceptions The mental health family advisory council meets monthly to discuss, promote, and improve family centered care. The council provides direct input of hospital initiatives. Individual interviews s erve as an outlet to analyze the de stigmatization efforts from the perceptions of patient families and members of the council. A majority of the members have been there before and after the push towards de stigmatization began. Employees Email interviews and content analysis ( coding framework)

PAGE 37

MENTAL HEALTH DE STIGMATIZATION EVALUATION 37 Appendix C : Interview Questions A series of 6 questions were asked in via email, where 6 out of the 16 employees contacted responded. Conventional coding framework was applied to the interview responses to quantify similarities and themes. Positive and negative themes were identified, with a majority of responses yielding positive and mixed results. 1. What has your experience at 2. What mental wellness efforts Hospital of Colorado? ( programs, opportunities, resources, changes, updates ) 3. Colorado? 4. How would you describe environment to discuss mental health related issues? 5. How wo uld you compare the current mental health culture to years past? 6. How would you describe the acceptance of mental illnesses by team members? (clarification: sensitive , judgmental) 7. What does success in promoting mental wellness look like to you?

PAGE 38

MENTAL HEALTH DE STIGMATIZATION EVALUATION 38 Appendix D: Hypothesis table Hypothesis Result H1: Employee perceptions and knowledge of hospital mental health efforts will increase from 2016 to 2017. Partially supported H2: Utilization of mental health resources by employees through the Vitality program will increase . Partially supported H3: Employee p articipation in educational trainings will increase . Supported H4: Mental health hospital communications will increase internally and externally. Supported H5: Mental health screenings b y hospital emergency departments and urgent care centers will increase. Supported H6: There will be decreases in high and at risk categories for mental wellbeing measures for resilience, psychological factors, and stressors , and increases in low risk cat egories. Partially supported

PAGE 39

MENTAL HEALTH DE STIGMATIZATION EVALUATION 39 Appendix E : Data Collection Results Variable Result s Internal Mental Wellness Site hits A total of 545 hits and 229 unique users in 2017 Employee Survey 16.60% Decrease in participants knowledge of how to access EAP Of the participants that utilized EAP in 2017, only 13.64% utilized mental health resources 7.87 % Decrease in stigma as a barrier for accessing Mental Wellness Resources culture that supports mental and physical health 2017: 35.11% say they would describe their health and wellness goals as focusing on mental health 2017: Among seminar topics offered, stress management (61.3%) and mental health (39.96%) were in the top four E mployee opinion survey: o 3% increase in respondents reporting that the amount of stress they feel is reasonable o 3.93 (out of 5) respondents reporting that t support mental wellbeing in 2017 o 3% Increase in EOS respondents reporting the level of stress they feel is reasonable Vitality points earned for MH activities, includes mental wellness review Decrease in high & at risk individuals indicated in psychological wellbeing ( .1%, 2.1%), stressors ( 1.2%, 1.4%), and resilience ( .1%, 1.4%) Increase in social support at risk populations (1.15%) Decrease in social support for low risk population ( 1.5%) No % change in high risk population social suppo rt Decrease in high ( .28%) and at risk ( .88%) populations for stress measures Increase in low risk population for stress measures (1.15%) Increase in utilization of vitality health review (.9%) Decrease in utilization of mental wellbeing review ( .8%) Decrease in individuals who reported being stressed in 2016 to 2017 (10.5%) Educational Trainings Increase in employees who participated in Youth Mental Health First Aid (78%) Increase in employees who participated in Unconscious Bias (73%)

PAGE 40

MENTAL HEALTH DE STIGMATIZATION EVALUATION 40 Increase/Decr ease in employees who completed LDI Increase in employees who completed the mindfulness course from Fall 2017 to Spring 2017 o 15.15% increase in registered o 60% increase in avg. per session Mental Health (MH) hospital communications Internal Increase in mental health references (34.6%) External Increase in mental health media stories (82%) Screening for mental health Screening rate of 78.2% in Emergency Departments & Urgent Care, 80% screening rate in other hospital areas. Increase from 1 clinic scr eening in 2016 to 15 clinics screening in 2017 Employee Interviews Mixed themes , overall, they revealed positive employee discuss mental health.

PAGE 41

MENTAL HEALTH DE STIGMATIZATION EVALUATION 41 Appendix F : Mental Health Training Participation 0 20 40 60 80 100 120 2016 2017 Participation Growth in MH Trainings Youth Mental Health First Aid Unconscious Bias

PAGE 42

MENTAL HEALTH DE STIGMATIZATION EVALUATION 42 Appendix G : Employee Survey Responses Do you know how to access the Employee Assistance Program (EAP) o 2016: n=100 83% yes 17% no o 2017: n=564 66.4% yes 33.16% no o 16.6% change in yes responses If yes did you utilize EAP? o 2016: n=538 13.01% yes 77.7% no o If yes, did you utilize mental health resources last year? o N=484 13.64% yes 86.36% no Which of the following describes your health and wellness goals? o N=581 o 35.11% focus on my mental health Among the seminar topics offered, select those that are of greatest interest to you o N=553 o 61.3% said stress management o 39.96% said mental health alth? o N=100 o 87% yes o 13% no

PAGE 43

MENTAL HEALTH DE STIGMATIZATION EVALUATION 43 Appendix H : Content Analysis Chart Question Answer (category) Cluster (overall) 1. Experience (n=6) Inadequate:11 Positive:1111 Positive 2. Awareness (n=6) Some awareness: 111111 N eu tral 3. MH Culture (n=6) Poor:11 Priority:1 Supportive: 11 Advanced:1 Positive 4. Environ. To discuss MH (n=6) Poor: 1111 Open: 11 Negative 5. Culture compared to previous years (n=5) Improved: 1 111 Inadequate: 1 Positive 6. Acceptance by team members (n=6) Sensitive: 11 Uninformed:1 Supportive: 111 Judgemental:1 N eu tral

PAGE 44

MENTAL HEALTH DE STIGMATIZATION EVALUATION 44 Appendix I : Content Analysis Coding Process Content Analysis Steps: Codes Categories Definitions and descriptions for each of the codes Cluster: Good/Bad Respondent 1: 1. W they have to work to promote mental wellness and the importance of integration with physical health. Mental health is not elevated to where it needs to be as a priority in the Codes: A lot of effort, hard work Category: Inadequate experience Defi ni tion: and needs to be improved. Negative 2. (programs, opportunities, resources, changes, updates) The stigma reduction campaign. Codes: Somewhat aware of efforts Category: Some Awareness Defi ni tion: The individual is aware of some of the resources available at Colorado is failing to convey the available r esources to employees. Nuetral 3. Colorado? Our team is very high performing and with that comes an expectation and a drive to take on more and more work. Most of our te am is stressed or stretched thin at least 90% of the time. Mental health PTO days are not commonplace on our team particularly because day off for that reason. The lea dership team in our department all work well over 40 hours and are answering emails late at night because of the demands of the job. However, my mangers have always been good at checking in on my work life balance but it is not modeled well by leadership which then makes it trickle down to everyone else. Codes: Stressed, high expectation, need MH days off, need enhanced work/life balance, need improvement in leadership, overworked Category: Poor MH culture Defi ni tion: The current mental health culture an d atmosphere is inadequate to the needs of its employees and needs to be improved. Negative

PAGE 45

MENTAL HEALTH DE STIGMATIZATION EVALUATION 45 4. discuss mental health related issues? Since our team works on mental health, it is a common topic of d iscussion. However, priority for employees to discuss it and have more awareness of mental health issues. Codes: a lot of work to be done, not adequate, needs improve ment Category: Poor MH culture Defi ni tion: The current environment to discuss mental health needs to be improved. Negative 5. How would you compare the current mental health culture to years past? I would say awareness is getting better from past years and stigma is being reduced. Codes: More awareness, reduced stigma Category: Improved Defi ni tion: The mental health culture has/is improving from past years. Positive 6. How would you describe the acceptance of mental illnesses by team members? (clarification: sensitive, judgmental) Sensitive. Codes: Sensitive Category: Sensitive Defi ni tion: Respectful of others feelings/situation Positive 7. What does success in promoting mental wellness look like to you? Recognizing the importance of not just physical health but also mental health and making that a topic of discussion in hospital and department policies to support employees and their families with mental wellness and supporting better work life balance. Codes: Increase discussion , work life balance, develop or ganizational policies Category: Organizational policies and increased discussion Defi ni tion: Develop organizational policies to meet the needs of employees. As well as increase opportunity to discuss mental wellness. Respondent 2 : 1. What has your Overall it has been positive. I have had great managers who have supported my growth and who want to make work a positive environment. Codes: Great managers, support growth, positive environment Category: Positive/supportive environment Defi ni tion: environment. Positive 2. (programs, opportunities, resources, changes, updates) EAP, Break the Stigma campaign

PAGE 46

MENTAL HEALTH DE STIGMATIZATION EVALUATION 46 Codes: Awareness of two programs Category: Some Awareness Defi ni tion: The i ndividual is aware of some of the resources available at Colorado is failing to convey the available resourc es to employees. Nuetral 3. Colorado? be because I work in PMHI whe re we are all about mental health. Codes: Increase discussion of MH importance in department Category: Priority Defi ni tion: Mental health has become a higher priority. Positive 4. ental health related issues? profession discuss mental health related issues much unless they are discussing patient care. The psychologists I work with are very open to discussing mental health related issues in their own lives and in mine when appropriate. Codes: Open in PMHI, not as common throughout the hospital in general Category: Poor (overall, supportive in PMHI) Defi ni tion: The overall hospital does not reflect an environment that is discussing mental health. PMHI does a good job being open in discussion mental health and wellness. Negative 5. How would you compare the current mental health culture to years past? Hard to say since I changed positions, but it feels more open in PMHI than it did in Ambulatory. Codes: Enhanced, more open Category: Improved Defi ni tion: The mental health culture has/is improving from past years. Positive 6. How would you describe the acceptance of mental illnesses by team members? (clarification: sensitive, judgmental) I think people are sensitive to it, but unsure how to handle mental illnesses. Codes: Sensitive, confused, not qualified/informed Ca tegory: Sensitive & uninformed Defi ni tion: Respectful of others feelings/situation. Lacking knowledge of mental health. Nuetral (both) 7. What does success in promoting mental wellness look like to you?

PAGE 47

MENTAL HEALTH DE STIGMATIZATION EVALUATION 47 More normalization of mental wellness and mental hea lth first aid (preventative daily mental wellness activities). Also, ability/acceptance for taking mental health days similar to taking sick days . Codes: Normalization, increased discussion, acceptance, prevention Category: Increased Discussion Defi ni tion: Respondent 3 : 1. I have loved working at Children's. I have the ability to work closely with patients, my coworkers are excellent, and our floor administrators are great communicators and consistently look for ways to improve patient care. Codes: Positive work experience, good communication and relationships Category: Positive environment Defi ni tion: Positive 2. Wh (programs, opportunities, resources, changes, updates) I have attended resiliency rounds and I know that there is counseling available to staff. Codes: Some awareness Cat egory: Some Awareness Defi ni tion: Colorado, but does not reveal knowledge of a majority of any of the resources and failing to convey the available resources to employees. Nuetral 3. Colorado? I work in the CCBD and I believe the mental health culture on our floor is especially t olerant and transparent. The CCBD team is especially close with the patients and their families during their extended stays, so we engage in respite time, REST sessions, room blessings, etc. to allow us space to grieve deaths and process the emotional diff iculty of dealing with cancer on a day to day basis. Maintaining mental wellness and allowing space to process sorrow seems to be a priority on our floor. Codes: tolerant and transparent, good environment, resources/opportunities to address mental wellness , relational Category: Supportive Defi ni tion: The culture of the hospital is supportive in addressing mental health. Positive 4. related issues? Thinking back on the ye ar, I have received numerous emails from Children's regarding mental health initiatives for parents and patients. It seems that mental health is as much a priority at Children's as other physical ailments, and there is an atmosphere of openness surrounding these

PAGE 48

MENTAL HEALTH DE STIGMATIZATION EVALUATION 48 issues. Likewise, while sitting with patients with mental health issues, I have noticed that the staff is decidedly nonjudgmental. Codes: equal focus/attention, open, nonjudgmental Category: Open Defi ni tion: The culture of the hospital is open and accepting to discussing mental health. Positive 5. How would you compare the current mental health culture to years past? I have just completed my first year at Children's, so I have nothing to compare. N/A 6. How would you describe the acceptance of mental illness by team members? (clarification: sensitive, judgmental) Regarding patients with mental illness, team members are nonjudgmental and strive to be positive encouragers. Similarly, in the CCBD the sta ff is understanding that we each have to go through the grieving process for different patients and need time to heal. However, I have seen frustration with team members who struggle with anxiety or other mental illnesses that affect their ability to work. So there is sensitivity and acceptance until the mental illness diminishes job performance or becomes a negative influence. Codes: nonjudgmental, positive, encouraging, understanding, sensitivity, acceptance, frustration Category: Supportive Defi ni tion: H ospital team members are supportive of mental illness among patients and employees. Positive 7. What does success in promoting mental wellness look like to you? It seems to me that Children's is already relatively successful at promoting mental welln ess. I receive hospital wide newsletters with information on mental health initiatives, the CCBD unit managers consistently check in to see how we are coping, there are numerous traditions in the CCBD that allow us to process grief and celebrate life, and overall it seems there is an understanding that mental illness is common and should not be hidden. Category: Awareness, Leadership Defi ni tion: Awareness of mental health and involve ment of leadership. Respondent 4 : 1. Stressful. Somewhat rewarding but disappointing overall. Codes: Stressful, disappointing Category: Inadequate experience Defi ni tion: be improved. Negative 2. (programs, opportunities, resources, changes, updates) Yoga/meditation classes, mindfulness classes. Codes: Some awareness

PAGE 49

MENTAL HEALTH DE STIGMATIZATION EVALUATION 49 Category: Some Awareness Defi ni tion: Colorado, but does not reveal knowledge of a majority of any of the resources and available resources to employees. Nuetral 3. Colorado? not sure that one exists. Codes: Lacking , no culture/support Category: Poor MH culture Defi ni tion: The current mental health culture and atmosphere is inadequate to the needs of its employees and needs to be improved. Negative 4. How would you desc related issues? Supportive but in a superficial way. Codes: superficial Category: Poor MH culture Defi ni tion: The current environment to discuss mental health needs to be improved. Negative 5. How would you compare the current mental health culture to years past? No change. Codes: No change in culture Category: Inadequate change in culture. Defi ni tion: The mental health culture has not enhanced from previous years. Negative 6. How would you describe the acceptance of mental illnesses by team members? (clarification: sensitive, judgmental) Judgmental, fearful, lacking knowledge. Codes: Judgmental, fearful, lacking knowledge, bad Category: Judgmental Defi ni tion: Critical and not accepting of mental illness. Negative 7. What does success in promoting mental wellness look like to you? Promotion at the same intensity as other health issues. Codes: Equal emphasis on MH Category: Awareness Defi ni tion: Awareness through equal promotion and focus of mental health to that of other illnesses throughout the hospital. Respondent 5 : 1.

PAGE 50

MENTAL HEALTH DE STIGMATIZATION EVALUATION 50 I worked in radiology for the first two years and have been in Apheresis for the remaining years. I do miss the smaller feel of the hospital but understand that with success growth will follow. Codes: enjoyed, positive experience Category: Positive Defi ni tion: has been a positive work environment. Positive 2. (programs, opportunities, resources, changes, updates): I know that there is an employee assistance program as well as some classes through HR that help with stress. Codes: Some awareness Category: Some Awareness Defi ni tion: Colorado, but does not reveal knowledge of a majority of any of the resources and available resources to employees. Nuetral 3. How would you describe the current mental health Colorado? Codes: ahead, succeeding Category: Advanced Defi ni tion: hospital. Positive 4. related issues? proactive manner. I think managers could do a better job of doing mental health check in with staff: Something simple like: rate your stress level this week or your level of enjoyment with current job. Codes: needs improvement, lacking leadership Category: Poor MH culture Defi ni tion: The current environment to discuss mental health needs to be improved. Negative 5. How would you compare the current mental health culture to years past? I think there is more awareness of these problems Codes: Increased awareness Category: Improved Defi ni tion: The mental health culture has/is improving from past years. Positive 6. How would you describe the acceptance of mental illnesses by team members? (clarification: sensitive, judgmental): I would say that overall people are very supportive of other staff that have any type of illness.

PAGE 51

MENTAL HEALTH DE STIGMATIZATION EVALUATION 51 Codes: Supportive Category: Supportive Defi ni tion: Hospital team members are supportive of mental illness among patients and employees. Positive 7. Wha t does success in promoting mental wellness look like to you? Respondent 6 : 1. Everyone is willing to help each other, and we have a great work environment Codes: S upportive, positive work environment Category: Positive Defi ni tion: Positive 2. (programs, opportunities, resources, changes, updates) Vitality, The Gary and Spark Talks Codes: Some awareness Category: Some Awareness Defi ni tion: Colorado but does not reveal knowledge of a majority of any of the resources and available re sources to employees. Nuetral 3. Colorado? . Codes: Open, aware Category: Supportive Defi ni tion: The culture of the hospital is supportive in addressing mental health. Positive 4. related issues? We have a very accepting environment of metal heath issues. Codes: Accepting, positive environment Category: Open Defi ni tion: The culture of the hospital is open and accepting to discussing mental health. Positive 5. How would you compare the current mental health culture to years past? ght shed on it in the last year or so . Codes: Increased awareness, discussion/focus Category: Improved culture Defi ni tion: The mental health culture has/is improving from past years. Positive

PAGE 52

MENTAL HEALTH DE STIGMATIZATION EVALUATION 52 6. How would you describe the acceptance of mental illnesse s by team members? (clarification: sensitive, judgmental) I think everyone is very open and supportive of mental health. Codes: Open and supportive Category: Supportive Defi ni tion: Hospital team members are supportive of mental illness among patients and employees Positive 7. What does success in promoting mental wellness look like to you? I think if people felt comfortable talking about their mental health issues without feeling like they were getting judged, that would be a success. Codes: nonjudgmental, comfort, openness Category: Nonjudgmental Defi ni tion: Being an organization that has an environment that is accepting and free of judgment.

PAGE 53

MENTAL HEALTH DE STIGMATIZATION EVALUATION 53 Appendix J : MPA Core Competencies Supplement The core competencies that this capstone project most relied on are described below, as well an explanation of the knowledge and skills gained from each of the courses in relation to the project. To lead and m anage in public governance Throughout the MPA program at the University of Colorado, efficiency is consistently stressed as a priority for both government and non profit organizations. To ensure an efficient needed to identify and evaluate its status regarding mental health and direct de stigmatization efforts towards continued effectiveness. Working with a nonprofit entity to promote mental health without stressing the current infrastructure of the organizat ion was deemed an efficient and effective approach. Skills and understanding behind the importance of implementing initiatives that highlight cooperation of multiple departments to ensure success were also garnered. The collaboration of the multidisciplina ry workgroup was crucial part in developing a successful de stigmatization approach to mental health. Recommendations for further collaboration were based on the workgroups success as well as their ability to reach multiple facets of the hospital. The find ings that emerged from the scholarly literature present the client with evidence that paralleled the findings from the evaluation. Application of the four approaches represented by the literature are the key avenues to provoking organizational wide behavio r change and incremental changes, specifically with mental health stigma. Further, when proposing an organizational policy or initiative it is crucial that the proposed approaches success is alluded to in its ease of implementation, organizational acceptan ce, and technical feasibility. PUAD 5002 Organizational Management, PUAD 5008 ,

PAGE 54

MENTAL HEALTH DE STIGMATIZATION EVALUATION 54 Evidence Based Decision Making, and PUAD 5004 , Economics served as the primary outlets for application of knowledge and skills garnered from the courses to conduct the research and recommendations for the project. To analyze, synthesize, think critically, solve problems and make decisions This competency was referenced throughout multiple aspects of the capstone project. The research methods selected, as well as analysis of th e findings served as a crucial component to the client. The information was distributed among the mental health de stigmatization work group, as well as at the Association of Community Health Improvement (ACHI) conference to express the organizations succe ss. As alluded to in the previous section, the recommendations stemmed not only from the scholarly literature, but also as a result of analysis of the data. Measures that indicated success were recommended for continuation, with slight improvements to addr ess levels of participation. Throughout the analysis of the variables for this project numerous limitations arose that required problem solving in assessing which data to include to tell the story of the hospitals efforts. This competency relied heavily on knowledge and skills garnered from PUAD 5008 Evidence Based Decision Making, and PUAD 5003 Research and analytical methods. Communicate and interact productively with a diverse and changing workforce and citizenry This competency was a crucial compone nt throughout the project process due to the time sensitivity that arose soon after the project was initiated. This aspect of the project not only effected the researcher but other departments that served as major contributors to distributing the secondary data. Due to the diversity of the multi disciplinary workgroup , interactions and communications with each directly reflected the diverse backgrounds and viewpoints that are outlined in the competency. This competency is further illustrated in the program evaluation

PAGE 55

MENTAL HEALTH DE STIGMATIZATION EVALUATION 55 paper being a communication tool to the diverse audiences. Knowledge and skills from all MPA courses were applied, PUAD 5001 Introduction to Public Administration, and PUAD 5005 Policy process played an essential role in developing the project t o be understood by various audiences.

PAGE 56

Form Name: capstone repository permission Submission Time: May 14, 2018 9:04 am Browser: IE 11.0 / Windows 7 IP Address: 66.128.217.192 Unique ID: 407759652 Location: 39.729000091553, -104.95279693604 Description Area SCHOOL OF PUBLIC AFFAIRS ELECTRONIC CAPSTONE REPOSITORY Description Area Dear Capstone Author and Capstone Client:The Auraria Library Digital Library Program is a nonprofit center responsible for the collection and preservation of digital resources for education.The capstone project, protected by your copyright, and/or created under the supervision of the client has been identified as important to the educational mission of the University of Colorado Denver and Auraria Library.The University of Colorado Denver and Auraria Library respectfully requests non-exclusive rights to digitize the capstone project for Internet distribution in image and text formats for an unlimited term. Digitized versions will be made available via the Internet, for onand off-line educational use, with a statement identifying your rights as copyright holder and the terms of the grant of permissions.Please review, sign and return the follow Grant of Permissions. Please do not hesitate to call me or email your questions.Sincerely,Matthew C. MarinerAuraria LibraryDigital Collections ManagerMatthew.mariner@ucdenver.edu303.556.5817 Grant of Permissions Description Area In reference to the following title(s): Author (Student Name) Sarah Aldrich Title (Capstone Project Title) Colorado Children's Hospital Mental Health De-Stigmatization Evaluation Publication Date 5-2-18 I am the: Client Description Area As client of the copyright holder affirm that the content submitted is identical to that which was originally supervised and that the content is suitable for publication in the Auraria Library Digital Collections.

PAGE 57

Description Area This is a non-exclusive grant of permissions for on-line and off-line use for an indefinite term. Off-line uses shall be consistent either for educational uses, with the terms of U.S. copyright legislation's "fair use" provisions or, by the University of Colorado Denver and/or Auraria Library, with the maintenance and preservation of an archival copy. Digitization allows the University of Colorado Denver and/or Auraria Library to generate imageand text-based versions as appropriate and to provide and enhance access using search software. This grant of permissions prohibits use of the digitized versions for commercial use or profit. Signature Your Name Katie O'Connor Date 5-14-18 Email Address ATTENTION Description Area Grant of Permissions is provided to: Auraria Digital Library Program / Matthew C. MarinerAuraria Library1100 Lawrence | Denver, CO 80204matthew.mariner@ucdenver.edu303-556-5817

PAGE 58

Form Name: capstone repository permission Submission Time: May 14, 2018 4:32 pm Browser: Chrome 58.0.3029.110 / Windows IP Address: 71.196.235.242 Unique ID: 407910214 Location: 39.731300354004, -104.96920013428 Description Area SCHOOL OF PUBLIC AFFAIRS ELECTRONIC CAPSTONE REPOSITORY Description Area Dear Capstone Author and Capstone Client:The Auraria Library Digital Library Program is a nonprofit center responsible for the collection and preservation of digital resources for education.The capstone project, protected by your copyright, and/or created under the supervision of the client has been identified as important to the educational mission of the University of Colorado Denver and Auraria Library.The University of Colorado Denver and Auraria Library respectfully requests non-exclusive rights to digitize the capstone project for Internet distribution in image and text formats for an unlimited term. Digitized versions will be made available via the Internet, for onand off-line educational use, with a statement identifying your rights as copyright holder and the terms of the grant of permissions.Please review, sign and return the follow Grant of Permissions. Please do not hesitate to call me or email your questions.Sincerely,Matthew C. MarinerAuraria LibraryDigital Collections ManagerMatthew.mariner@ucdenver.edu303.556.5817 Grant of Permissions Description Area In reference to the following title(s): Author (Student Name) Sarah Aldrich Title (Capstone Project Title) Children's Hospital of Colorado Mental Health De-Stigmatization Evaluation Publication Date May 2, 2018 I am the: Author (student) Description Area As copyright holder or licensee with the authority to grant copyright permissions for the aforementioned title(s), I hereby authorize Auraria Library and University of Colorado Denver to digitize, distribute, and archive the title(s) for nonprofit, educational purposes via the Internet or successive technologies.

PAGE 59

Description Area This is a non-exclusive grant of permissions for on-line and off-line use for an indefinite term. Off-line uses shall be consistent either for educational uses, with the terms of U.S. copyright legislation's "fair use" provisions or, by the University of Colorado Denver and/or Auraria Library, with the maintenance and preservation of an archival copy. Digitization allows the University of Colorado Denver and/or Auraria Library to generate imageand text-based versions as appropriate and to provide and enhance access using search software. This grant of permissions prohibits use of the digitized versions for commercial use or profit. Signature Your Name Sarah Aldrich Date May 14, 2018 Email Address ATTENTION Description Area Grant of Permissions is provided to: Auraria Digital Library Program / Matthew C. MarinerAuraria Library1100 Lawrence | Denver, CO 80204matthew.mariner@ucdenver.edu303-556-5817