Citation
School-based counseling services with undeserved populations : a multisite evaluations of outcomes

Material Information

Title:
School-based counseling services with undeserved populations : a multisite evaluations of outcomes
Creator:
Metzler, Gabriela Carol
Place of Publication:
Denver, CO
Publisher:
University of Colorado Denver
Publication Date:
Language:
English

Thesis/Dissertation Information

Degree:
Doctorate ( Doctor of psychology)
Degree Grantor:
University of Colorado Denver
Degree Divisions:
School of Education and Human Development
Degree Disciplines:
School psychology
Committee Chair:
Crepeau-Hobson, Franci
Committee Members:
Harris, Bryn
Stein, Rachel

Notes

Abstract:
This study sought to evaluate the school-based counseling services that were provided to 95 students as part of the Children First counseling program. The primary evaluation question focused on the effectiveness of the program in positively impacting traditionally underserved populations in the following domains: increasing social skills, decreasing problem behaviors, and increasing academic competence. A review of relevant literature revealed the impact of school based services on decreasing barriers to mental health services and improving the coordination of care among agencies, specifically with underserved populations. The results of the evaluation study indicated that the Children First program is effectively preventing mental health issues and intervening with children experiencing such challenges, including those who are traditionally underserved. Implications for future research are discussed.

Record Information

Source Institution:
University of Colorado Denver
Holding Location:
Auraria Library
Rights Management:
Copyright Gabriela Carol Metzler. Permission granted to University of Colorado Denver to digitize and display this item for non-profit research and educational purposes. Any reuse of this item in excess of fair use or other copyright exemptions requires permission of the copyright holder.

Downloads

This item has the following downloads:


Full Text
SCHOOL-BASED COUNSELING SERVICES WITH UNDERSERVED POPULATIONS:
A MULTISITE EVALUATION OF OUTCOMES
by
GABRIELA CAROL METZLER B.S., North Greenville University, 2015
A thesis submitted to the Faculty of the Graduate School of the University of Colorado in partial fulfillment of the requirements for the degree of Doctor of Psychology School Psychology Program
2019


©2019
GABRIELA CAROL METZLER
ALL RIGHTS RESERVED


This thesis for the Doctor of Psychology degree by Gabriela Carol Metzler has been approved for the School Psychology program By
Franci Crepeau-Hobson, Chair Bryn Harris Rachel Stein
Date: May 18, 2019


Metzler, Gabriela Carol (PsyD., School Psychology Program)
School-Based Counseling Services with Underserved Populations: A Multisite Evaluation of Outcomes
Thesis directed by Associate Professor Franci Crepeau-Hobson
ABSTRACT
This study sought to evaluate the school-based counseling services that were provided to 95 students as part of the Children First counseling program. The primary evaluation question focused on the effectiveness of the program in positively impacting traditionally underserved populations in the following domains: increasing social skills, decreasing problem behaviors, and increasing academic competence. A review of relevant literature revealed the impact of school-based services on decreasing barriers to mental health services and improving the coordination of care among agencies, specifically with underserved populations. The results of the evaluation study indicated that the Children First program is effectively preventing mental health issues and intervening with children experiencing such challenges, including those who are traditionally underserved. Implications for future research are discussed.
The form and content of this abstract are approved. I recommend its publication.
Approved: Franci Creapeau-Hobson
IV


DEDICATION
To my husband and best friend, Joey:
Thank you for being someone that I can look up to and learn from. You have demonstrated what it means to be a student of people, of curiosities, and of life. You push me to be better, to dream bigger, to love deeper, and to pursue my passions with dedication and zeal. I am grateful to experience all of life’s adventures with you by my side, but especially the past four years of graduate school. Thank you for providing unending support and encouragement.
v


TABLE OF CONTENTS
CHAPTER
I. INTRODUCTION............................................1
II. LITERATURE REVIEW.......................................6
III. METHOD.................................................13
IV. RESULTS................................................17
V. DISCUSSION.............................................20
REFERENCES..................................................25
vi


CHAPTERI
INTRODUCTION
In the United States, one out of every five children will experience a clinically significant mental health condition during their childhood (Powers, Swick, Wegmann, & Watkins, 2016). It is estimated that two-thirds of these children never receive treatment (Garrison, 1999) and the small number of children who do receive treatment are most likely to receive those services in their school environment (Rones & Hoagwood, 2000). This is because nearly 85% of children who receive mental health services report experiencing symptoms of their mental health challenges within the classroom (Bradshaw, Buckley, & Ialongo, 2008). Externalizing symptoms (i.e., aggression) and internalizing symptoms (i.e., anxiety) have both been shown to increase academic difficulties for children (Bradshaw et al., 2008). As such, school-based intervention of mental health difficulties has implications for both academic and mental health outcomes for children (Bradshaw et al., 2008).
School-based mental health services integrate traditional mental health services such as counseling and psychological assessment within academic settings. Although the definition of school-based services has long been debated, this paper will utilize a definition taken from Rones and Hoagwood (2000), which defines school-based mental health services as, “any program, intervention, or strategy applied in a school setting that was specifically designed to influence students’ emotional, behavioral, and/or social functioning” (p. 224). School-based mental health services can include those provided by school-employed staff or can be provided by agencies that partner with districts to provide mental health services in the school building (Dryfoos,
1994).
1


Historically, the earliest provision of social services within the school setting occurred in the early 1900’s when school clinics provided medical services to low-income students and their families (Dryfoos, 1994). However, these services proved difficult to integrate into the school setting due to concern that the services of the school should not compete with private sector agencies (Doll, Nastasi, Cornell, & Song, 2017). Throughout the rest of the 20th century, the integration of school-based mental health services sporadically rose and fell in popularity. Many times, the difficulty of integration was due to the siloed effect of mental health services, in which school and community mental health exist as individual silos without integration of care (Doll et al., 2017). At the present time, many school districts around the United States are beginning to contract with community mental health agencies to provide more comprehensive mental health services in the school building (Bradshaw et al., 2008). These contracted services are typically for a specific service such as individual counseling or health education classes. These types of school-community partnerships have the potential to greatly reduce the siloed effect of mental health services. However, the implementation of these types of partnerships has historically been difficult to maintain (Doll et al., 2017).
Regardless of these difficulties, schools remain one of the most critical early identification and treatment systems for children with mental health concerns (Powers et al., 2015). School-based services are in the unique position to decrease barriers to mental health services, to improve coordination of care among agencies, and to implement well-rounded prevention and intervention strategies for children (Powers et al., 2015). By observing a child’s functioning in an everyday setting, school-based service providers can understand how a child’s skills and needs unfold in their natural form.
2


Program Evaluation: Children First
The mental health services evaluated in this study are part of a school-based therapeutic counseling program called Children First. The Children First program is coordinated by a nonprofit mental health agency in Denver, CO. The overall mission of this organization is based on the belief that counseling services should be accessible to all people regardless of their ability to pay. The organization strives to reduce barriers to mental health care and to meet the significant unmet mental health needs of the Denver population. This program provides free therapeutic counseling for children in their school buildings by contracting with local districts.
Description of the Program
The Children First program was based on the premise that early prevention and intervention help children gain adaptive skills before mental health issues can manifest. This program was established in 1990 to support children and adolescents who are experiencing emotional, social, or behavioral problems with free-of-charge, on-site counseling services in their schools (Maria Droste, 2018). In this way, the Children First program offers a creative solution for schools that need mental health support in their buildings but lack the budget to hire a mental health counselor. This program also provides experience and internship credit to master’s and doctoral level interns from a variety of graduate programs focused on mental health. Currently, the Children First program is located in over 20 schools in Denver, many of which are located in diverse and economically disadvantaged areas (Maria Droste, 2018).
To provide services to Children First clients, graduate-level interns are placed in a local school to provide therapeutic counseling services to students. The interns receive supervision from a licensed mental health professional on staff with Maria Droste and receive on-going training during the academic year. At each Children First school, the intern is also supported by a
3


school-based mental health professional such as a school psychologist or school social worker. Each intern is expected to be at their school site for a minimum of 10 hours per week, with some interns completing up to 16 hours a week in their school depending on their track of study. It is expected that for each hour the intern is in the school, they have one student client (i.e., an intern who works 10 hours a week should have 10 students on their case load). Counseling sessions are generally advised to be 30 minutes long, with some sessions lasting up to 50 minutes depending on the student’s needs. While Maria Droste provides training on various therapeutic orientations and techniques, there are no set requirements for the activities that must be completed in counseling. Instead, Maria Droste allows each intern to use their professional training in their area of specialization. Some Children First counselors use traditional cognitive behavioral therapy, while others utilize play therapy techniques, trauma-informed techniques, and many other therapeutic approaches. The program also encourages interns to have both individual and group-based sessions.
Purpose of the Study
The purpose of this study was to evaluate the extent to which the Children First program is meeting its goal of providing effective early prevention and intervention services. Specifically, this program evaluation measured whether the Children First program is effective in developing social and emotional skills, reducing problem behaviors, and increasing academic skills in clients served. While also looking to the mission of Maria Droste, this program evaluation sought to understand how the Children First program is achieving these outcomes with the following traditionally underserved populations: racial/ethnic minorities, English-language learners, and first-generation immigrants.
4


Evaluation Question
Is the Children First counseling program effective in developing social and emotional skills, reducing problem behaviors, and increasing academic skills, particularly with regard to the following traditionally underserved populations: minorities, English-language learners, and first-generation immigrants?
5


CHAPTER II
LITERATURE REVIEW The School as a Setting for Mental Health Services
Behavioral, emotional, and social difficulties can affect children in any context.
However, these difficulties can be particularly troublesome in the school environment because children are in their school building for roughly six hours a day, 180 days of the year (Bradshaw et al., 2008). The school setting is one in which children are learning more than division and geography. Children are learning how to navigate social relationships, how to regulate their emotions, how to communicate their needs, and how to develop many more social-emotional skills. If a child has difficulty with social-emotional skills, they are also at an increased risk of academic difficulties because these skills are so deeply interconnected (Powers et al., 2015). The impact of these skills in the academic environment makes it a perfect setting to incorporate mental health services directly in the school.
Of the many benefits to the integration of mental health care in the school setting, the most obvious is the increased access to care for children with mental health concerns. Barriers are reduced when children have access to care in a convenient location, during a time that they will already be in school, and when the services are low-cost or free (Doll et al., 2017). Additionally, students and families who are already familiar with the school building and the school staff are less likely to experience stigma in receiving mental health services (Doll et al., 2017). Most schools already employ mental health professionals who are prepared to provide a continuum of services and to coordinate care with outside providers (Bradshaw et al., 2008).
This type of integrated care impacts the child’s ecological context by including teachers, families, and larger systems of the child’s environment (Bradshaw et al., 2008).
6


By embedding mental health services within schools, the capacity for prevention and intervention services is enhanced (Owens, Murphy, Richerson, Girio, & Himawan, 2008).
Recent frameworks of treatment suggest that schools and community mental health centers must work together to provide a continuum of care (Owens et al., 2008). This partnership is needed because many school-employed providers are not able to deliver intensive supports. In a qualitative study conducted with school-employed mental health providers, many participants indicated that schools need designated, in-house mental health providers who offer different types of services like individual counseling, special education evaluations, or family therapy (Gamble & Lambros, 2014). Unfortunately, many schools are not able to employ a sufficient number of mental health providers to meet the needs of all students. Because the school setting is primarily an academic setting, many students will only receive services if their needs are impacting their academic progress. In this way, school-based mental health providers may not be able to apply concepts of evidence-based practice to everyday, individualized treatments due to the nature of the school setting (Beehler, Birman, & Campbell, 2012). However, research still demonstrates that most children who do receive mental health services will receive them from a school-based provider (Bradshaw et al., 2008).
The Provision of School-Based Services to Underserved Populations
Throughout history, minority groups have experienced greater levels of unmet mental health need due to limited access of care and an inferior quality of treatment (Gamble & Lambros, 2014). There is significant research that also demonstrates that minority youth not only have greater mental health needs compared to their majority peers, but also underutilize mental health care services (Gamble & Lambros, 2014). Minority children and adolescents are more
7


likely to be impacted by poverty and its associated risk factors, such as living in neighborhoods with fewer resources (Gamble & Lambros, 2014).
Despite the increased risk for mental health challenges, minorities are less likely to receive necessary mental health services when compared to non-minorities due to a range of barriers (Gonclaves & Moleiro, 2012). Families may not recognize that their child is in distress, may not be aware of support services, may not have transportation to get to service centers, and language barriers may make services hard to find (O’Shea et al., 2000). School-based staff may misunderstand or may not even recognize that a child is experiencing mental health difficulties (Gonzales-Ramos & Gonzalez, 2005). Gonclaves and Moleiro (2012) separate these barriers into objective and subjective barriers. Objective barriers tend to be structural or system downfalls, while a subjective barrier typically has to do with the client themselves (Gonclaves & Moleiro, 2012). For example, an objective barrier might involve a lack of available providers and a subjective barrier might involve a client’s hesitation to enter into therapy. Due to these extensive barriers, there is even greater reliance on non-medical settings, like schools to prevent, detect, and intervene with mental health concerns among culturally and linguistically diverse children (O’Shea etal., 2000).
Research has found that school-based services are more likely to be utilized by children of color. For example, in one systematic review of school-based mental health clinics, researchers noted that minority youth were 20 times more likely to engage in mental health services at their school-based clinic than at a community mental health center (Bear, Finer, Guo, & Lau, 2014). While there is evidence that school-based clinics reduce barriers to care for traditionally underserved populations, inequities may still exist in school-based mental health care (Bear et al., 2014). For example, Bear and colleagues found that non-minority students are
8


more likely than minorities to receive care in their schools regardless of evidence that suggests that minorities are at risk for greater mental health problems (Bear et al., 2014). This study demonstrates how school-based services can maintain the same inequities of treatment for traditionally underserved students (Bear et al., 2014). This is especially concerning for immigrant and refugee populations because these children are often at higher risk of mental health concerns due to psychological risk factors like stress from acculturation and prejudice from their communities (Goncalves, 2012). In particular, refugees have an increased risk of psychopathology due to the increased likelihood of experiencing traumatic events such as exposure to violence or war activities (O’Shea, Hodes, Down, & Bramley, 2000). During all stages of their migration journey, these populations experience chronic adversity: in their native country, on their journey to a new country, and once settled in a new country (Fazel, Doll, & Stein, 2009). This journey affects their families, their education, their social relationships, and other critical systems in their lives (Fazel et al., 2009).
In an article by O’Shea et al (2000), the authors illustrated how the conventional system of referring refugee and immigrant students to mental health clinics generally leads to low rates of follow through and ultimately, a lack of service provision for these populations. If these populations do receive care, it is often of lower quality due to a lack of cultural responsiveness from mental health service providers (Gonclaves & Moleiro, 2012). Given the increased risk for mental health challenges in diverse populations, as well as the increase in minority groups in the U.S., it is critical that mental health interventions be culturally sensitive. However, most evidence-based treatments have been developed for the majority population without consideration of factors like culture, language acquisition, and socioeconomic status (Castro-Olivo, 2017). Not surprisingly, many practitioners still have difficulty identifying and
9


implementing effective treatments for students from culturally or linguistically diverse backgrounds (Castro-Olivo, 2017). This affects immigrant and refugee populations most of all because these students do not lose their cultural backgrounds simply because they are located in a new country, but rather, they adapt to new cultural situations using their pre-existing worldview.
In a qualitative study completed by Gonclaves and Moleiro (2012), culturally and linguistically diverse students, their teachers, and their parents were interviewed about the ways that mental health service providers could demonstrate cultural competence. Throughout the interviews, students consistently noted that language, traditions, habits, preferences, family, ways of life, education, and the acculturation process were most important for their service provider to understand. Teachers believed that a culturally competent service provider would have knowledge about the individual history of the client, as well as the client’s family dynamics (Gonclaves & Moleiro, 2012). Service providers must be aware of the both the student’s overall culture and their individual experiences to effectively demonstrate cultural competence.
While there is a wealth of studies that have explored the disparities of mental health care among minorities, there are few studies that generalize the findings of program applications with traditionally underserved populations (Gamble & Lambros, 2014). In particular, very little research has evaluated real-world, school-based mental health services for immigrant and refugee children (Fazel et al., 2009 & Beehler et al., 2012). In one of the few studies in this area, Fazel et al. (2009) investigated the impact of school-based mental health services with a group of refugee students. These students received consultative and counseling services by a mental health team at three schools. These students were assessed by their teachers before receiving services and again at the end of the school year using the Strengths and Difficulties
10


Questionnaire (Fazel et al., 2009). The refugee students were compared to other groups who were also receiving services and the refugee group had statistically significant improvements in two domains in particular: hyperactivity and peer relations (Fazel et al., 2009). The authors speculated that “peer relations” were most improved because the services often focused on issues of integration such as adjusting to a new environment, understanding the school system, and developing a sense of self in a new culture. This study also compared students who were receiving direct services with those who were receiving indirect or consultative services. In this case, direct services were found to be more beneficial for refugee students than indirect services (Fazel et al., 2009).
To determine what type of direct services are most beneficial culturally and linguistically diverse children, a review of studies was conducted. Tyrer and Fazel (2014) found that verbalprocessing and creative-art based therapies led to the most significant reduction in mental health symptoms, most particularly those related to depression, anxiety, PTSD, and peer problems. In this case, verbal-processing therapies include cognitive behavioral therapy and trauma systems therapy, while creative-art therapies include play therapy and movement-based therapies. The authors attributed the success of these programs on the incorporation of both individual and systems-based therapeutic processing (Tyrer & Fazel, 2014).
Although these studies have explored the impacts of school-based mental health services, there is little research that has examined the following factors: the effectiveness of school-based mental health services and the provision of those services with traditionally underserved populations such as minorities, English language learners, and immigrants and refugees. Due to the lack of research and practical examination of programs, it is largely unknown what types of school-based mental health services are most effective with culturally and linguistically diverse
11


students, as well as what constitutes effective practice. There is also little research on how school-based services impact these populations in a broader context which includes their mental health, social skills, behavior, and other constructs outside the social and emotional realm like academic achievement. As such, it is difficult for service providers to implement culturally sensitive, evidence-based practices in their schools. It is clear, however, that school-based mental health professionals need to tailor their services to each individual child and take into account their cultural background, particularly due to the growing populations of culturally and linguistically diverse children in the United States. The present evaluation is an attempt to bridge this gap in the literature by examining the effectiveness of school-based mental health services provided by the Children First program in meeting the needs of children from diverse backgrounds.
12


CHAPTER III
METHOD
Procedures and Treatment
Clients were referred to the Children First program consistent with the referral procedures of their respective schools. For example, many schools had the school-based liaison (i.e., the school psychologist) collect intake information and determine if that student should be referred to the Children First intern for counseling. Once referred for counseling, the Children First intern was provided general information about the child and their family by the school district. The intern then met with the family to gather referral information and explore the parent’s concerns. During this intake process, the intern obtained written permission for the child’s participation in treatment, as well as for the completion of pre- and post-treatment teacher ratings. Once treatment was initiated, the intern provided the Social Skills Improvement System (SSIS; Gresham & Elliott, 2008) form to the student’s primary teacher to collect baseline data on the child’s functioning. The intern then re-administered the form around the six-month mark of treatment as the post-test data.
Participants
Although the Children First program served close to 300 students in the 2017-2018 academic school year, complete and reliable data could only be collected for 95 students. These 95 students were referred to and served by the Children First program during the 2017-18 school year, in their respective schools. These 95 participants were spread across 15 schools in the Denver metro area.
Sample demographics are provided in Table 1. A little more than half of the students who made up the sample were male (55%). Every grade from kindergarten through eighth grade was
13


represented in the sample. However, the majority of the sample (72%) was in grades two through five. Of the 95 participants, the majority of the sample belongs to a non-white minority population (67%), and a little over one-third of the sample (37%) learned English as a second or alternative language. While the majority of the sample was bom and raised in the United States,
11 students (11.6%) were bom outside of the U.S., and a total of 11 countries were represented in the sample (including the United States).
Table 1
Demographic Characteristics of Participants
Characteristic n Percentage of Sample
Sex:
Female 43 45.26%
Male 52 54.74%
Grade:
Kindergarten 4 0.04%
1st grade 9 0.09%
2nd grade 14 0.15%
3 rd grade 19 0.20%
4th grade 19 0.20%
5th grade 17 17.89%
6th grade 3 0.03%
7th grade 8 0.08%
8th grade 2 0.02%
Ethnicity:
Minority 64 67.37%
Non-Hispanic White 31 32.63%
Country of Origin:
United States 84 88.42%
Other country 11 11.58%
Language of Origin:
English 61 64.21%
Other language 34 35.79%
Measures
Teachers completed the Social Skills Improvement System (SSIS; Gresham & Elliot, 2008) to assess the student’s social skills, problem behaviors, and academic competence both pre- and post-treatment. The SSIS was selected because Maria Droste Counseling Center has
14


historically used this measure to determine functioning in several domains. The SSIS is also a brief measure that can easily be administered and scored. The SSIS assesses the child’s functioning in each of the following domains:
1. Social Skills:
Communication, Cooperation, Assertion, Responsibility, Empathy, and Self-Control
2. Problem Behaviors:
Externalizing, Bullying, Hyperactivity/Inattention, and Internalizing behaviors
3. Academic Competence:
Reading and Math Performance, Motivation, Parental Support, and Overall Functioning
The SSIS includes 83 items which are assessed on a Likert scale. On the Social Skills and Problem Behaviors domains, the teacher indicates the frequency of which the student exhibits a behavior with a 4-point scale that includes Never, Seldom, Often, and Almost Always. Then, on the Academic Competence scale, teachers indicate the student’s level of competence in reading, math, and learning behaviors by comparing the student in question to the rest of the class. Using a 5-point scale, the teacher rates 1) if the student is in the lowest 10% of the class, 2) the next lowest 20%, 3) the middle 40%, 4) the next highest 20%, or 5) the highest 10%. Administration of the SSIS takes roughly 5-10 minutes.
The SSIS was normed with a nationally representative sample of nearly 5,000 children aged 3-18 years old. The SSIS has demonstrated acceptable psychometric properties with alpha coefficients ranging from .83 to .97 (Gresham & Elliot, 2008). There are also three measures built into the SSIS to help the examiner judge the validity of the scores: a negativity index (F-scale), a response pattern index, and a response consistency index. These scores help determine
15


if a rater is too negative, answers questions in unusual patterns (i.e., rates a “1” on the first half of the questions), and if the rater answers related questions in similar ways.
For each of the domains, raw scores are added and converted into standard scores which have a mean of 100 and a standard deviation of 15. When interpreting the SSIS standard scores, the Social Skills and Academic Competence scales assess positive behaviors while the Problem Behaviors scale assesses negative behaviors. This means that higher scores are desired on the Social Skills and Academic Competence scales while lower scores are desired on the Problem Behaviors scale. For each of the subdomains, raw scores are converted into a category based on standard deviations to fit into the Below Average, Average, or Above Average level.
Analyses
To assess the Children First program’s overall effectiveness in meeting its goals of increasing social skills and academic competence and decreasing problem behaviors, three repeated measures ANOVAs were conducted with respective pre- and post-treatment SSIS scale scores (Social Skills, Academic Competence, and Problem Behaviors). To determine if the program was effective with the specific populations of interest (minorities, English language learners, and immigrants), three separate paired sample t-tests were conducted for each group. Data was analyzed using the Statistical Package for Social Sciences (SPSS). Descriptive statistics were analyzed to demonstrate demographic characteristics of each participants.
16


CHAPTER IV
RESULTS
Whole Group Analysis
As described above, three repeated-measures ANOVAs were conducted to assess whether there were significant differences between the average ratings of pre- and post-test measures on each of the following scales of the SSIS: Social Skills, Problem Behaviors, and Academic Competence. Assumptions were checked and met. Results indicated that students had significantly higher average scores on the Social Skills scale post-treatment, F (1, 94) = 12,81,/; < .001, partial Q = .12. This suggests that teachers rated students as having increased social skills after receiving counseling services.
There was also a statistically significant difference in Problem Behaviors scale scores pre- and post-treatment, F (1, 94) = 4.32,p = .05, partial Q = .04. Results indicate that teachers rated students as having significantly fewer problem behaviors after receiving counseling services.
No statistically significant differences were found for pre- and post-treatment scores on the Academic Competence scale. Pre- and post-treatment SSIS data are presented in Table 2.
Table 2
Means and Standard Deviations of the Pre- and Post-treatment SSIS Ratings
SSIS Scale Pre-treatment X (SD) Post-treatment X (SD)
Social Skills** 87.43 (13.45) 91.52(14.75)
Problem Behaviors* 113.54 (16.36) 110.95 (14.77)
Academic Competence 92.35 (14.15) 93.49(14.40)
* p < .05 **p < .001
17


Sub-Group Analysis
Following the analyses of overall pre- and post-treatment SSIS ratings, paired samples t-tests were conducted to examine differences for specific subgroups of interest. Analyses of the Social Skills, Problem Behavior, and Academic competence scales were conducted for the following groups of students: racial/ethnic minorities, English language learners, and immigrants.
The first set of paired-samples t-tests was conducted to compare minority students’ social skills and problem behaviors scores before and after receiving counseling services. A statistically significant difference between pre- and posttest scores on the Social Skills scale was obtained, t{63) = -3.20,/> = .002. These results suggest that minority students had increased social skills after receiving counseling services. In the Problem Behaviors domain, a statistically significant difference was also observed, t{63) = 2,09, p = .04, with significantly lower scores obtained on the post-test. These results indicate that minority students had fewer Problem Behaviors after receiving counseling services.
The second set of paired-samples t-tests were conducted to compare English-language learning students’ social skills and problem behaviors scores before and after receiving counseling services. As with the analyses for minority students, statistically significant differences were obtained for the Social Skills scale, t{33) = -2.40,/) = .02, as well as for the Problem Behaviors scale, t{33) = 2.88,p = .007. These results suggest that English-language learning students had increased social skills and decreased problem behaviors after receiving counseling services.
The third set of paired-samples t-tests were conducted to compare immigrant students’ social skills and problem behavior scores before and after receiving counseling services. A statistically
18


significant difference between the pretest and posttest scores was obtained, t(10) = -3.62,p = .005. These results suggest that immigrant students had increased social skills after receiving counseling services. No statistically significant difference was observed for the Problem Behaviors domain for immigrant students. Table 3 provides a summary of the data related to the sub-group analyses.
No statistically significant differences on pre- and post-test scores within the Academic Competence domain were observed for any of the subgroups (minorities, English language learners, or immigrants).
Table 3
Results ofpaired sample t-tests comparing pre and posttest scores, separated by domain and sample group__________________________________________________________________
Sample Group Pre-test Posttest
Domain M SD M SD t df Sig.
Social Skills Minorities 84.94 13.59 89.70 15.81 -3.20 63 .002*
ELLs 84.68 15.12 90.21 16.73 -2.40 33 .02*
Immigrants 87.73 13.54 96.09 17.21 -3.62 10 .005*
Problem Behaviors Minorities 115.77 16.31 112.27 15.17 2.09 63 o *
ELLs 119 16.51 112.74 15.30 2.88 33 .007*
Immigrants 115.45 17.68 114.73 13.56 .184 10 .86
Academic Competence Minorities 90.30 13.30 91.30 13.10 -.78 63 .44
ELLs 87.26 13.39 90.12 13.46 -1.75 33 .09
Immigrants 91 16.41 90.55 15.81 .212 10 .84
19


CHAPTER V
DISCUSSION
The purpose of this study was to evaluate the Children First program in meeting its goal of providing effective mental health intervention to underserved populations. To date, virtually no research has been conducted examining the effectiveness of school-based mental health services with minority populations. To examine program effectiveness, 95 participants were assessed before and after receiving counseling through the Children First program. Specifically, participants’ social skills, problem behaviors, and academic skills were rated by teachers pre-and post-treatment. To assess the impact of this program on underserved populations, the following populations were examined within the participant group: racial/ethnic minorities, English-language learners, and immigrants.
Study results indicate that the Children First program is effectively providing therapeutic interventions to children with mental health concerns. The counseling provided through this program was associated with increased social-emotional skills and reduced problem behaviors for participants. This was true overall and also for the historically underserved groups of racial/ethnic minorities, English language learners, and immigrants. However, the Children First program did not appear to have a significant impact on the academic competence of participants. While this result is not surprising since counseling focuses more on social-emotional and behavioral needs than those that are more academic in nature, it is worth noting that gaining social-emotional skills may also have the potential to increase academic competence (Bradshaw et al., 2008). This is because children are better able to access the curriculum when they have the skills to regulate themselves (Bradshaw et al., 2008). However, there may have been inadequate
20


time for the participants to fully utilize and apply their improved social-emotional skills to result in a notable difference within the Academic Competence domain.
This evaluation highlights the importance of school-based mental health services, especially for traditionally underserved populations. It has been well established that minority groups are less likely to receive mental health care, yet they have greater levels of need that their non-minority peers (Gamble & Lambros, 2014). Despite the level of need, minorities are also less likely to utilize mental health care when it is available to them (Gamble & Lambros, 2014). However, previous research has indicated that minority youth are 20 times more likely to receive mental health care in their school than in a community mental health center (Bear et al., 2014). These mental health services are valuable to the school community due to their ability to impact children in multiple domains including their social, emotional, and behavioral wellbeing.
In this evaluation, the majority of the participants in the current study belongs to a nonwhite or minority race (67%). Additionally, 36% of program participants were English language learners and 12% of participants were immigrants. This data is proportionate to the demographics of the school district as a whole with 76% of the student population belonging to a minority race and 37% of students classified as English language learners (Denver Public Schools, 2017). As such, the Children First program is not disproportionately identifying students who belong to minority groups as needing mental health care.
Limitations and Future Research
This program evaluation illuminates the potential that school-based counseling services can have in meeting the mental health needs of children from diverse backgrounds. However, there are several limitations that should be considered when examining the findings. The current study utilized a relatively small sample size (n = 95) from which to draw conclusions. Further,
21


not all children who received services from the Children First program participated in the evaluation study. The Children First program served almost 300 students in the 2017-2018 school year, but only 95 students were included in the current study due to missing data.
Another limitation is found in the design of this evaluation. This study analyzed the effects of the Children First program without controlling for other influencing factors. For this reason, causal inferences related to the effects of the Children First program cannot be made. Other variables certainly influence a child’s development of social-emotional skills. Therefore, it can be difficult to determine how these students would have progressed without the services offered through Children First. Future research can address this issue by including a matched control group in the study.
Within the services offered through the Children First program, there was a wide range of counseling orientations, styles, and methods. Although the program met its goal of providing effective prevention and intervention services, it is difficult to determine if a particular method of counseling was more effective than others based on the data that was collected. This can make it difficult to provide meaningful feedback to the program on how to improve their services. For this reason, future research should collect more in-depth data regarding the actual counseling sessions. For example, information about theoretical orientation, style of treatment, length of sessions, and activities completed would further enhance the understanding of how the Children First program is impacting students. This would also allow the research to more accurately determine the effect of various types of counseling with these populations.
Conclusions and Implications for Practice
This evaluation sought to understand the impact of real-life interventions in real-world settings on populations that don’t traditionally receive mental health services. The results of this
22


program evaluation led to several important implications for practice. Firstly, this evaluation demonstrates the great need for mental health services in schools, particularly for culturally and linguistically diverse students. These populations are both historically and currently underserved in nearly all mental health settings, and as seen throughout the literature, schools are in a unique position to intervene with these populations. School services are convenient, typically low-cost or free, decrease the stigma of receiving services, and result in greater access to care (Doll et al., 2017).
This study contributes to the evidence that school-based mental health providers are effectively impacting children’s mental health. In particular, school-community partnerships like the one examined in this evaluation can contribute to the development of more socially and emotionally healthy children. School-community partnerships allow for a variety of mental health services to be provided to more children because each provider can engage in a particular area of service provision. For example, a school-based psychologist can focus on special education evaluations and services, a school-based social worker can engage in community integrations and family work, and a school-based therapist can engage in individual and group therapy for general education students. These school-community partnerships can fill in the gaps to promote a greater level of care when schools are not able to hire multiple school-based professionals for budgetary reasons. In this way, school-community partnerships can act as a low cost supplement to the services that are already being provided.
These types of programs also provide valuable experience for graduate-level interns. All services provided to clients via the Children First program were delivered by graduate students in mental health training programs. Such closely supervised field experiences are common practice and an effective means of developing counseling skills in trainees (Schuermann, Borsuk, Wong,
23


& Somody, 2018). The results of the present study clearly demonstrate that placing interns in the school setting is an efficacious way to provide counseling to children in need with little to no cost to the clients or the schools.
This evaluation has demonstrated that coordination among agencies, schools, and other community resources leads to a greater availability and impact of services. When available, school-based mental health services can overcome barriers to care and lead to significant, positive mental health outcomes. The Children First program effectively intervened in the participant’s lives by increasing their social/emotional skills and reducing their problem behaviors in school. The collaboration of school professionals, parents and children, and mental health professionals made these results possible and is critical to the achievement of these outcomes in the future.
24


REFERENCES
Adams, L. (2016). School-based mental health services for newly arriving immigrant adolescents. Journal of Immigrant and Minority Health, 18, 715-717.
Bear, L., Finer, R., Guo, S., & Lau, A. (2014). Building the gateway to success: An appraisal of progress in reaching underserved families and reducing racial disparities in school-based mental health. Psychological Services, 77(4), 388-397.
Beehler, S., Birman, D., & Campbell, R. (2012). The effectiveness of cultural adjustment and
trauma services (CATS): Generating practice-based evidence on comprehensive, school-based mental health intervention for immigrant youth. American Journal of Community Psychology, 50, 155-168.
Bradshaw, C., Buckley, J., & Ialongo, N. (2008). School-based service utilization among urban children with early onset educational and mental health problems: The squeaky wheel phenomenon. School Psychology Quarterly, 23(2), 169-186.
Castro-Olivo, S. (2017). Introduction to special issue: Culturally responsive school-based mental health interventions. Contemporary School Psychology, 21, 177-190.
Denver Public Schools (2017). Facts & Figures: Student Member ship. Retrieved from https://www.dpskl2.org/about-dps/facts-figures/
Dryfoos, J. (1994). Full-service schools: A revolution in health and social services for children, youth, andfamilies. San Francisco, CA: Jossey-Bass.
Doll, B., Nastasi, B., Cornell, L. & Song, S. (2017). School-based mental health services: Definitions and models of effective practice. Journal of Applied School Psychology,
33(3), 179-194.
Fazel, M., Doll, H., & Stein, A. (2009). A school-based mental health intervention for refugee
children: An exploratory study. Clinical child psychology and psychiatry, 14(2), 297-309.
Gamble, B., & Lambros, K. (2014). Provider perspectives on school-based mental health for urban minority youth: Access and services. Journal of Urban Learning, 10, 25-38.
Gonclaves, M. & Moleiro, C. (2012). The family-school-primary care triangle and the access to mental health care among migrant and ethnic minorities. Journal of Immigrant and Minority Health, 14(4), 682-690.
Gonzalez-Ramos, G., & Gonzalez, M. J. (2005). Chapter 3: Mental health care of Hispanic immigrant children. Journal of Immigrant & Refugee Services, 3(1-2), 47-58. doi:10.1300/jl91v03n01_03
25


Goodman R. (1997). The Strengths and Difficulties Questionnaire; a research note. Journal of Child Psychology and Psychiatry, 38(5), 581-586.
Gresham, F. & Elliott, S. N. (2008). Social Skills Improvement System (SSIS) Rating Scales. San Antonio, TX: Pearson.
Maria Droste. (2018). Children First Program. Retrieved from
http://www.mariadroste.org/what-we-do/programs/children-first-program/
O’Shea, B., Hodes, M., Down, G., & Bramley, J. (2000) A school-based mental health service for refugee children. Clinical Child Psychology and Psychiatry, 5(2), 189-201.
Owens, J., Murphy, C., Richerson, L., Girio, E., & Himawan, L. (2008). Science to practice in underserved communities: The effectiveness of school mental health programming. Journal of Clinical Child and Adolescent Psychology, 37(2), 434-447.
Powers, J., Swick, D., Wegmann, K., & Watkins, C. (2016). Supporting prosocial development through school-based mental health services: A multisite evaluation of social and behavioral outcomes across one academic year. Social Work in Mental Health, 14(1), 22-41.
Rones, M., & Hoagwood, K. (2000). School-based mental health services: A research review. Clinical Child and Family Psychology Review, 5(4), 223-241.
Schuermann, H., Borsuk, C., Wong, C., & Somody, C. (2018). Evaluating Effectiveness in a Hispanic-Serving Counselor Training Clinic. Counseling Outcome Research and Evaluation, 1-13.
Tyrer, R. A., & Fazel, M. (2014). School and Community-Based Interventions for Refugee and Asylum Seeking Children: A Systematic Review. PLoS ONE, 9(2).
26


Full Text

PAGE 1

SCHOOL BASED COUNSELING SERVICES WITH UNDERSERVED POPULATIONS: A MULTISITE EVALUATION OF OUTCOMES b y GABRIELA CAROL METZLER B.S., North Greenville University, 2015 A thesis submitted to the Faculty of the Graduate School of the University of Colorado in partial fulfillment of the requirements f or the degree of Doctor of Psychology School Psychology Program 2019

PAGE 2

ii © 2019 GABRIELA CAROL METZLER ALL RIGHTS RESERVED

PAGE 3

iii This thesis for the Doctor of Psychology degree by Gabriela Carol Metzler has been approved for the School Psychology program B y Franci Crepeau Hobson, Chair Bryn Harris Rachel Stein Date: May 18 , 2019

PAGE 4

iv Metzler, Gabriela Carol (PsyD., School Psychology Program) S chool Based Counseling Services with Underserved Populations: A M ultisite Evaluation of Outcomes Thesis directed by Associate Professor Franci Crepeau Hobson ABSTRACT This study sought to evaluate the school based couns eling services th at were provided to 95 students as part of the Children First counseling program. The primary evaluation question focuse d on the effectiveness of the program in positively impacting traditionally underserved populations in the following do mains: increasing social skills, decreasing problem behaviors, and increasing academic competence. A review of relevant literature revealed the impact of school based services on decreasing barriers to mental health services and improving the coordination of care among agencies, specifically with underserved populations. The results of the evaluation study indicated that the Children First program is effectively preventing mental health issues and intervening with children experiencing such challenges , incl uding those who are traditionally underserved. Implications for future research are discussed. The form and content of this abstract are approved. I recommend its publication. Approved: Franci Creapeau Hobson

PAGE 5

v DEDICATION To my husband and best friend, Joey: Thank you f or being someone that I can look up to and learn from. You have demonstrated what it means to be a student of people, of curiosities, and of life. You push me to be better, to dream bigger, to love deeper, and to pursue my passions with de dication and zeal. I adventures with you by my side, but especially the past four years of graduate school. Thank you for providing unending support and encouragement.

PAGE 6

vi TABLE OF CONTENTS CHAPTER I. INTRODUCTION II. LITERATURE REVIEW 6 III. METHOD 13 IV. RESULTS 17 V. DISCUSSION 20 REFERENCES .. 25

PAGE 7

1 CHAPTER I INTRODUCTION In the United States, one out of every five children will experience a clinically significant mental health condition during their childhood ( Powers, Swick, Wegmann, & Watkins, 2016 ). I t is estimated that two thirds of these children never receive treatment (Garrison, 1999) and the small number of children who do receive treatment are most likely to receive those services in their school environment (Rones & Hoagwood, 2000). This is beca use nearly 85% of children who receive mental health services report experiencing symptoms of their mental health challenges within the classroom (Bradshaw, Buckley, & Ialongo, 2008). Externalizing symptoms (i.e., aggression) and internalizing symptoms (i. e., anxiety) have both been shown to increase academic difficulties for children (Bradshaw et al., 2008). As such, school based intervention of mental health difficulties has implications for both academic and mental health outcomes for children (Bradshaw et al., 2008). School based mental health services integrate traditional mental health services such as counseling and psychological assessment within academic settings. Although the definition of school based services has long been debated, this paper wil l utilize a definition taken from Rones and Hoagwood (2000), which defines school intervention, or strategy applied in a school setting that was specifically designed to influence based mental health services can include those provided by school employed staff or can be provided by agencies that partner with districts to provide mental health services in the school building (Dryfoos, 1 994).

PAGE 8

2 Historically, the earliest prov ision of social services within the school setting occurred in income students and their families (Dryfoos, 1994). However, these services proved dif ficult to integrate into the school setting due to concern that the services of the school should not compete with private sector agencies ( Doll, Nastasi, Cornell, & Song, 2017 ). Throughout the rest of the 20 th century, the integration of school based ment al health services sporadically rose and fell in popularity . Many times, the difficulty of integration was due to the siloed effect of mental health services, in which school and community mental health exist as individual silos without integration of care (Doll et al., 2017). At the present time, many school districts around the United States are beginning to contract with community mental health agencies to provide more comprehensive mental health services in the school building (Bradshaw et al., 2008). T hese contracted services are typically for a specific service such as individual counseling or health education classes. These types of school community partnerships have the potential to greatly reduce the siloed effect of mental health services. However, the implementation of these types of partnerships has historically been difficult to maintain (Doll et al., 2017). Regardless of these difficulties, schools remain one of the most critical early identification and treatment systems for children with mental health concerns (Powers et al., 2015). School based services are in the unique position to decrease barriers to mental health services, to improve coordination of care among agencies, and to implement well rounded prevention and intervention strategies for children (Powers et al., 2015). By observing functioning in an everyday setting, school based service providers can understand skills and needs unfold in their natural form.

PAGE 9

3 Program Evaluation: Children First The mental health services evaluated in this study are part of a school based therapeutic counseling program called Children First. The Children First program is coordinated by a non profit mental health agency in Denver, CO . The overall mission of this organization is based on the belief that counseling services sho uld be accessible to all people regardless of their ability to pay. The organization strives to reduce barriers to mental health care and to meet the significant unmet mental health needs of the Denver population . This program provides free therapeutic counseling for children in their school buildings by contracting with local districts. Description of the Program The Childr en First program was based on the premise that early p revention and intervention help children gain adaptive skills before mental health issues can manifest. This program was established in 1990 to support children and adolescents who are experiencing emot ional, social, or behavioral problems with free of charge, on site counseling services in their schools (Maria Droste, 2018) . In this way, the Children First program offers a creative solution for schools that need mental health support in their buildings but lack the budget to hire a mental health counselor. This program also provides experience and internship credit to master s and doctoral level interns from a variety of graduate programs focused on mental health. Currently, the Children First program i s located in over 20 schools in Denver, many of which are located in diverse and e conomically disadvantaged areas (Maria Droste, 2018). To provide services to Children First clients, graduate level interns are placed in a local school to provide t herapeuti c counseling services to students . The interns receive supervision from a licensed mental health professional on staff with Maria Droste and receive on going training during the academic year. At each Children First school, the intern is also supported by a

PAGE 10

4 school based mental health professional such as a school psychologist or school social worker. Each intern is expected to be at their school site for a minimum of 10 hours per week , with some interns completing up to 16 hours a week in their school depen ding on their track of study. It is expected that for each hour the intern is in the school , they have one student client (i.e., a n intern who works 10 hour s a week should have 10 students on their case load). Counseling sessions are generally advised to b e 30 minutes long, wit h some sessions lasting up to 50 minutes depending on the needs. While Maria Droste provides training on various therapeutic orientations and techniques, there are no set requirements for the activities that must be complete d in counseling. Instead, Maria Droste allows each intern to use their professional training in their area of specialization. Some Children First counselors use traditional cognitive behavioral therapy, while others utilize play thera py techniques, trauma informed techniques, and many other therapeutic approaches . The program also encourages interns to have both individual and group based sessions. Purpose of the Study The purpose of this study wa s to evaluate the extent to which the Children First program is meet ing its goal of providing effective early prevention and intervention services . Specifically, this program evaluation measure d whether the Children First program is effective in developing social and emotional skills, reducing problem behaviors, and increasing academic skills in clients served . While also looking to the mission of Maria Droste , this program evaluation s ought to understand how the Children First program is achieving these outcomes with the following traditionally underserved populatio ns: racial/ethnic minorities, English language learners, and first generation immigrants.

PAGE 11

5 Evaluation Question Is the Children First counseling program effective in developing social and emotional skills, reducing problem behaviors, and increasing academi c skills, particularly with regard to the following traditionally underserved populations: minorities, English language learners, and first generation immigrants?

PAGE 12

6 CHAPTER II LITERATURE REVIEW The School as a Setting for Mental Health Services Behavioral, emotional, and social difficulties can affect children in any context. However, these difficulties can be particularly troublesome in the school environment because children are in their school building for roughly six hours a day, 180 days of the year (Bradshaw et al., 2008). The school setting is one in which children are learning more than division and geography. Children are learning how to navigate social relationships, how to regulate their emotions, how to communicate their needs, and how to develop many more social emotional skills. If a child has difficulty with social emotional skills, they are also at an increased risk of academic difficulties because these skills are so deeply interconnected (Powers et al., 2015). The impact of th ese skills in the academic environment makes it a perfect setting to incorporate mental health services directly in the school. Of the many benefits to the integration of mental health care in the school setting, the most obvious is the increased access to care for children with mental health concerns. Barriers are reduced when children have access to care in a convenient location, during a time that they will already be in school, and when the services are low cost or free (Doll et al., 2017). Additionally , students and families who are already familiar with the school building and the school staff are less likely to experience stigma in receiving mental health services (Doll et al., 2017). Most schools already employ mental health professionals who are pre pared to provide a continuum of services and to coordinate care with outside providers (Bradshaw et al., 2008). nt (Bradshaw et al., 2008).

PAGE 13

7 By embedding mental health services within schools, the capacity for prevention and intervention services is enhanced ( Owens, Murphy, Richerson, Girio, & Himawan, 2008 ). Recent frameworks of treatment suggest that schools and co mmunity mental health centers must work together to provide a continuum of care (Owens et al., 2008). This partnership is needed because many school employed providers are not able to deliver intensive supports. In a qualitative study conducted with school employed mental health providers, many participants indicated that schools need designated, in house mental health providers who offer different types of services like individual counseling, special education evaluations, or family therapy ( Gamble & Lambr os, 2014 ). Unfortunately, many schools are not able to employ a sufficient number of mental health providers to meet the needs of all students. Because the school setting is primarily an academic setting, many students will only receive services if their n eeds are impacting their academic progress. In this way, school based mental health providers may not be able to apply concepts of evidence based practice to everyday, individualized treatments due to the nature of the school setting ( Beehler, Birman, & Ca mpbell, 2012 ). However, research still demonstrates that most children who do receive mental health services will receive them from a school based provider (Bradshaw et al., 2008). The Provision of School Based Services to Underserved Populations Throughou t history, minority groups have experienced greater levels of unmet mental health need due to limited access of care and an inferior quality of treatment ( Gamble & Lambros, 2014 ). There is significant research that also demonstrates that minority youth not only have greater mental health needs compared to their majority peers, but also underutilize mental health care services ( Gamble & Lambros, 2014 ). Minority children and adolescents are more

PAGE 14

8 likely to be impacted by poverty and its associated risk factors , such as living in neighborhoods with fewer resources ( Gamble & Lambros, 2014 ). Despite the increased risk for mental health challenges, minorities are less likely to receive necessary mental health services when compared to non minorities due to a range of barriers ( Gonclaves & Moleiro, 2012 ). Families may not recognize that their child is in distress, may not be aware of support services, may not have transportation to get to service centers, and et al., 2000). School based staff may misunderstand or may not even recognize that a child is experiencing mental health difficulties (Gonzales Ramos & Gonzalez, 2005). Gonclaves and Moleiro (2012) separate these barriers into objective and subjective barr iers. Objective barriers tend to be structural or system downfalls, while a subjective barrier typically has to do with the client themselves ( Gonclaves & Moleiro, 2012 ). For example, an objective barrier might involve a lack of available providers and a s barriers, there is even greater reliance on non medical settings, like schools to prevent, detect, and intervene with mental health concerns among culturall y and linguistically diverse children Research has found that school based services are more likely to be utilized by children of color. For example, in one systematic review of school based mental health clinics, researchers noted that minority youth were 20 times more likely to engage in mental health services at their school based clinic than at a community mental health center ( Bear, Finer, Guo, & Lau, 2014 ). While there is evidence that school based clinics reduce barriers to ca re for traditionally underserved populations, inequities may still exist in school based mental health care ( Bear et al., 2014 ). For example, Bear and colleagues found that non minority students are

PAGE 15

9 more likely than minorities to receive care in their scho ols regardless of evidence that suggests that minorities are at risk for greater mental health problems ( Bear et al., 2014 ). This study demonstrates how school based services can maintain the same inequities of treatment for traditionally underserved stude nts ( Bear et al., 2014 ). This is especially concerning for immigrant and refugee populations because these children are often at higher risk of mental health concerns due to psychological risk factors like stress from acculturation and prejudice from their communities (Goncalves, 2012). In particular, refugees have an increased risk of psychopathology due to the increased likelihood of experiencing traumatic events such as exposure to violence or war activities ( ). During all stages of their migration journey, these populations experience chronic adversity: in their native country, on their journey to a new country, and once settled in a new country ( Fazel, Doll, & Stein, 2009 ). This journey affects their families, their e ducation, their social relationships, and other critical systems in their lives (Fazel et al., 2009). of referring refugee and immigrant students to mental health cl inics generally leads to low rates of follow through and ultimately, a lack of service provision for these populations. If these populations do receive care, it is often of lower quality due to a lack of cultural responsiveness from mental health service p roviders ( Gonclaves & Moleiro, 2012 ). Given the increased risk for mental health challenges in diverse populations, as well as the increase in minority groups in the U.S., it is critical that mental health interventions be culturally sensitive. However, mo st evidence based treatments have been developed for the majority population without consideration of factors like culture, language acquisition, and socioeconomic status (Castro Olivo, 2017). Not surprisingly, many practitioners still have difficulty iden tifying and

PAGE 16

10 implementing effective treatments for students from culturally or linguistically diverse backgrounds (Castro Olivo, 2017). This affects immigrant and refugee populations most of all because these students do not lose their cultural backgrounds simply because they are located in a new country, but rather, they adapt to new cultural situations using their pre existing worldview. In a qualitative study completed by Gonclaves and Moleiro (2012), culturally and linguistically diverse students, their teachers, and their parents were interviewed about the ways that mental health service providers could demonstrate cultural competence. Throughout the interviews, students consistently noted that language, traditions, habits, preferences, family, ways of life, education, and the acculturation process were most important for their service provider to understand . Teachers believed that a culturally competent service provider would have knowledge about the individual history of the client, as well as the clie ( Gonclaves & Moleiro, 2012 culture and their individual experiences to effectively demonstrate cultural competence. While there is a wealth of studies that have explo red the disparities of mental health care among minorities, there are few studies that generalize the findings of program applications with traditionally underserved populations ( Gamble & Lambros, 2014 ). In particular, very little research has evaluated re al world, school based mental health services for immigrant and refugee children (Fazel et al., 2009 & Beehler et al., 2012). In one of the few studies in this area, Fazel et al. (2009) investigated the impact of school based mental health services with a group of refugee students. These students received consultative and counseling services by a mental health team at three schools. These students were assessed by their teachers before receiving services and again at the end of the school year using the Str engths and Difficulties

PAGE 17

11 Questionnaire (Fazel et al., 2009). The refugee students were compared to other groups who were also receiving services and the refugee group had statistically significant improvements in two domains in particular: hyperactivity and peer relations (Fazel et al., 2009). The authors of integration such as adjusting to a new environment, understanding the school system, and developing a sens e of self in a new culture. This study also compared students who were receiving direct services with those who were receiving indirect or consultative services. In this case, direct services were found to be more beneficial for refugee students than indir ect services (Fazel et al., 2009). To determine what type of direct services are most beneficial culturally and linguistically diverse children, a review of studies was conducted. Tyrer and Fazel (2014) found that verbal processing and creative art based t herapies led to the most significant reduction in mental health symptoms, most particularly those related to depression, anxiety, PTSD, and peer problems. In this case, verbal processing therapies include cognitive behavioral therapy and trauma systems the rapy, while creative art therapies in clude play therapy and movement based therapies. The authors attributed the success of these programs on the incorporation of both individual and systems based therapeutic processing (Tyrer & Fazel, 2014). Although thes e studies have explored the impacts of school based mental health services, there is little research that has examine d the following factors: the effectiveness of school based mental health services and the provision of those services with traditionally un derserved populations such as minorities, English language learners, and immigrants and refugees. Due to the lack of research and practical examination of programs, it is largely unknown what types of school based mental health services are most effective with culturally and linguistically diverse

PAGE 18

12 students, as well as what constitutes effective practice. There is also little research on how school based services impact these populations in a broader context which includes their mental health, social skills, behavior, and other constructs outside the social and emotional realm like academic achievement. As such, it is difficult for service providers to implement culturally sensitive, evidence based practices in their schools. It is clear, however, that school based mental health professionals need to tailor their services to each individual child and take into account their cultural background, particularly due to the growing populations of culturally and linguistically diverse children in the United States. T he present evaluation is an attempt to br idge this gap in the literature by examining the effectiveness of school based mental health services provided by the Children First program in meeting the needs of children from diverse backgrounds.

PAGE 19

13 CHAPTER III METHOD Procedures and Treatment Clients were referred to the Children First program consistent with the referral procedures of their respective schools. For example, many schools had the school based liaison (i.e., the school psychologist) collect intake information and determine if that student should be referr ed to the Children First intern for counseling. Once referred for counseling, the Children First intern was provided general information about the child and their family by the school district . The intern then me t with the family to gather referral information and explore During this intake process, the intern obtained written permission treatment, as well as for the completion of pre and post treatment teacher ratings. Once treatment was initiated, the intern provide d the Social Skills Improvement System ( SSIS ; Gresham & Elliott, 2008 ) form t The intern then re administered the form around the si x month mark of treatment as the post test data. Participants Although the Children First program served c lose to 300 students in the 2017 2018 academic school year, complete and reliable data could only be collected for 95 students. These 95 students were referred to and served by the Childr en First program during the 2017 18 school year, in their respective schools. These 95 participants were s pread across 15 schools in the Denver metro area. Sample demographics are provided in Table 1. A little more than half of the students who made up the sample were male (55%). Every grade from kindergarten through eighth grade was

PAGE 20

14 represented in the sample . However, the majority of the sample (72%) was in grades two through five. Of the 95 participants, the majority of the sample belongs to a non white minority population (67%), and a little over one third of the sample (37 %) learned English as a second or alternative language. While the majority of the sample was born and raised in the United States, 11 students (11.6%) were born outside of the U.S. , and a total of 11 countrie s were represented in the sample (including the United States). Table 1 . Demograp hic Characteristics of Participants Characteristic n Percentage of Sample Sex : Female 43 45.26% Male 52 54.74% Grade : Kindergarten 4 0.04% 1 st grade 9 0.09% 2 nd grade 14 0.15% 3 rd grade 19 0.20% 4 th grade 19 0.20% 5 th grade 17 17.89% 6 th grade 3 0.03% 7 th grade 8 0.08% 8 th grade 2 0.02% Ethnicity : Minority 64 67.37% Non Hispanic White 31 32.63% Country of Origin : United States 84 88.42% Other country 11 11.58% Language of Origin : English 61 64.21% Other language 34 35.79% Measures Teachers completed the Social Skills Improvement System (SSIS ; Gresham & Elliot, 2008 social skills, problem behaviors, and academic competence both pre and post treatment . The SSIS was selected because Maria Droste Counseling Center has

PAGE 21

15 historically used this measure to determine functioning in several domains. The SSIS is also a brief measure that can easily be administered and score d. The SSIS functioning in each of the following domains: 1. Social Skills : Communication, Cooperation, Assertion, Responsibi lity, Empathy, and Self Control 2. Problem Behaviors : Externalizing, Bullying, Hyperactivity/Inattention, and Inter nalizing behaviors 3. Academic Competence : Reading and Math Performance , Motivation, Parental Support, and Overall Functioning The SSIS includes 83 items which are assessed on a Likert scale. On the Social Skills and Problem Behaviors domains, the teacher indicates the frequency of which the student exhibits a behavior with a 4 point scale that includes Never, Seldom, Often, and Almost Always . Then, on the Academic Competence scale , tea chers indicate the math, and learning behaviors by comparing the student in question to the rest of the class. Using a 5 point sca le, the teacher rates 1) if the student is in the lowest 10% of the class, 2) the ne xt lowest 20%, 3) the middle 40%, 4) the next highest 20%, or 5) th e highest 10%. Administration of the SSIS takes roughly 5 10 minutes. The SSIS was normed with a nationally representative sample of nearly 5,000 chi ldren aged 3 18 years old . The SSIS has demonstrated acceptable psychometric properties with alpha coefficients ranging from .83 to .97 (Gresham & Elliot, 2008). There are also three measures built into the SSIS to help the examiner j udge the validity of the scores: a negativity index (F scale) , a response pattern index, and a response consistency index. These scores help determine

PAGE 22

16 of the questions), and if the rater answers related questions i n similar ways. For each of the domains, raw scores are added and converted into standard scores which have a mean of 10 0 and a standard deviation of 15 . When interpreting the SSIS standard scores, the Social Skills and Academic Competence scales assess po sitive behaviors while the Problem Behaviors scale assesses negative behaviors. This means that higher scores are desired on the Social Skills and Academic Competence scales while lower scores are desired on the Problem Behaviors scale. For each of the subdomains, raw scores are converted into a category based on standard deviations to fit into the Below Average, Average, or Above Average level. Analyses f increasing social skills and academic competence and decreasing problem behaviors, three repeated measures ANOVAs were conducted with respective pre and post treatment SSIS scale scores (Social Skills, Academic Competence, and Problem Behaviors). To det ermine if the program was effective with the specific populations of interest (minorities, English language learners, and immigrants), three separate paired sample t tests were conducted for each group. Data was analyzed using the Statistical Package for S ocial Sciences (SPSS). Descriptive statistics were analyzed to demonstrate demographic characteristics of each participants.

PAGE 23

17 CHAPTER IV RESULTS Whole Group Analysis As described above, three repeated measures ANOVAs were conducted to assess whether there were significant differences between the average ratings of pre and post test measures on each of the following scales of the SSIS: Social Skills, Problem Behaviors, and Academic Competence. Assumptions were checked and met . Results indicated that students had significantly higher average scores on the Social Skills scale post treatment , F (1, 94) = 12.81, p < .001, partial = .12. This suggests that teachers rated students as having increased social skills after receiving counseling services. There was also a statistically significant difference in Problem Behaviors scale scores pre and post treatment , F (1, 94) = 4.32, p = .05, partial 2 = .04. Results indicate that teachers rated students as having significantly fewer problem behaviors after receiving counseling services. No statistically significant differences were found for pre and post treatment scores on the Academ ic Competence scale. Pre and post treatment SSIS data are presented in Table 2. Table 2 Means and Standard Deviations of the Pre and Post treatmen t SSIS Ratings SSIS Scale Pre treatment (SD) Post treatment (SD) Social Skills ** 87.43 (13.45) 91.52(14.75) Problem Behaviors * 113.54 (16.36) 110.95 (14.77) Academic Competence 92.35 (14.15) 93.49 (14.40) * p < .05 ** p < .001

PAGE 24

18 Sub Group Analysis Following the analyses of overall pre and post treatment SSIS ratings, paired samples t tests were conducted to examine differences for specific subgroups of interest. Analyses of the Social Skills , Problem Behavior , and Academic competence scales were conducted for the following groups of students: racial/ethnic minorities, English language learners, and immigrants. The first set of paired samples t skills and problem behaviors scores before and after receiving counseling services. A statistically significant difference between pre and posttest scores on the Social Skills scale was obtained , t (63) = 3.20, p = .002. These results suggest that minority students had increased social skills after receiving counseling services. In the Problem Behaviors domain, a statistically significant difference was also observed, t (63) = 2.09, p = .04 , with significantly lower scores obtained on the post test . These results indicate that minority students had fewer Problem Behaviors after receiving counseling services. The second set of paired samples t tests wer e conducted to compare English language counseling services. As with the analyses for minority students, statistically significant differences were obtained for the So cial Skills scale, t (33) = 2.40, p = .02, as well as for the Problem Behaviors scale, t (33) = 2.88, p = .007. These results suggest that English language learning students had increased social skills and decreased problem behaviors after receiving counsel ing services. The third set of paired samples t skills and problem behavior scores before and after receiving counseling services. A statistically

PAGE 25

19 significant difference between the pretest and pos ttest scores was obtained , t (10) = 3.62, p = .005. These results suggest that immigrant students had increased social skills after receiving counseling services. No statistically significant difference was observ ed for the Problem Behaviors domain for imm igrant students. Table 3 provides a summary of the data related to the sub group analyses. N o statistically significant differences on pre and post test scores within the Academic Competence domain were observed for any of the subgroups (minorities, Engli sh language learners, or immigrants). Table 3 Results of paired sample t test s comparing pre and posttest scores, separated by domain and sample group Pre test Posttest Domain Sample Group M SD M SD t df Sig. Social Skills Minorities 84.94 13.59 89.70 15.81 3.20 63 .002* ELLs 84.68 15.12 90.21 16.73 2.40 33 .02* Immigrants 87.73 13.54 96.09 17.21 3.62 10 .005* Problem Behaviors Minorities 115.77 16.31 112.27 15.17 2.09 63 .04* ELLs 119 16.51 112.74 15.30 2.88 33 .007* Immigrants 115.45 17.68 114.73 13.56 .184 10 .86 Academic Competence Minorities 90.30 13.30 91.30 13.10 .78 63 .44 ELLs 87.26 13.39 90.12 13.46 1.75 33 .09 Immigrants 91 16.41 90.55 15.81 .212 10 .84

PAGE 26

20 CHAPTER V DISCUSSION The purpose of this study was to evaluate the Children First program in meeting its goal of providing effective mental health intervention to underserved populations. To date, virtually no research has been conducted examining the effectiveness of school based mental he alth services with minority populations . To examine program effectiveness, 95 participants were assessed before and after receiving counseling through the Children First program. Specifically, ills were rated by teachers pre and post treatment. To assess the impact of this program on underserved populations, the following populations were examined within the participant group: racial/ethnic minorities, English language learners, and immigrants. Study results indicate that the Children First program is effectively providing therapeutic interventions to children with mental health concerns. The counseling provided through this program was associated with increased social emotional skills and reduc ed problem behaviors for participants. This was true overall and also for the historically underserved groups of racial/ethnic minorities, English language learners, and immigrants. However, the Children First program did not appear to have a significant i mpact on the academic competence of participants. While this result is not surprising since counseling focuses more on social emotional and behavioral needs than those that are more academic in nature, it is worth noting that gaining social emotional skill s may also have the potential to increase academic competence (Bradshaw et al., 2008). This is because children are better able to access the curriculum when they have the skills to regulate themselves (Bradshaw et al., 2008). However, there may have been inadequate

PAGE 27

21 time for the participants to fully utilize and apply their improved social emotional skills to result in a notable difference within the Academic Competence domain. T his evaluation highlights the importance of school based mental health service s, especially for traditionally underserved populations . It has been well established that minority groups are less likely to receive mental health care, yet they have greater levels of need that their non minority peers ( Gamble & Lambros, 2014 ). Despite t he level of need, minorities are also less likely to utilize mental health care when it is available to them ( Gamble & Lambros, 2014 ). However, previous research has indicated that minority youth are 20 times more likely to receive mental health care in th eir school than in a community mental health center ( Bear et al., 2014 ). These mental health services are valuable to the school community due to their ability to impact children in multiple domains including their social, emotional, and behavioral wellbei ng. In this evaluation, t he majority of the participants in the current study belong s to a non white or minority race (67%). Additionally, 36% of program participants were English language learners and 12% of participants were immigrants. This data is pro portionate to the demographics of the school district as a whole with 76% of the student population belonging to a minority race and 37% of students classified as English language learners (Denver Public Schools, 2017). As such, the Children First program is not disproportionately identifying students who belong to minority groups as needing mental health care. Limitations and Future Research This program evaluation illuminates the potential that school based counseling services can have in meeting the mental health needs of children from diverse backgrounds. However, there are several limitations that should be considered when examining the findings. The current study utilized a relatively small sample size (n = 95) from which to draw conclusions. Furth er,

PAGE 28

22 not all children who received services from the Children First program participated in the evaluation study. The Children First program served almost 300 students in the 2017 2018 school year, but only 95 students were included in the current study due to missing data. Another limitation is found in the design of this evaluation. This study analyzed the effects of the Children First program without controlling for other influencing factors. For this reason, causal inferences related to the effects of t he Children First program cannot be made. emotional skills. Therefore, it can be difficult to determine how these students would have progressed without the services offered through Childr en First. Future research can address this issue by including a matched control group in the study. Within the services offered through the Children First program, there was a wide range of counseling orientations, styles, and methods. Although the program met its goal of providing effective prevention and intervention services, it is difficult to determine if a particular method of counseling was more effective than others based on the data that was collected. This can make it difficult to provide meaningf ul feedback to the program on how to improve their services. For this reason, future research should collect more in depth data regarding the actual counseling sessions. For example, information about theoretical orientation, style of treatment, length of sessions, and activities completed would further enhance the understanding of how the Children First program is impacting students. This would also allow the research to more accurately determine the effect of various types of counseling with these populat ions. Conclusions and Implications for Practice This evaluation sought to understand the impact of real life interventions in real world

PAGE 29

23 program evaluation le d to several important implications for practice. Firstly, this evaluation demonstrates the great need for mental health services in schools, particularly for culturally and linguistically diverse students. These populations are both historically and cu rrently underserved in nearly all mental health settings, and as seen throughout the literature, schools are in a unique position to intervene with these populations. School services are convenient, typically low cost or free, decrease the stigma of receiv ing services, and result in greater access to care ( Doll et al., 2017). This study contributes to the evidence that school based mental health providers are , school community partnerships like the one examined in this evaluation can contribute to the development of more socially and emotionally healthy children. School community partnerships allow for a variety of mental health services to be provided to more children because each provider can e ngage in a particular area of service provision. For example, a school based psychologist can focus on special education evaluations and services, a school based social worker can engage in community integrations and family work, and a school based therapi st can engage in individual and group therapy for general education students. These school community partnerships can fill in the gaps to promote a greater level of care when schools are not able to hire multiple school based professionals for budgetary re asons. In this way, school community partnerships can act as a low cost supplement to the services that are already being provided. These types of programs also provide valuable experience for graduate level interns. All services provided to clients via th e Children First program were delivered by graduate students in mental health training programs. Such closely supervised field experiences are common practice and an effective means of developing counseling skills in trainees ( Schuermann, Borsuk, Wong,

PAGE 30

24 & S omody , 2018). The results of the present study clearly demonstrate that placing interns in the school setting is an efficacious way to provide counseling to children in need with little to no cost to the clients or the schools. This evaluation has demonst rated that c oordination among agencies, schools, and other community resources leads to a greater availability and impact of services. When available, school based mental health services can overcome barriers to care and lead to significant, positive menta l health outcomes. The Children First program effectively intervened in the behaviors in school. The collaboration of school professionals, parents and children, and mental health professionals made these results possible and is critical to the achievement of these outcomes in the future.

PAGE 31

25 REFERENCES Adams, L. (2016). School based mental health services for newly arriving immigrant adolescents. Journal of Immigrant and Minority Health, 18 , 715 717. Bear, L., Finer, R., Guo, S., & Lau, A. (2014). Building the gateway to success: An appraisal of progress in reaching underserved families and reducing racial disparities in school based mental health. Psychological Servic es, 11 (4), 388 397. Beehler, S., Birman, D., & Campbell, R. (2012). The effectiveness of cultural adjustment and trauma services (CATS): Generating practice based evidence on comprehensive, school based mental health intervention for immigrant youth. American Journal of Community Psychology , 50 , 155 168. Bradshaw, C., Buckley, J., & Ialongo, N. (2008). School based service utilization among urban children with early onset educational and mental health problems: The squeaky wheel phenomenon. School Psyc hology Quarterly, 23 (2), 169 186. Castro Olivo, S. (2017). Introduction to special issue: Culturally responsive school based mental health interventions. Contemporary School Psychology, 21 , 177 190. Denver Public Schools (2017). Facts & Figures: Student Me mbership . Retrieved from https://www.dpsk12.org/about dps/facts figures/ Dryfoos, J. (1994). Full service schools: A revolution in health and social services for children, youth, and families . San Francisco, CA: Jossey Bass. Doll, B., Nastasi, B., Cornell , L. & Song, S. (2017). School based mental health services: Definitions and models of effective practice. Journal of Applied School Psychology, 33 (3), 179 194. Fazel, M., Doll, H., & Stein, A. (2009). A school based mental health intervention for refugee children: An exploratory study. Clinical child psychology and psychiatry, 14 (2), 297 309. Gamble, B., & Lambros, K. (2014). Provider perspectives on school based mental health for urban minority youth: Access and services. Journal of Urban Learning, 10 , 25 38. Gonclaves, M. & Moleiro, C. (2012). The family school primary care triangle and the access to mental health care among migrant and ethnic minorities. Journal of Immigrant and Minority Health, 14 (4), 682 690. González Ramos, G., & Go nzález, M. J. (2005 ). Chapter 3: Mental health care of Hispanic immigrant c hildren. Journal of Immigrant & Refugee Services, 3 (1 2), 47 58. doi:10.1300/j191v03n01_03

PAGE 32

26 Goodman R. (1997). The Strengths and Difficulties Questionnaire; a research note. Journal of Child Psychology and Psychiatry, 38 (5), 581 586. Gresham, F. & Elliott, S. N. (2008). Social Skills Improvement System (SSIS) Rating Scales . San Antonio, TX: Pearson. Maria Droste. (2018). Children First Program. Retrieved from http://www.mariadroste.org/what we do/programs/children first program/ based mental health service for refugee children. Clinical Child Psychology and Psychiatry, 5 (2), 189 201. Owens, J., Murphy, C., Richerson, L., Girio, E., & Himawan, L. (2008). Science to practice in underserved communities: The effectiveness of school mental health programming. Journal of Clinical Child and Adolescent Psychology, 37 (2), 434 447. Powers, J., Swick, D., Wegmann, K., & Watkins, C. (2016). Supporting prosocial development through school based mental health services: A multisite evaluation of social and behavioral outcomes across one academic year. Social Work in Mental Health, 14 (1), 22 41. Rones, M., & Hoagwood, K. (2000). School based ment al health services: A research review. Clinical Child and Family Psychology Review, 3 (4), 223 241. Schuermann, H., Borsuk, C., Wong, C., & Somody, C. (2018). Evaluating Effectiveness in a Hispanic Serving Counselor Training Clinic. Counseling Outcome Research and Evaluation , 1 13. Tyrer, R. A., & Fazel, M. (2014). School and Community Based Interventions for Refugee and Asylum Seeking Children: A Systematic Review. PLoS ONE, 9 (2).