Citation
Performance measures, marginalized populations, and democratic participation

Material Information

Title:
Performance measures, marginalized populations, and democratic participation the case of permanent supportive housing
Added title page title:
Case of permanent supportive housing
Creator:
Fenley, Vanessa M. ( author )
Place of Publication:
Denver, Colo.
Publisher:
University of Colorado Denver
Publication Date:
Language:
English
Physical Description:
1 electronic file (169 pages) : ;

Thesis/Dissertation Information

Degree:
Doctorate ( Doctor of philosophy)
Degree Grantor:
University of Colorado Denver
Degree Divisions:
School of Public Affairs, CU Denver
Degree Disciplines:
Public affairs

Subjects

Subjects / Keywords:
Public administration -- Evaluation ( lcsh )
Public administration -- Decision making ( lcsh )
Public administration -- Social aspects ( lcsh )
Genre:
bibliography ( marcgt )
theses ( marcgt )
non-fiction ( marcgt )

Notes

Review:
Democratic participation has been integrated in public administration theory and practice since the mid 20th century. It is a strategy used by public entities to enhance social equity by engaging citizens, including those who are marginalized and lack political efficacy. Most efforts to engage the public focus on program or policy planning rather than developing and interpreting performance measures. Even more limited are efforts to engage marginalized citizens in this work. This dissertation explores how preferences for performance measures differ between public sector representatives at the federal, state, and local levels and the marginalized citizens served by the public program of focus. Permanent supportive housing, or housing designed specifically for individuals moving out of homelessness, serves as a case in point to explore this research question. Research findings indicate that marginalized citizens differ frequently from public sector representatives in regards to both the importance assigned to selected performance measures and to their perceptions of what specific performance measures may indicate. Findings contribute to the literature on performance measurement, democratic participation, and social equity, and have practical applications for public administrators seeking to engage citizens in public processes.
Thesis:
Thesis (Ph.D.)--University of Colorado Denver, 2017.
Bibliography:
Includes bibliographical references.
System Details:
System requirements: Adobe Reader.
Statement of Responsibility:
Vanessa M. Fenley.

Record Information

Source Institution:
University of Colorado Denver
Holding Location:
Auraria Library
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
on10307 ( NOTIS )
1030749702 ( OCLC )
on1030749702

Downloads

This item has the following downloads:


Full Text
PERFORMANCE MEASURES, MARGINALIZED POPULATIONS,
AND DEMOCRATIC PARTICIPATION:
THE CASE OF PERMANENT SUPPORTIVE HOUSING
by
VANESSA M. FENLEY B.S., Missouri State University, 2004 M.A., University of Kansas, 2006
A thesis submitted to the Faculty of the Graduate School of the University of Colorado in partial fulfillment of the requirements for the degree of Doctor of Philosophy Public Affairs Program
2017


This thesis for the Doctor of Philosophy degree by Vanessa M. Fenley has been approved for the Public Affairs Program by
Todd Ely, Chair Mary E. Guy, Advisor Christine Martell Donald Klingner
Date: December 16, 2017
n


Fenley, Vanessa M. (PhD, School of Public Affairs)
Performance Measures, Marginalized Populations, and Democratic Participation: The Case of Permanent Supportive Housing Thesis directed by Professor Mary E. Guy
ABSTRACT
Democratic participation has been integrated in public administration theory and practice since the mid 20th century. It is a strategy used by public entities to enhance social equity by engaging citizens, including those who are marginalized and lack political efficacy. Most efforts to engage the public focus on program or policy planning rather than developing and interpreting performance measures. Even more limited are efforts to engage marginalized citizens in this work. This dissertation explores how preferences for performance measures differ between public sector representatives at the federal, state, and local levels and the marginalized citizens served by the public program of focus. Permanent supportive housing, or housing designed specifically for individuals moving out of homelessness, serves as a case in point to explore this research question. Research findings indicate that marginalized citizens differ frequently from public sector representatives in regards to both the importance assigned to selected performance measures and to their perceptions of what specific performance measures may indicate. Findings contribute to the literature on performance measurement, democratic participation, and social equity, and have practical applications for public administrators seeking to engage citizens in public processes.
The form and content of this abstract are approved. I recommend its publication.
Approved: Mary E. Guy
m


ACKNOWLEDGEMENTS
Throughout my doctoral studies, Mary Guy has served as an incomparable mentor and advisor. She kept me motivated, and helped me to see the true potential of my research during this dissertation process. Christine Martell also provided tremendous support, particularly during the beginning phases of my dissertation when my research ideas were just beginning to take shape. My other committee members, Todd Ely and Don Klingner, both were supportive and engaged committee members, offering advice and recommendations that strengthened this dissertation and my research agenda. I am appreciative of their support and the time they took to serve on my committee.
The staff at Housing Catalyst have been exceptional colleagues. Without their willingness to work with me and assist me in arranging to conduct my research at Redtail Ponds, I would not have been able to complete this dissertation. In addition, I appreciate all they do everyday to serve the people of Fort Collins, providing homes for people who otherwise may not have a roof over their head.
IV


TABLE OF CONTENTS
CHAPTER
I. STATEMENT OF THE PROBLEM 1
Participatory Democracy as a Critical Component of Public Administration 2
Participatory Democracy and Performance Regimes 6
Homelessness as a Marginalizing Experience 8
Homelessness in Fort Collins, Colorado 11
Research Question 14
IT LITERATURE REVIEW 17
Participatory Democracy and Citizen Engagement 17
A Brief History of Citizen Participation in the Public Sector 17
The Rationale Behind Citizen Engagement 23
Considerations in Engaging the Public in Decision-Making 25
Special Considerations for Marginalized Populations 28
Performance Measurements and Performance Regimes 31
Citizen Participation and Performance Regimes 33
Findings from Current Research on Citizen Engagement in Performance Regimes 37
Special Considerations for Marginalized Populations 40
Housing and Homeless Assistance in the United States 41
Permanent Supportive Housing in Fort Collins, Colorado 45
III. RESEARCH DESIGN 52
Sampling and Recruitment 53
v


Study Participant Selection 55
Data Collection and Measurement Strategy 58
Explanation of Performance Measures 62
Analysis Strategy 65
IV. RESULTS 68
Characteristics of Respondents 68
Summary of Performance Measure Rankings for Residents 70
Housing Retention 71
Chronicity and Vulnerability 74
Resident Satisfaction 75
Reduced Jail Bed Days 76
Increased Income 78
Reduced Shelter Bed Days 81
Reduced Emergency Department (ED) Visits 83
Community Satisfaction 85
Voluntary Participation in Services 87
Summary of Performance Measure Rankings for Professionals 89
Housing Retention 90
Chronicity and Vulnerability 91
Voluntary Participation in Services 93
Reduced Emergency Department (ED) Visits 95
Reduced Jail Bed Days 97
Resident Satisfaction 99
vi


Reduced Shelter Bed Days 101
Community Satisfaction 102
Increased Income 104
Other Themes Among Respondents 108
Alignment of Preferences for Performance Measures Between Residents and Professionals 115
Resident Involvement in Establishing Performance Regimes 121
V. DISCUSSION AND CONCLUSIONS 125
Impacts for Theory and Practice 131
Limitations 138
Future Re search 141
REFERENCES 144
APPENDIX
A. Comparisons of Resident and Professional Responses 152
B. Interview Tool for Residents 155
C. Interview Tool for Professionals 157
D. Codebook 160
vii


LIST OF TABLES
TABLE
1. Overview of performance measures 64
2. Number of housing professionals interviewed, by level of authority and role 69
3. Residents scores on nine performance measures 71
4. Professionals scores on nine performance measures 89
5. Overview of findings 116
viii


CHAPTER I
STATEMENT OF THE PROBLEM
Engaging citizens in performance measurement efforts has not been as strategic as efforts to incorporate participatory practices in other aspects of government work, such as program planning. Engaging citizens that are marginalized or in the minority has been given even less attention, both in research and in an applied context. As public entities strive to address performance gaps, particular attention should be given to the insights provided by those who are marginalized and often excluded from public discourse and decision-making. Intentionally incorporating these diverse perspectives may allow communities to develop more comprehensive, thorough, and nuanced performance measurements that better address the varied outcomes and goals that citizens as well as public managers desire.
Individuals experiencing homelessness are often stigmatized and marginalized. Although structural factors, such as a lack of affordable housing or high unemployment rates, contribute to homelessness, blame is often placed on the individuals behaviors or choices (Belcher & DeForge, 2012). This creates a perception and norm in communities that homelessness is a choice, the result of poor decisions, and the fault of the individual who is homeless. With blame shouldered by the victims, they are seen as no longer useful or capable of contributing to a community (Belcher & DeForge, 2012). Instead, they are negatively constructed as powerless users or deviants (Schneider & Ingram, 1993). Their input to community programs and processes is rarely sought or is seen as unmerited. This failure to incorporate marginalized populations voices leaves significant gaps in our knowledge on how to best develop performance protocols.
1


Participatory Democracy as a Critical Component of Public Administration
The American state was developed to stand in stark contrast to the monarchy with which the countrys founders were most familiar. While the intention was to create a system that would serve the people rather than a single ruler, these processes were exclusively reserved for wealthy, well-positioned men. For example, the drafting of the constitution, the document serving as the cornerstone of the federal government, was closed to the public (Elkins, Ginsburg, & Blount, 2008). Subsequent ratification involved more individuals, but was still limited to a group of elite men.
Mosher (1982) labeled these early years the period of government by gentlemen given the status of those appointed to federal positions. Most of the political elites (positions comparable to todays presidential appointees and other leaders in the executive branch) had upper class backgrounds, with their fathers coming from high-ranking professions such as merchants and landowners. These gentlemen were skeptical of the capabilities of the average citizen to make decisions regarding the governance of the state. James Madison is often cited as one of the most fervent advocates for opting for a representative republic instead of a democracy with direct participation from citizens (deLeon, 1997). Madison outlined a series of checks and balances among branches of government, which were eventually adopted into the Constitution and became the framework for the current American state. This system could simultaneously limit and temper citizens untamed passions while also helping to prevent one faction from establishing themselves as the majority ruler, with no option for the minority factions to also be represented.
The inauguration of Andrew Jackson in 1829 brought a shift in how citizens were perceived within the context of the American state. This era, an era of government by the
2


common man (Mosher, 1982), brought overarching themes of pluralism and egalitarianism. While the intention was to open up governing to those with merit and skill, the resulting system was still fairly exclusive. This approach ushered in the spoils system, awarding political position to those close to elected officials. Government, during this era, was no more accessible to the common man and no more responsive to the will of the people.
Subsequent reform efforts addressed the spoils system and set the path for a professionalized civil service, but these changes failed to address how citizens were represented in the public sphere. Mosher (1982) describes two forms of representation: active and passive. Active representation refers to administrators or elected officials who intentionally advocate on behalf of those they represent. Mosher (1982) saw this form of representation as a major threat to orderly democratic government (p. 15) since the unequal distribution of power that resides in the different interest groups represented would lead to an inequitable state.
Passive representation generally refers to the make-up of administrators elected officials and whether they demographically mirror the communities they represent. It is assumed that individuals with similar backgrounds will also share values and beliefs that will be translated into policies responding to the needs of citizens from comparable backgrounds (Lim, 2006). Passive representation can be problematic given elected positions are not as accessible to certain minority or marginalized groups. Through passive representation, not all populations or interests can be represented. Portions of the American population are unaccounted for or underrepresented among policymakers, making the opportunity for passive representation to influence policy decisions unlikely.
3


Even without a perfectly representative elected body, officials from minority groups can still effect change (Lim, 2006). First, minority members may check members of the majority, bringing attention and disapproval of actions of the majority that are biased against minority groups. Majority members may respond to the potential to be checked by acting with greater restraint than if they were surrounded only by others in the majority. The adjustments of actions and behaviors that result from those in the minority checking those in the majority and the majority restraining their own biased actions can result in eventual resocialization wherein the values and beliefs of the majority shift.
These options outline an ongoing issue with the current structure for American governance, established by Madison and other federalists. The core system was established to limit participation to those who owned property, and even with a more diverse body of public officials today, the process of re-socializing individuals to adhere to the values and beliefs of those in minority or marginalized groups is tedious and lengthy. Given these issues, other strategies are needed to ensure that perspectives of the diverse American citizenry are incorporated into the development and implementation of policies and public programs. Participatory democracy, in which citizens are actively engaged in decision making, extends an individuals ability to impact public processes beyond the more passive option of aligning with a particular political party or candidate.
Participatory democracy is fundamental to public processes for multiple reasons.
First, it is seen as contributing to a more socially equitable state. The concepts of equality and equity were woven into the founding of the American state, but were not addressed in administrative practices until more recently. Grounded in the idea of a social contract and emphasizing equality among citizens of a state, the concept of social equity that is common
4


in public administration today did not gain traction until the 1960s as racial and social tensions heightened (Guy & McCandless, 2012).
The Minnowbrook Conference of 1968 and Fredericksons (1971) subsequent theoretical developments outlined in New Public Administration emphasized the need to consider equity alongside efficiency and effectiveness in assessing government processes. As participatory democracy seeks to engage perspectives of all those impacted by government processes, participation is a strategy now commonly used to enhance social equity.
Participation by citizens can also increase accountability and transparency in the public sector. Accountability for the practices and outcomes of government shifted from residing in the elected official who could be held responsible for missteps (Dubnick, 2005) to residing with bureaucrats and those responsible for implementing public processes and carrying out the work of the public sector (Barberis, 1998). This shift came in tandem with other reform efforts in the 1980s, under the umbrella of New Public Management, which promoted efficiency and incorporated private sector practices in public work (Hood, 1991). Alongside these efficiency-driven changes in administrative practices grew public mistrust in government. Engaging the public through participatory processes can enable citizens to directly experience how government work was being conducted, increasing the accessibility and transparency of public processes, and ultimately enabling the citizenry to hold public officials accountable for their actions.
In addition, strategically involving members of certain communities (either geographical or demographic) can help public officials understand the needs of that community and the potential repercussions of any policies or programs implemented. This in into the community is also beneficial if the outcomes of any decisions are not as positive
5


as intended. With members of the community agreeing to and reinforcing decisions made by elected officials or administrators by participating in decision-making processes, the results of those decisions cannot be seen as solely the responsibility of public officials. Rather, the responsibility is shared among all participants of public processes constructing and implementing the program or policy.
Participatory Democracy and Performance Regimes
Participatory democracy is a critical component of modern administrative practices, which are seen as advancing social equity, enhancing accountability, and providing access into communities not commonly represented among public officials. One component of the public sector sometimes overlooked in regard to participatory practices is performance measurement and management. While governments commonly establish performance measures and follow through on data collection, citizens are largely absent from the process (Spray Kinney, 2008; Heikkila & Roussin Isett, 2007).
Performance measurement involves collecting both outputs and outcomes from public programs, policies, and processes. Outputs are those measurements that quantify services or products and generally communicate how much has been accomplished. Examples of outputs include the number of individuals served, the number of miles of streets paved, or the number of tons of recyclables processed. Outcomes are measurements that describe the impact of a service or product. Outcomes could measure impacts such as academic growth in a classroom, recidivism rates among participants in a criminal justice intervention, or improvement in air or water quality.
Performance management involves using these outputs and outcomes to change or reinforce management practices (Hatry, 2010). The term performance management still
6


does not encompass the breadth of uses of performance information for some scholars. Moynihan, et al (2011) use the term performance regimes to refer not just to the practices of measuring and managing performance indicators but also to capture the embedded nature of these practices in almost all aspects of contemporary governance (p. i 141). This definition extends the use of performance information beyond management practices and into virtually all aspects of the public sector, including how resources are allocated, how values of a public entity are formed and expressed, and how government structures its work and relationships with outside entities. To accommodate the widespread uses of performance information in the public sector, this paper utilizes the term performance regimes when discussing the application of performance information to effect change within the public sector.
Performance regimes, in both practice and study, have gained prominence in public administration, starting primarily in the 1980s when New Public Management and its emphasis on increasing efficiency in the public sector increased in popularity and prominence (Hood, 1991). Performance regimes are tools for measuring government effectiveness, and subsequently affecting program implementation and policy design. Given the widespread use of performance regimes in theory and practice, they must develop alongside and incorporate other trends in public administration to remain useful and be in alignment with other values of the field.
With the emergence of New Public Management, much of the emphasis on social equity that had been prominent under New Public Administration was buried under attempts to increase efficiencies and incorporate private sector practices into the public sector. Rather than seen as a critical and foundational component of the administrative state, social equity
7


would only have a place in New Public Management if it were incorporated as a performance goal or indicator (Hood, 1991). The use of performance regimes persists, and incorporating participatory democracy practices into these regimes can create a system that simultaneously works to achieve the diverse values of efficiency and social equity.
At the most fundamental level, utilizing participatory democracy strategies in performance regimes can have utility because the perspectives of citizens can differ from those of government officials tasked with creating performance measurements and making decisions based on those performance measurements. Citizens can provide a context regarding other factors that may be impacting the intended outputs or outcomes a program. With the knowledge of both administrators and citizens involved, those developing and finalizing performance regimes can have more complete and nuanced information to inform decisions. The process of involving citizens comes with costs of additional time, effort, and resources spent. Without this engagement, though, administrators may be unaware of what information they are missing, limiting their ability to assess whether citizens participation yields different or improved outcomes from what would have been developed otherwise. In addition, the process of engaging citizens upholds the value of social equity critical to public administration.
Homelessness as a Marginalizing Experience
Participatory democracy enhances social equity in part because of its focus on engaging marginalized voices. Marginalization occurs when a particular grouping of people (by race, gender, social class, disability status, or another identifying quality) collectively have limited political efficacy. Political efficacy is composed of three interdependent components: a norm, a disposition, and a behavior (Easton & Dennis, 1967). As a norm,
8


political efficacy describes the assumption that within a democracy, all members will be able to participate in and influence their government. As a disposition, political efficacy reflects community members feelings about their ability to fulfill the norm of political participation and influence. As a behavior, political efficacy reflects community members actual actions to participate in and influence the public sphere, regardless of their perceived ability to act efficaciously. If any of these three components of political efficacy is diminished, a group of community members can be considered marginalized.
Depending on the group, marginalization may occur for several reasons. Political efficacy may be limited as a result of direct, intentional discrimination against the group, restricting its access to political decision-making. It may also result from stigma and fear of others differentness. This stigma can limit opportunities for participation because of the individuals perceived lack of ability to or worth of participating in public processes. Marginalization also can occur as a result of the status quo, with processes and rules that limit marginalized community members ability to participate in public processes institutionalized into standing systems.
The degree of marginalization will differ among groups and can change over time. This may occur because of changes in cultural acceptance of certain groups that allow for individuals representing those groups to be invited in to political decision-making processes. Laws and policies that push for greater equity in access to power and decision-making processes will also shift the degree of marginalization for different groups. Often, both the cultural acceptance of certain groups and the policies impacting their participation influence one another and lead to greater access to public participation.
9


One group of community members that qualifies as marginalized is individuals experiencing homelessness. Homelessness is a stigmatizing experience (Belcher & DeForge, 2012; Phelan, Link, Moore, & Stueve, 1997). Stigma is not an internally-driven perception, but rather is reflected in social processes that shift non-normative traits to negative traits (Goffman, 1963). In a society rooted in a sense of rugged individualism, someone who utilizes assistance from public or non-profit entities may be seen as inadequate, lazy, or a drain on others (Belcher & DeForge, 2012; Jo, 2013). Given these perceptions, individuals experiencing homelessness may be seen as less capable or less worthy to participate in public processes.
Stigma also impacts the individuals perception of him or herself. This can, in turn, affect peoples feelings about their ability to influence public decisions and their competence to participate in public processes. In addition, given the extraordinarily low incomes of those who are homeless, many may not have the time or resources to participate in public processes. The costs of missing work and finding transportation are prohibitive. Homelessness, including both the stigma attached to it and the difficulties navigating day-to-day functions, may limit public participation. Therefore, it affects all components of political efficacy norms, dispositions, and behaviors and individuals experiencing homelessness are marginalized in public decision-making processes.
To examine participatory practices within performance regimes, homelessness serves as a relevant case for study. The first federal efforts to incorporate the public into decisionmaking processes began with efforts to engage communities that were marginalized given their economic status (a component of the marginalization of homelessness). Engaging community members with homeless experiences in studies of participatory practices builds
10


on research and theory that was developed in communities with a similar basis of marginalization.
Homelessness has also been given greater attention in recent years at federal, state, and local levels. It is a timely issue to study and the outcomes of this research have the potential to impact practices in homelessness programs and policies. In 2010, the Obama administration released Opening Doors (USICH, 2010), the first federal plan to end and prevent homelessness. This plan advocates for research-based solutions and guides federal funding to align with those strategies. Permanent supportive housing is included in the strategies outlined in Opening Doors. This type of housing is designed for individuals for whom significant barriers exist to access housing on their own. These may include disabilities or an extensive history of homelessness. Permanent supportive housing provides time-unlimited housing support, coupled with supportive services that are often offered onsite. As described in the following section, a permanent supportive housing program in Fort Collins, Colorado, funded in part by federal dollars specified in Opening Doors, serves as the setting for this research.
Homelessness in Fort Collins, Colorado
Fort Collins, Colorado sits in the foothills of the Rocky Mountains in Northern Colorado, approximately one hour north of Denver. Serving as the county seat of Larimer County, the town has a total population of approximately 164,000, excluding the student population of Colorado State University, situated in the center of Fort Collins. The downtown area, or Old Town, extending slightly north is home to many non-profit homeless service providers including two overnight shelters (each provide daily meals), three day shelters, the food bank, Larimer County Human Services, the community mental health
11


center, a Federally Qualified Health Center, and others. Homelessness in Fort Collins is often most visible in Old Town.
In 2006, as conversations of development and preservation in Old Town and along the Poudre River corridor just north of Old Town began to take shape, local homeless advocate Sister Mary Alice Murphy called on a small group of local citizens to consider the needs of people experiencing homelessness when crafting plans for the Old Town area. In 2010, after four years work to better identify the current status of homelessness in Fort Collins, to research best practices, and to understand federal and state trends in ending homelessness, Fort Collins launched a local 10 Year Plan to Make Homelessness Rare, Short-Lived, and Non-Recurring. The Plan aligned closely with the strategies outlined in Opening Doors, released that same year. Homeward 2020, a community-based non-profit initiative, was formed alongside the Plan to drive its implementation.
Leaders from Fort Collins Plan also participated in the development of Pathways Home Colorado, a statewide initiative to ensure all Coloradoans have a place to call home (Pathways Home, 2012). In a few years time, efforts and strategies for addressing homelessness at the local, state, and federal levels began to better align, with overall goals focused on reducing homelessness permanently through housing rather than managing the effects of homelessness through temporary shelter and services.
To remain in compliance with federal funding requirements and to better understand the current state of homelessness, Fort Collins administers an annual Point-in-Time Survey that counts the number of people experiencing homelessness on a single night in January. Based on the 2017 Point-in-Time Survey conducted on the night of January 24, 2017, at least 331 individuals experience literal homelessness on a single night in Fort Collins. Those
12


canvassed as part of the Point-in-Time survey are only those staying in emergency shelters, in transitional housing, or outside in unsheltered areas including sleeping in cars, sleeping in sheds or abandoned buildings, or camping. Around 77% of all respondents stayed in a shelter the night of the survey. Men comprised 65% of survey respondents. Around 81% of respondents were single adults. Another 11% of respondents were unaccompanied children or youth under the age of 25. The remaining 9% of respondents were families with at least one child under the age of 18.
Almost half of respondents self-identified as having a disabling condition, including a mental health disorder, substance abuse disorder, traumatic brain injury, developmental disability, or a chronic physical illness or disability. In addition, 28% of respondents qualify as chronically homeless, meaning they had been homeless for a year or more or had four or more episodes of homelessness within the past three years, and had a long-term disabling condition. Individuals and families experiencing chronic homelessness face some of the highest barriers to being able to access and to retain housing. It is generally recommended that chronic homelessness is solved through permanent supportive housing (USICH, 2010).
In March of 2015, new housing opened in Fort Collins to provide a permanent residence for individuals with disabilities who are also experiencing homelessness. Redtail Ponds is a 60-unit apartment complex in south Fort Collins, developed by Housing Catalyst, the public housing authority for Fort Collins, using a combination of low-income housing tax credits, federal, state, and local funds, and private donations. Forty of the 60 units are dedicated as permanent supportive housing with the remaining 20 serving as general affordable housing. Ongoing operations are supported by a grant from the U.S. Department of Housing and Urban Development (HUD), renewed annually. Funding for this grant comes
13


through the Colorado Balance of State Continuum of Care, a 56-county region that collectively applies to HUD for funding to support housing and services for people experiencing homelessness in Colorado.
To qualify to live at Redtail Ponds, residents must be experiencing literal homelessness as defined by HUD. This means that they must meet the same qualifications as those surveyed by the Point-in-Time Survey (i.e., staying outside, in emergency shelters, or in transitional housing). In addition, to live in the permanent supportive housing units at Redtail Ponds, residents must have an income that is 30% or less of area median income and have a disability. Qualifying disabilities include physical disabilities, chronic physical illnesses, mental health disorders, substance use disorders, brain injuries, or post traumatic stress disorder. Although not a requirement for residence, most residents also qualify as chronically homeless.
All residents at Redtail Ponds are required to pay 30% of their monthly income (whether acquired through employment or cash benefits, such as disability or veteran benefits) towards rent. Each resident is provided with a private one-bedroom apartment (or two-bedroom for those requiring additional space for medical equipment, live-in care, or partner), multiple community spaces, the security and support of 24/7 staff coverage, and access to on-site supportive services including behavioral health care, group sessions, social activities, and case management.
Research Question
Participatory democracy is grounded in the idea of providing opportunities to those commonly excluded from political decision-making, with limited efficacy and access to decision-making processes. This research builds on previous studies of participatory
14


democracy that similarly examine participatory practices as one option for enhancing social equity. It expands the literature by examining a component of public administration that infrequently utilizes citizen participation: performance regimes. The case of supportive housing for people moving out of homelessness in Fort Collins, Colorado provides an opportunity to conduct research into how marginalized voices can impact performance protocols.
To begin to understand this impact and to contribute to the literature on citizen participation and performance regimes, this study focuses on the following research question: How do preferences for performance measures differ between public sector representatives at the federal, state, and local levels, and the marginalized public served?
Exploring this research question matters for three primary reasons. First, if the preferences of individuals who are marginalized, with less political efficacy, differ from administrators and the general public but are not incorporated, the resulting performance regimes lack the breadth of perspective to fully gauge, report, and utilize performance information in a complete and nuanced manner. Second, the perspectives of marginalized communities are critical to further enhance social equity in the public sector. While relevant in any public work, engaging individuals from marginalized communities when it relates to policies and programs directly serving that population better ensures the program is not inequitably designed, implemented, and assessed for effectiveness without consideration of the viewpoints of those served. Third, examining the differences in preferences for performance regimes between those representing the public sector can contribute to a more robust understanding of where, when, and in what ways preferences align with the marginalized public served.
15


This study offers both theoretical and applied contributions by expanding the minimal research conducted on citizen input to performance regimes. It addresses the specific gap of how input from marginalized populations impacts the development of performance measures and performance regimes by examining marginalized citizens preferences for different performance measures. This research will also contribute to applied practices around performance regimes, particularly in permanent supportive housing programs. It provides initial evidence on what residents in permanent supportive housing programs deem most important in measuring the success of supportive housing programs, as well as how residents interpret commonly-used performance measures. Continued research and modifications to applied practices that evolve from this line of research can ultimately inform more robust and inclusive performance regimes in supportive housing programs and other programs serving marginalized populations.
16


CHAPTER II
LITERATURE REVIEW
This chapter reviews the following three streams of literature: participatory democracy, performance regimes, and homelessness and housing policy. Research questions are identified at the intersection of these streams.
Participatory Democracy and Citizen Engagement Citizen participation is a key democratic ideal of the American public sector (Day, 1997). In todays public sector, the concept of citizen participation allows for a substantive role that extends beyond that of a passive voter. Opportunities exist for citizens to serve as collaborators and experts in developing public strategies that will respond effectively to their needs.
Engaging the public gives legitimacy to public action and allows for decisions to be made beyond a small group of elite policymakers. This provides an avenue to enhance social equity and to promote a sense of justice, reinforcing the concept of public action as a tactic for building a strong, democratic state (Fung, 2006). With expected outcomes and benefits for both the individual participants and the public sector, participatory practices have been used across jurisdictional levels and for a variety of purposes since the first large-scale engagement efforts were implemented in the 20th century.
A Brief History of Citizen Participation in the Public Sector
Until recently, citizens informed public decisions most frequently through non-engaged methods of voting and party involvement. A model of public participation relying on discourse and with more equitable opportunities for non-policymakers to actively participate in decision making is a recent development (Webler & Tuler, 2000).
17


Demonstration projects implemented in the mid-20th century seeking to alleviate poverty and improve educational outcomes in poor schools sought to engage local residents as leaders for community action (Rubin, 1969). These projects provided a foundation for modern public participation efforts.
The first widespread efforts to insert participatory practices into public programs occurred with the Housing Act of 1954, in which communities were required or strongly encouraged to involve citizens affected by the urban renewal projects funded by the Act. In this case, citizen participation was sought less to ensure equity in public processes and more to co-opt community members so they would not object to the projects once development and rehabilitation began (Johnstone, 1958).
In the late 1950s, around the same time as the Housing Act of 1954 was being implemented, Mobilization for Youth, Inc. was founded with the intention of improving issues of juvenile delinquency in the Lower East Side of New York (Moynihan, 1970). This effort, spearheaded by private philanthropists and notable academics with community members at the helm, conceived of community change occurring through community action. In 1961, a request for funding from the National Institute of Mental Health outlined the structure of this community-based program. The details of the program, including the use of community action and language used to describe it aligned with the Economic Opportunity Act of 1964 presented two years later (Moynihan, 1970).
The 1964 Economic Opportunity Act established the Community Action Program as a strategy in President Johnsons War on Poverty. The legislation mandated maximum feasible participation among local communities, specifically among those citizens living in poverty and frequently politically isolated, with the intention of building community and
18


improving service delivery through better coordination (Adler, 2012). The clause, given almost no attention in legislative discussions, was broadly criticized. Some insisted it would encourage revolution, local officials claimed it impinged on their authority, and welfare workers asserted that people living in poverty would not participate as collaborative decision makers (Rubin, 1969).
Although broad community participation was encouraged, individual organizations that were already prepared to mobilize and serve within the community could seek federal funding (Adler, 2012). With funding secured, there was little incentive to work through conflict and intentionally engage stakeholders with divergent backgrounds and perspectives in order to achieve maximum feasible participation. As Adler (2012) summarizes, Given the social, political, and racial tensions of the sixties, in addition to the animosity many civil rights groups and the poor felt toward municipal political machines, in many cities around the country.. .consensus was more of an ideal goal than a political possibility (p. 553).
At the same time these federal legislative efforts and demonstration projects were developed and implemented, the field of public administration was responding to citizens changing attitudes toward government in attempts to remain relevant. Amid a sense of public unrest, scholars in the 1960s began highlighting the role of democracy and the importance of social equity in the public sector (Kim, OLeary, Van Slyke, Frederickson, & Lambright, 2010). These themes were core to the first Minnowbrook Conference in 1968, in which scholars met to discuss, debate, and wrangle with the complex social issues confronting the public sector at the time.
New Public Administration (NPA) arose out of the Minnowbrook Conference and the on-going debates on how to resolve the disconnect between administration and democracy
19


(OToole, 1977). Frederickson (1971) argued that administration had become more about the
institution of government than problems faced by the public; NPA could refocus the field on
resolving those complex social problems. To do this, the field would need to re-examine
those values to which it held itself accountable. Rather than being dominated by the
bureaucratic values of efficiency and productivity, Frederickson (1976) noted that public
administration needed to adhere to and be dominated by another set of values:
.. .a concept of new public administration would have to begin with the argument that a different (and certainly not new) set of values should predominate. These values would be humanistic and would be realized in decentralized, democratic organizations which distribute public service equitably. New public administration, therefore, would be the attempt to organize, describe, design, or make operative organizations that further these norms (p. 167).
Creating opportunities for involvement in public decision-making processes with the
intention of having stronger outcomes as a result, as the Economic Opportunity Act sought to
do, reflects this shift in values.
In administrative reform movements that have developed since NPA, the role of the citizen has maintained a strong presence while shifting in the expectations of what that role should involve. Beginning in the 1980s and 1990s, as practitioners and scholars began to look toward the private market for models of how the public sector could be more effective, efficient, and economical, the citizen took on the role of the governments customer (Thomas, 2012). For example, the Reinventing Government movement, sparked by Osborne and Gaeblers (1992) book by the same name, emphasizes competition and customer service, both of which are common themes in the private sector.
New Public Management (NPM), sharing enough tenets with the reinvention movement that they are sometimes treated interchangeably, similarly emphasizes a market-approach to public service delivery (Hood, 1991). NPM relies on the principle that if
20


government competes with the private market to provide certain services, it must remain responsive to the needs of its citizen-customers. Considering citizens as customers reduces the likelihood that a government can go about its work adhering to the rules and processes with less attention to the effects on the public.
Considering citizens as customers has been questioned and criticized for several reasons. First, the choices that arise when entities compete to provide public services may not have a profound effect on citizens choice for services or goods. Instead, more choices are provided to public administrators as they choose between potential providers to serve the citizens. While the process pulls from private sector practices, the resulting scenario involves contracting out for services rather than citizens selecting from among multiple choices of public services or goods they prefer (Thompson & Riccuci, 1998).
Second, citizens are encouraged to make those decisions that are best for themselves or their family, responding as they would in a competitive market as a rational human being. As Frederickson (1996) notes, [t]he values of individual satisfaction are judged to be more important than the values of achieving collective democratic responsiveness (p. 265). This tendency undermines the democratic ideals of the public sector and does not account for the irrationalities expressed in human decision-making and human behavior.
Finally, this approach relies primarily on the possibility for customers to leave a particular organization if it is not performing well. While this may be effective motivation to improve services or products in the private sector, citizens cannot easily exit participation in their government. If a citizen is discontent with the national defense strategy and its implementation, there is no way for that citizen to opt out of receiving protection from national defense agencies. Government relies on a process of citizens claiming ownership
21


and working to change and improve its functionality in order to improve performance (Hirschman, 1970).
Collaborative governance, another concept for understanding administrative functioning that emerged in the 1990s, is also grounded in the ideal of participation of multiple stakeholders, which can include citizens. Ansell and Gash (2008) define collaborative governance as [a] governing arrangement where one or more public agencies directly engage non-state stakeholders in a collective decision-making process that is formal, consensus-oriented, and deliberative and that aims to make or implement public policy or manage programs or assets (p. 544). Others argue that collaborative governance practices can extend beyond those formal interactions between state and non-state partners, and involve other arrangements such as public-private partnerships (Emerson, Nabatchi, & Balogh, 2011).
Now, citizens are viewed as not only collaboratively participating in the public sector, but also in partnering with government to co-produce services (Thomas, 2012). For example, producing public education requires students to show up and learn and parents to prepare their children for school; producing public waste services requires residents to take their trash to the curb for collectors to pick up; and, producing the public safety services of the police requires citizens to report potential criminal activity.
In addition to co-producing these and other services, the public also has a role in producing public values (Rosenbloom & Gong, 2013; Thomas, 2012). Citizens choose to uphold or challenge certain policies, rules, and programs through their actions, reinforcing or redefining the communitys conception of what is important. For example, a citizen calling and reporting criminal activity is co-producing public safety services and is also co-
22


producing the value of public safety (Rosenbloom & Gong, 2013; Thomas, 2012). Citizens in this role reinforce their desire for a safe community and express which behaviors are acceptable or not. For instance, citizens may be more inclined to report a robbery, which is seen as threatening to the safety of others, than to report someone jaywalking. The action of reporting crime and the discretion of what crimes will be commonly reported define the value of public safety through citizen co-production.
From co-production of services and values to engaged participation in public decision-making processes to passive participation via voting and partisan identification, the concept of citizen participation in government has seen multiple iterations. Several public administration and public management theories and frameworks have incorporated public participation as a key concept for an effective government. While in theory, citizens should and do have multiple opportunities for participation, engaging the public in meaningful decision-making processes in practice can have costs and can prove difficult for practitioners. The Rationale Behind Citizen Engagement
Although the founding fathers wanted to establish a government directed and controlled by the people, the state was never intended to allow for a widespread, active citizenry. Common citizens were seen as incapable of making the right choice, according to political leaders. James Madison, in particular, is noted for advocating for a state that could temper the irrational or hasty interests of the common citizens by limiting their direct involvement and input in public policy (deLeon, 1997). As a result of these initial designs, our public sector has unfolded as one that seeks to involve citizens in politics while simultaneously not allowing for too much active public participation (King, Feltey, & Susel,
23


1998). This foundational paradox illuminates why participatory strategies can be troublesome and difficult to implement.
Most see an engaged public as being more positive than one that is passive (King, Feltey, & Susel, 1998; Putnam, 2000). In addition to enhancing social equity in public processes, public entities engage citizens for many other reasons. Citizen participation is often cited as encouraging collaboration between the community, policymakers, and administrators. This results in more effective and higher quality policies and programs that are more responsive to citizen needs (Bryson, Quick, Slotterback, & Crosby, 2012; Newman, Barnes, Sullivan, & Knops, 2004). Engaging citizens allows for citizens and policymakers to learn from and have the opportunity to inform the other (Bryson et al, 2012; Irvin & Stansbury, 2004). For government partners, this educational process may help mollify fears or hesitations in the community around a proposed policy. Particularly if a policy or program is controversial, a lack of communication may be perceived as a disingenuous attempt to make unfavorable or damaging changes. By being informed early on of a proposed change, citizens may instead perceive the government as being more transparent and more accountable to the residents of that community (Leroux, 2009). In addition, public officials may also see public participation as an opportunity to educate citizens on the limitations of government, enforcing why the bureaucracy works as it does and why certain changes cannot happen (Newman, Barnes, Sullivan, & Knops, 2004).
Attempts to engage citizens are not equally effective in every community and for every issue. Generally, when the community or the multiple groups of stakeholders are diverse, consensus is difficult to reach, even with extensive community deliberation (Irvin & Stansbury, 2004). Rather than achieving a consensus, the result may instead be a decision
24


reflective of only a few peoples ideas. Stakeholders may leave the process feeling ignored rather than engaged.
The decision-making process also becomes lengthier and more costly with greater citizen input, which leaves less funds for implementation. If the issue is not one that is likely to draw much controversy or potential litigation, extensive community engagement may be more detrimental than effective (Irvin & Stansbury, 2004). In addition, if the input provided by citizens is not incorporated into the final decisions, the public may develop greater resentment and distrust of the government (Irvin & Stansbury, 2004). While the rationale for engaging citizens in public processes is strong, in some cases it is not necessary or helpful given the costs of greater and more diverse public participation.
Considerations in Engaging the Public in Decision-Making
Public participation is not a singular concept but rather involves three elements: the scope of participation, the methods of interaction and communication, and the level of authority given to citizen participants (Fung, 2006). The role of the citizen and the methods used to engage citizens will vary depending on the intended outcomes of the project. Other factors, including the timeline, funding available, intended participants, and requirements from any funding or regulatory agencies responsible for the program, will also influence the methods used. Common forms of public participation include public comment periods, advisory committees, feedback surveys, and various forms of public deliberation (Leroux, 2009; Thomas, 2012). To accommodate multiple outcomes and work through multiple barriers, successful citizen participation strategies may utilize several methods to engage citizen stakeholders (Yang & Pandey, 2011).
25


The scope of participation in participatory democracy processes is related to the mode of selection for citizen participants. This dimension encompasses who is involved in the public process and how they are selected (Fung, 2006). Some projects may call for open participation from the designated community and participants self-select. Depending upon the target community, this can yield a large and diverse participant body, with varying interests and perspectives. Although inclusivity may be an objective of engaging the public, this make-up may cause difficulty when attempting to reach consensus on a topic. Collaborative decision-making has benefits, but it is most successful in small, homogeneous groups rather than large, diverse communities (Ostrom, 1990).
One limitation of a self-selection process is that while some select in, some will select out. Rarely does a self-selection process yield an equitable and representative group of participants. For projects needing the input and participation of specific groups or specific individuals within a community, targeted recruitment may yield a more exclusive, but higher functioning group to tackle the designated project. The desired participants may be engaged through strategic recruitment or by representing organized public interest groups (Thomas, 2012).
Even if a specific groupparticularly a marginalized or minority group of the populationis invited to participate, the perspectives they offer may not be held in the same regard as the majority of the community. Public officials generally see citizens as capable of contributing to public processes, but as Fung (2006) notes, some populations may not be perceived in the same manner. For example, those who receive welfare benefits wield little power and influence and are thought of more negatively (Schneider & Ingram, 1993). These constructions may affect the interactions within decision-making bodies or affect the ways in
26


which policymakers choose to use the information gathered from those citizen groups. Additional research is needed to fully understand the manner and degree to which this may happen.
In addition to the influence that social constructions play in citizen participation, very real barriers will also exist for some communities. Language barriers, lack of access to public processes (because of lack of transportation, childcare, or technology), or a general discomfort or feeling of exclusion, can all hinder the ability of people to participate in public processes. Although appearing neutral and allowing for equal participation, public processes will favor those who have the skills, knowledge, and resources to participate in bureaucratic processes. Without these skills, knowledge, and resources, political efficacy is limited. Public participation will be as much about acclimating these individuals to existing processes as to examining how the public processes may better adapt to suit the needs of a diverse public (Newman et al, 2004).
The methods of communication and decision-making to allow for public participation also vary. Participation in public processes can range from fairly passive, with community members serving as little more than spectators, to more active and engaged, during which citizens participate in developing a true, collaborative outcome (Fung, 2006).
In-person forums with smaller groups may provide the level of interaction that allows for quality citizen participation (Thomas, 2012). Officials can strategically invite members of specific populations, which allows for a more diverse representation than a self-selected process. This allows for a more collaborative process in which citizens and government officials learn from one another. Though the perspectives of the multiple stakeholders and participants involved may not initially align, face-to-face discussion allows for a process of
27


bargaining, deliberation, and negotiation to arrive at a final outcome that more effectively incorporates the viewpoints of participants (Fung, 2006).
Online forums, posting regulations and information on websites, and other technology-based methods allow for greater participation in public processes, which may or may not be beneficial. In an analysis of online comments submitted to regulations.gov, Bryer (2013) found the majority of comments were based on emotion, and commenters largely failed to establish any credibility in having informed knowledge of the subject matter at hand. Technology increases the volume of citizen participation, but the majority of participation is low quality, providing policymakers and administrators with little useable and credible information (Bryer, 2013).
Special Considerations for Marginalized Populations
The public is often defined in one of two ways: as the white, privileged, and able-bodied group or as the minority, disadvantaged group (Newman, Barnes, Sullivan, & Knops, 2004). Each of these groups is socially constructed (Schneider & Ingram, 1993; Barnes, Newman, Knops, & Sullivan, 2003), impacting their perceived ability to participate in public processes. The perception of whether or not a group is capable of effective citizen participation limits their political efficacy, marginalizing that population in the community.
Engaging minority, disadvantaged, or marginalized groups is seen as beneficial for the opportunity to educate or empower participants (Newman, Barnes, Sullivan, & Knops, 2004). With limited political efficacy, disseminating information can enhance citizen competence on the issue at hand or on ways individuals can participate in and influence the political process. Increasing individuals knowledge and advocacy skills improves the
28


outcomes of those engagement strategies as individuals learn how to participate in public processes given the established forums (Yang & Pandey, 2011; Leroux, 2009).
Improving knowledge and skills so that marginalized individuals can participate in established public processes allows for more equitable outcomes, though this process is still less than ideal. Approaching citizen engagement among marginalized communities with only the intention of assimilating those individuals to existing processes reinforces the power differential already in place. These goals for citizen engagement emphasize public processes that exclude marginalized portions of the public (Barnes et al, 2003; Newman et al, 2004). As participatory processes are generally meant to build consensus and represent the public broadly, relying on processes known to exclude portions of the population only highlights the differences between participating and non-participating community members (Barnes, Newman, & Sullivan, 2006). In addition to enhancing political efficacy through building skills and knowledge of marginalized communities, public officials can also examine altering public participation processes to better accommodate those individuals.
There is no singular process or set of methods established as best practice for engaging marginalized populations. Communities or organizations develop engagement strategies based on participant profile, purpose of engagement, resources available, or the history between the engaged community and the engaging organization (Bryson, Quick, Slotterback, & Crosby, 2012; Thomson, 2012). Bryson, Quick, Slotterback, & Crosby (2012) identified guidelines for public participation, constituting an iterative process that encourages evaluation and redesign of the participation process. An effective participation process, including one that engages marginalized communities, should be designed to (a) fit the problem at hand and the purpose of engagement; (b) foster effective leadership from the
29


engaged community; (c) engage diverse participation through inclusive processes; (d) use multiple technologies to heighten participation; (e) manage existing power differentials; and, (f) continue to generate resources for participation processes (Bryson, Quick, Slotterback, & Crosby, 2012).
When marginalized communities are successfully engaged in public processes, the effects are varied. Different perspectives are offered, which may impact the design or implementation of the program or policy discussed. Participation also impacts the individual. Gupte (2010) notes that public participation reinforces ones identity as a member of a certain group. With time, this identification strengthens that community and its advocacy power as more individuals develop closer ties to others. Stronger and better organized, that group is better positioned to advocate for changes needed to improve their community and generally promote equity.
Public participation of marginalized communities may not only have an immediate impact on the public process at hand, but also can have rippling effects. Citizen participation creates a different conversation, framing the needs of the community and the responses by the state in a way different than what would occur with only elected officials or administrators participating in decision-making. The interactions between citizens and public institutions serve to reshape those institutions and community values, influencing later responses from the public sector as well as influencing how citizens react to and interact with the public sector (Stivers, 1990). The outcomes of citizen participation, then, may have a more long-lasting effect than what is immediately noticeable.
Including marginalized groups allows for a more diverse and robust dialogue that will shape not only policy, but also the long-standing values and institutions in the community
30


(Gupte, 2010). People who have recently experienced homelessness are often marginalized, and their voices are frequently missing from public processes. They are even less frequently engaged in developing performance protocols given the limited use of participatory practices in general when establishing performance measurement and reporting processes. To improve performance processes and promote democratic participation, people experiencing homelessness have a role to play in developing performance regimes.
Performance Measurements and Performance Regimes
Measuring outcomes in the public sector began in the 1960s with a federal effort to connect program effectiveness to budget items (Newcomer, Forthcoming). This concept of effectiveness evolved into a focus on results and outcomes eventually developing into an emphasis on performance measurement and performance management. Performance measurement generally describes collecting performance information and data while performance management requires the use of that performance information to make management decisions. Performance information may have even broader uses, influencing many aspects of the public sector. For this reason, Moynihan, et al (2011) coined the term performance regimes to capture the extensive impact that performance information has on current government practices.
Performance management and performance regimes emerged in the 1980s, incorporated into the doctrines of New Public Management (Van Dooren, 2011). Using a market-driven approach to governing, New Public Management advocates for the use of measurements to determine what is being accomplished and how well. This information can be used to make budgeting decisions and ultimately create a more efficient public sector.
31


While used in some situations prior, the emphasis on performance measurement gained a prominent position in federal administrative processes beginning in 1993 with the passage of the Government Performance and Results Act (GPRA) (Lavertu & Moynihan, 2012). GPRA was grounded in the findings that a lack of clearly stated goals and related performance information diminished public trust for government, limited administrators in implementing effective and efficient public programs, and impeded policymaking and program oversight by legislative bodies (GPRA, 1993). By implementing performance regimes that included setting goals, developing performance indicators, identifying data collection methods, and creating opportunities for administrators and policymakers to utilize this information, public programs could function more effectively and could enhance public trust.
Since GPRA was initially implemented, researchers have developed a more comprehensive understanding of the uses of performance indicators and performance regimes beyond improving the effectiveness and efficiency of service delivery in public sector. First, as anticipated by the stated purposes of the GPRA, performance regimes enhance accountability and transparency of the public sector (Moynihan, 2008; Grosso & Van Ryzin, 2011). The public distrust arising in the 1960s fueled the desire for the government to be able to show what it was doing and how effective it was at its work. By aligning programs with outputs and outcomes, measuring them, and communicating results, the public ideally has more knowledge of how their government works and how their public dollars are spent. Performance information is also used to celebrate and promote accomplishments of the agency, and to motivate staff and stakeholders involved in the governments work (Behn, 2003).
32


While performance measurements can be beneficial if used appropriately, the reliance on performance information may encourage some managers to do the minimum work required to collect the required performance information and disseminate the findings (Moynihan, 2009). In such cases, performance information is rarely used to modify and improve any management practices or program activities. Rather, collecting performance indicators is viewed as little more than a mandate to comply with policies or program rules.
The emphasis on performance indicators and the pressure to perform may also lead some managers to manipulate the program or the collection of data to demonstrate outcomes. Concentrating on those aspects of the agency that are measured may lead to a perceived improvement in performance, but if other aspects of the program are correspondingly deteriorating, no actual program improvement can be expected (Van Thiel & Leeuw, 2002). These factors all result in a tendency for performance indicators to lose their power and ability over time to differentiate strong performers from weak performers (Meyer & Gupta, 1994). To ensure that performance information is used effectively, these possible shortcomings should be considered in the design of public performance regimes.
Citizen Participation and Performance Regimes
The trends of citizen participation and performance regimes have generally been dealt with separately, both in the literature and in practice. Public officials usually pull citizens into public processes simply to inform them. More active participation by citizens, in which they are involved in collaborative decision-making processes, is rare (Yetano, Royo, & Acerete, 2010). Even in communities with strong efforts to engage citizens, the public participates more frequently in planning efforts (Epstein, Wray, & Harding, 2006). While these impact the subsequent establishment of performance regimes, citizens preferences on goals,
33


outcomes, and measurements, or their knowledge of other factors that may impact performance goals, are often not reflected in final evaluation efforts.
Engaging citizens in processes to define, measure, and report outcomes is critical to establishing performance regimes that will benefit the public sector and its constituents. First, engaging citizens helps ensure the goals of the public sector are aligned with citizen preferences (Spray Kinney, 2008), potentially affecting what programs or services are prioritized, funded, and implemented to meet those citizen preferences. In addition, including citizens when establishing performance regimes can increase support for government action among citizens, increase social capital in a community, and increase the capacity of the public sector to deal with complex problems (Heikkila & Roussin Isett, 2007). Through citizen engagement, ties among community members and between citizens and their government can strengthen, all while establishing performance regimes that measure and produce outcomes that the citizenry values (Cohn Berman, 2008; Epstein, Wray, & Harding, 2006).
Citizen engagement can also improve the transparency and accountability of the public sector, one of the primary purposes of establishing performance regimes (Moynihan, 2008; Grosso & Van Ryzin, 2011). Even if actual practices or programs do not change as the result of engaging the public in performance regimes, educating and involving citizens in the process of measuring outcomes can lead to a perception of greater trust and accountability (Halachmi & Holzer, 2010). When perceived transparency, accountability, and trust increase, the public sector and its work is also perceived as more legitimate (Woolum, 2011).
Although perceived favorably and as a missing component of larger citizen engagement efforts, few public agencies and communities have strategically or effectively
34


involved citizens in performance efforts. Research from the Government Finance Officers Association found that fewer than five percent of communities engaged citizens when developing performance measures (Spray Kinney, 2008). Similarly, an analysis of special districts revealed that very few districts had any citizen input in developing performance measures (Heikkila & Roussin Isett, 2007).
The lack of citizen voice in performance measures is often reflected in evaluation reports published by public agencies. Government reports do not regularly address those services or attributes of an agency that citizens deem most critical (Cohn Berman, 2008;
Cohn Berman, 2012). While public agencies commonly report outputs, these measures mean little to citizens because they have no basis for comparison with other communities. Citizens prefer to focus on outcomes, as well as measures of the quality of service delivery, such as responsiveness and accessibility (Cohn Berman, 2008; Cohn Berman, 2012; Woolum, 2011). The disconnect between which indicators governments and citizens each deem relevant to measure and report can lead to a poor perception of government and the quality of its work, and general confusion and frustration among both citizens and public officials (Cohn Berman, 2012).
Citizens may also have different preferences with how information is communicated (Heikkila & Roussin Isett, 2007), opting for simple, honest, and complete accounts of how their government is performing and how their community compares to others (Cohn Berman, 2008). Citizens prefer having all information accessible, rather than only those measures that highlight positive outcomes, allowing them to make assessments about the efficacy of the work conducted (Cohn Berman, 2008).
35


Technological advances allow governments to provide easier access to performance information to citizens via websites, multiple social media channels, or emails. With these options available, government officials must weigh the advantages and costs of using various formats for information dissemination. For instance, publishing performance information online is less costly and allows for more data to be published, as compared to printed data reports. Despite these potential benefits, Des Moines, Iowa opted to mail their final performance report to everyone in the city with the assumption that more citizens would receive the information if sent directly to them (Matthes, 2008).
Technology not only makes disseminating aggregated performance information easier and less expensive, but it is also effective in providing more data, updated in real-time or more frequently than printed evaluation reports. For example, in the City of Chicago, the datasets on different community indicators are made available to citizens, allowing them to examine those data that are of most interest to them rather than the limited data that a public agency may determine is important to communicate (Spray Kinney, 2008). Citizens in Chicago can access information on everything from the number of WiFi sessions used per month at public library branches to the number of days taken to issue a building permit to the percentage of homeless shelter beds used each month (City of Chicago, 2014).
Although having thousands of indicators available for citizens to explore may have utility, understanding data is often complicated and complex. Without a mutual understanding between both citizens and public employees of what constitutes success, the conclusions drawn from data may still not lead to a greater understanding of how effectively the government is responding to a particular issue. In addition to providing the actual data,
36


effective performance regimes that are responsive to citizens needs also provide explanations and a context for understanding the numbers (Cohn Berman, 2012).
Multiple reasons have been cited for not actively engaging citizens in performance processes. First, managers may question citizens abilities to make informed, qualified decisions regarding performance measures and changes in management practices (Heikkila & Roussin Isett, 2007). Particularly those areas requiring specialized or technical expertise are difficult for most citizens to understand (Cohn Berman, 2008). This lack of knowledge may negatively impact citizens abilities to effectively participate in performance-related activities.
Public managers also perceive that citizens generally are not interested in being involved in many engagement efforts (Heikkila & Roussin Isett, 2007; Yetano, Royo, & Acerete, 2010). Particularly at the most active levels of participation, public agencies find it difficult to engage citizens and retain their participation throughout the performance regime process. Lack of resources, lack of political will, and a general resistance to change also impact efforts to engage citizens in performance regimes (Yetano, Royo, & Acerete, 2010). These concerns are valid, as even communities that have strategically changed their performance processes to allow for more robust citizen engagement have had to work through the challenges of involving citizens, gaining acceptance among public managers, finding the time and resources required, and collecting accurate and reliable data (Cohn Berman, 2012).
Findings from Current Research on Citizen Engagement in Performance Regimes
Although still uncommon, some local governments have made significant attempts to engage citizens in establishing performance regimes. The Center on Government
37


Performance at the National Center for Civic Innovation established a Trailblazers program in 2003 (with additional cohorts of communities participating in later years) as part of an effort to encourage and support government management practices that inform and are informed by the public (National Center for Civic Innovation, 2014).
Communities accepted into the Trailblazers program were already collecting data, primarily consisting of costs, revenues, workloads, and outputs. Public employees and managers typically used the information; reports compiled were generally not constructed for public dissemination or consumption. Trailblazer communities were asked to engage citizens in performance processes, using the book Listening to the Public: Adding the Voices of the People to Government Performance Measurement and Reporting (Cohn Berman, 2005) as a guide.
Trailblazer communities used a variety of methods to engage the public, with the most common approaches being surveys and focus groups. Some communities also utilized community conversations and citizen committees. As a result of these engagement efforts, Trailblazer communities tended to share two common observations: 1) the public actually is interested in their government, wants information on what is happening within the government, and wants to be involved in government efforts; and, 2) public agencies assumptions about what citizens want and need from their government are often incorrect (Cohn Berman, 2012).
After soliciting information from citizens, Trailblazer communities implemented changes to improve their performance regimes. Across all participating communities, five broad categories of change occurred. First, Trailblazer communities changed their practices by adopting new measures that better aligned with citizens preferences (Cohn Berman,
38


2012). Instead of relying solely on the output-related measures previously used, communities utilized outcomes and measures of service quality and responsiveness.
Second, performance reporting improved, with reports becoming more accessible to the public (Cohn Berman, 2012). For some communities, this process led them to produce a performance report for the first time. Other communities modified their reports to be more user-friendly for the public, including relying on graphs and charts to relay data, providing both quick summaries as well as more in-depth information, and including information to provide a context to many of the data. Communities also changed how information was disseminated back into the community. In addition to publishing more comprehensive reports, Trailblazer communities reported posting data and reports online and printing shorter performance summaries in brochures or report cards.
A third major change seen as a result of the Trailblazer program was the adoption of legislation by participating governments requiring performance reporting to the citizenry (Cohn Berman, 2012). This formalizes and institutionalizes the need for effective performance regimes that engage citizens in decision making, helping to ensure these practices are sustained after the initial champions of the Trailblazer program leave their posts.
Fourth, public managers and elected officials came to rely on the performance data to a greater degree than before (Cohn Berman, 2012). For many agency heads or managers, performance data became a valuable tool for evaluating current programs and making subsequent changes to improve those programs. Elected officials utilized performance data when making funding decisions or other programmatic decisions.
39


Finally, as a result of the Trailblazer program, governments reported their performance improved (Cohn Berman, 2012). Participating communities reported employees beginning to rely on data to look for opportunities for improvement and implement changes to enhance performance. All of these effects demonstrate the potential for public agencies to ultimately serve the public more effectively when citizens ideas and preferences inform the established performance regimes.
Special Considerations for Marginalized Populations
Research and practice in the past decade has brought a greater understanding of the impact of engaging citizens in performance regimes, including its impact on improving public program performance and better aligning performance measures with citizen preferences (Cohn Berman, 2012). For citizen participation to matter, citizens may relay different perspectives and ideas from those of administrators or policymakers. The concept of the citizen in these efforts, though, is still fairly homogeneous.
Newman, Barnes, Sullivan, and Knops (2004) note that citizens are often described in binary terms, as either members of the able-bodied, capable, majority; or as those belonging to minority groups, with a disability status, or somehow marginalized. References to the public in documentation or research on citizen engagement typically include few descriptors, resulting in a picture of the public that aligns with this first description. Little attention has been given to engaging groups that may possess minority, disadvantaged, or marginalized status in performance regimes. Given the limited research conducted on the influence of citizen participation on performance regimes, one topic that remains unexplored is the impact of engaging marginalized citizens in performance regimes.
40


One population that is marginalized is people experiencing homelessness. Their perspectives on the performance measures used to gauge the success and impact of the public programs, such as housing programs intended to serve them, are important. These perspectives can provide additional options for performance measures or can provide some context to the performance data collected. To accurately measure the impact of these homeless assistance and housing programs, input from those who have been homeless is needed.
Housing and Homeless Assistance in the United States
Housing for low-income populations has been a relevant policy issue nationwide for the past century. More recently, permanent housing for individuals experiencing homelessness has taken primacy in federal policy and funding over general homeless services or emergency shelters (McKinney-Vento, 2009). With this focus, more permanent supportive housing projects have been developed in recent years, serving some of the most stigmatized and marginalized individuals in a community. Performance regimes established to measure the success of these housing communities serve as an effective case in point to explore the research question: How do preferences for performance measures differ between public sector representatives at the federal, state, and local levels, and the marginalized public served?
The focus on affordable housing in the public sector began as industrialization and urbanization occurred concurrently during the late 1800s and early 1900s, bringing the need for housing for urban workers (Purdy & Kwak, 2007). Decades of conversations regarding the need for housing in American cities did not result in any significant policy change until the passage of the Housing Act of 1937, which established the United States Housing
41


Authority, a structure for disseminating federal funds for public housing development to local communities (Shester, 2011). The call of the Act to develop housing was just as much a response to the Great Depression and the desire to increase employment opportunities and stimulate the economy as it was a response to the growing need for urban housing (Purdy & Kwak, 2007; Stoloff, 2004).
Twelve years later, the Housing Act of 1949 was passed with the intention to develop over 800,000 units of public housing over six years. Over a decade later, less than 40 percent of the units had been developed or were in the process of development (Coulibaly, Green, & James, 1998). The Housing Act of 1949 also established subsidized housing, including guidelines on rental and income limits and a prioritization of very low-income individuals, and moved the task of developing housing for low-income workers to the private sector (Stoloff, 2004).
Over the following decades, as a result of income limits as well as efforts by advocates to have public housing used for those who were most disadvantaged, units more commonly became occupied by some of the poorest and most marginalized in communities (Stoloff, 2004). As reported by the National Commission on Severely Distressed in Public Housing (HUD, 1992), as of 1981, around 2.5 percent of all public housing residents had an income level of 10 percent of the area median income or less (a measure of extreme poverty). By 1991, almost 20 percent of public housing residents fell into this income bracket.
At the same time that public housing was serving more individuals at lower income levels, the federal government was responding to calls from advocates and communities to recognize homelessness as a national issue requiring a federal response. In 1987, Congress passed the McKinney-Vento Homeless Assistance Act (titled at that time the Stewart B.
42


McKinney Homeless Assistance Act). This legislation established the United States Interagency Council on Homelessness and authorized multiple funding streams and programs serving homeless individuals including programs addressing food, shelter, housing, health care, and education (National Coalition for the Homeless, 2006).
The McKinney-Vento Act was amended in 1988, 1990, 1992, and 1994, expanding and strengthening the original programs and provisions outlined in the legislation. In 2009, the original act was amended with the passage of the Homeless Emergency Assistance and Rapid Transition to Housing (HEARTH) Act. While final rules for all of the programs falling under the HEARTH Act have not been published, interim rules and additional guidance from the Department of Housing and Urban Development (HUD) point to an increased emphasis on housing rather than emergency shelter. Specifically, funding guidelines under the McKinney-Vento Act promote and prioritize permanent supportive housing, particularly for individuals who are chronically homeless, meaning they have been homeless for a year or more or have had four or more episodes of homelessness in the past three years, and have a long-term disabling condition (HUD Exchange, 2012).
Permanent supportive housing is a general term for subsidized housing with no time limits that is accompanied by supportive services that include medical care, mental health and substance use disorder treatment, case management, employment assistance and job training, general life skills, and any other services that may be needed for individuals to retain their housing. Permanent supportive housing can either be developed as a single-site building, with all individuals residing there being formerly homeless individuals in need of this level of care; as a mixed-use building in which some individuals are formerly homeless and others do not meet this definition; or as a scattered-site project in which individuals
43


reside in apartments rented on the private market (typically with the help of housing vouchers or other rental assistance) and supportive services are offered to that individual to help them retain their housing.
While the HEARTH Act has formalized a response to homelessness driven by permanent housing options, additional guidance on implementation of the HEARTH Act and Opening Doors have advocated for communities to use a Housing First approach in homeless assistance efforts. Housing First describes an approach that provides access to safe, affordable, permanent housing for people experiencing homelessness first, then offers services and resources to residents to aid in retaining their housing (NAEH, 2006). As opposed to housing ready programs requiring sobriety, compliance with services, or other evidence of programmatic or clinical success prior to being housed, Housing First relies on the assumption that individuals moving out of homelessness should be held to comparable standards as any other housed person as outlined in a standard lease agreement. Supportive services provided to those in housing largely involve taking steps to avoid any issues that may jeopardize ones ability to uphold the lease.
Housing First programs have been shown to be as or more effective in helping individuals who are chronically homeless or who are homeless with mental health and/or substance use disorders in retaining housing as other housing programs (Groton, 2013). Housing First programs have also demonstrated significant cost savings to public systems when used to house individuals with high emergency service utilization. A Denver-based Housing First program reduced costs in emergency services including detoxification services, emergency department visits, incarceration costs, inpatient and outpatient medical care, and shelter usage by 73% in two years among program participants, totaling an average net
44


savings per person of $31,545 annually (Colorado Coalition for the Homeless, 2006). These outcomes, for both participating individuals and communities, have helped promote the use of Housing First strategies to address homelessness.
Permanent Supportive Housing in Fort Collins, Colorado
In 2010, a group of community leaders, prompted by local advocate Sister Mary Alice Murphy, launched a ten-year strategic plan to make homelessness rare, short-lived, and non-recurring. Among other strategies, the plan called for an increase in permanent supportive housing, particularly to serve residents experiencing chronic homelessness or with barriers severe enough to warrant long-term housing assistance and services. The plan also advocates for the community to utilize a Housing First philosophy in planning and service implementation, aligning with the prioritizations of the federal government outlined in the HEARTH Act and Opening Doors.
In Fort Collins, the public housing authority (Housing Catalyst) has the largest portfolio of affordable housing units of any housing provider. In addition, Housing Catalyst manages most voucher programs serving the area, including the Housing Choice Voucher (formerly Section 8 voucher) program, the HUD-Veterans Affairs Supportive Housing (VASH) voucher program, Tenant-Based Rental Assistance program, and Family Unification Program (FUP) vouchers. Housing Catalyst also develops new affordable housing throughout Fort Collins for a variety of income levels.
In 2011, new federal funding became available in the form of bonus funding through the Continuum of Care process, which is a HUD homeless assistance funding stream that allocates dollars to communities in alignment with the strategies outlined in the HEARTH Act. Housing Catalyst applied for funds that would provide ongoing operations support for a
45


new permanent supportive housing community. Additional grant monies were secured, along with low income housing tax credits, to develop a 60-unit apartment complex, with 40 of those units dedicated as permanent supportive housing.
This apartment complex, Redtail Ponds, secured its Certificate of Occupancy in Spring 2015, and residents moved in beginning in March 2015. Individuals occupying the 40 permanent supportive housing apartments all moved directly out of homelessness, with some previously staying primarily in emergency shelters, some primarily staying outside in unsheltered areas, and others staying in buildings unfit for human habitation. All residents in the permanent supportive housing units at Redtail Ponds have a diagnosed disabling condition, whether physical, medical, mental, or behavioral. The building is staffed 24 hours a day, seven days a week with, at minimum, a security person. Additional full-time staff include front desk staff, a property manager, a program manager, a case manager funded through the local community mental health center (SummitStone Health Partners), and a case manager for HUD-VASH clients. The full-time staff work closely with tenants to identify and arrange for additional services including medical check-ups, job training and employment coaching, or behavioral health treatment.
Redtail Ponds integrates many Housing First practices in its design and operations. The lease generally mirrors those used in other traditional affordable housing units, with expectations for tenancy revolving around paying rent on time, maintaining ones unit, and generally being a good neighbor. In line with Housing First, sobriety is not required for residents to stay in compliance with their lease.
For the first two years of operations, any individual meeting the qualifications of the building (for example, income falling between zero and 30 percent of area median income,
46


currently homeless, and living with a disability) could apply to be a resident. Prioritization, however, was given to those applicants referred from partnering service providers, including the community mental health agency, a local emergency shelter, and the Department of Veteran Affairs. Beginning in Summer 2017, Redtail Ponds began receiving referrals through a community-wide process called Coordinated Entry. Rather than every housing project maintaining their own waitlist and referral process, Coordinated Entry consolidates those separate lists into a single community-wide list on which individuals are organized not on a first-come, first-served basis, but instead by vulnerability. Those with the most severe disabilities or the longest length of time homeless are now referred first into any available housing units. Housing projects funded with Continuum of Care funding, such as Redtail Ponds, are required to accept referrals from the Coordinated Entry process existing in a community beginning in early 2018.
Those residents interviewed for this study were selected as tenants at Redtail Ponds prior to the Coordinated Entry process. Therefore, every tenant had been referred from a partnering service provider. Particularly for the first group of tenants moving in, Housing Catalyst favored selecting residents who remained in good standing with the referring service provider. Accepting tenants with positive referrals from these partner service providers was intended to increase the likelihood of success of Redtail Ponds, specifically in those first months as staff were discovering how best to manage the property and the community was keeping a close eye on how well the project would blend into the surrounding neighborhoods. As the first true permanent supportive housing project in Fort Collins, success at Redtail Ponds can help to build community and political will for future permanent supportive housing developments.
47


Redtail Ponds provides an ideal environment for exploring the research question outlined in this thesis for three reasons. First, given the stigmatizing nature of homelessness and of mental health and substance use disorders, residents of Redtail Ponds will have direct experience of living in one of the most marginalizing situations one can experience. Second, while anyone who has been homeless may be able to recount the experience and the perspectives it provided, residents at Redtail Ponds will have recently moved out of homelessness. Those experiences will be fresh in mind, but with being housed currently, they will also have the experience to comment on the evaluation and performance measurements of the housing project of which they are now residents and know well. Third, because of the federal funding supporting operations of the building, Redtail Ponds operates in a way that aligns with common best practices promoted by HUD and collects data for federally-established performance measures for permanent supportive housing.
HUD requires three standard performance measures that each funded permanent supportive housing project must report annually. The first performance measure relates to housing retention for the project overall, with projects expected to keep 80 percent of residents housed for at least one year. The second and third performance measures relate to changes in income, with the second performance measure focused on overall income (including benefits), and the third performance measure focused on earned income. These performance measures are narrow in scope and may not capture the breadth of measures that residents in permanent supportive housing perceive as capturing success.
In addition to each housing project funded by HUD collecting and reporting on specific performance measures, data from the region as a whole receiving funding (the Continuum of Care) are also collected and submitted to HUD. The highest performing
48


Continuums of Care are more likely to have all projects within that region funded each year. While the income-related measures for permanent supportive housing deem either maintained or increased income of residents a success, the system-level performance measures a Continuum of Care must collect focus on employment and income growth. As a whole, all of the funded housing projects in a Continuum of Care should be working towards increasing residents income and working with residents to secure employment.
Housing Catalyst must collect and report the performance measures outlined by HUD to remain in compliance with grant funding expectations. In addition to these performance measures, Housing Catalyst also has specified the following broad descriptors of success for Redtail Ponds: 1) A project the whole community can be proud of and, 2) A safe and beautiful housing community (Housing Catalyst, 2013a). These reflect Housing Catalysts general commitment to good neighbor practices including developing and managing attractive, safe housing communities; holding residents accountable to certain standards of conduct respectful of the surrounding neighborhoods and neighbors; and, communicating openly with area residents regarding any relevant development or management concerns (Housing Catalyst, 2015). To create the expectations for operating as a good neighbor and creating a safe and beautiful housing community that the entire community can be proud of, Housing Catalyst developed a Good Neighbor Statement of Operations for Redtail Ponds in conjunction with area neighbors. This document outlines general management practices for the building, tenant selection processes, and measures taken to ensure safety and security of residents and surrounding neighborhoods (Housing Catalyst, 2013b).
In addition to the HUD-required and locally-developed performance measures and goals, more research has been conducted in recent years on the effects of permanent
49


supportive housing and the best practices for management. Supportive housing projects have been able to demonstrate a reduction in the use of more expensive emergency services, including hospitals, jails, and shelters, once individuals are living in permanent housing. In addition, having robust supportive services available voluntarily is a standard best practice. Successful supportive housing projects are able to find creative ways to engage residents who may be reluctant to participate in any traditional case management or therapy. There is also an increasing emphasis on permanent supportive housing projects to accept those residents who are highly vulnerable and chronically homeless upon move-in. All of these components point to the following profile for a successful permanent supportive housing project: A project which accepts highly vulnerable and chronically homeless residents; which creatively engages residents in services while not requiring participation in case management, therapy, or other services; which reduces the impact to emergency rooms, jails, and other emergency services; which keeps residents housed in permanent housing for at least one year; and, which assists residents in, at minimum, maintaining their income if not increasing their income. Redtail Ponds also holds itself to the standard of being a good neighbor and a beautiful and safe housing community.
Previous research has shown that the preferences for performance measures established by public officials differ from what the citizenry deems important and relevant. No research has specifically examined how these preferences differ when comparing public officials to a marginalized citizenry. To begin to explore these differences, this research centers on the following question: How do preferences for performance measures differ between public sector representatives at the federal, state, and local levels, and the
50


marginalized public served? To inform this research question, the following sub-questions are also included in this research:
1. How do marginalized citizens and public sector professionals differ in the importance assigned to performance measures based on their impact at either the individual level, the system level, or the community level?
2. How do public sector professionals and marginalized citizens understanding of performance measures (i.e., what information they can provide or what change they may indicate) differ?
Two populations were involved in this research study. People living in permanent supportive housing at Redtail Ponds represented the marginalized public served. Each person moving into permanent supportive housing has experienced homelessness and been politically marginalized. Professionals working in the field of affordable housing or homelessness, with familiarity of the goals, intentions, and basic program design of permanent supportive housing represent the public sector professionals. Focusing specifically on individuals living in permanent supportive housing and the professionals who work closely with these housing projects also has the additional benefit of relating to the well-defined performance measures established by federal funding streams supporting the development and operations of permanent supportive housing.
51


CHAPTER III
RESEARCH DESIGN
As outlined in the previous chapter, this dissertation addresses the following research question: How do preferences for performance measures differ between public sector representatives at the federal, state, and local levels, and the marginalized public served? The premise behind this research question extends from findings within the limited literature on citizen engagement efforts in performance regimes. When citizens are called to the table, most communities find that the goals and values outlined by citizens differ from those outlined by employees and officials within the government (Cohn Berman, 2008; Cohn Berman, 2012; Spray Kinney, 2008; Woolum, 2011). The public generally prefers those measures related to outcomes rather than basic outputs, and values measures of service delivery, such as responsiveness.
The minimal research that has been conducted on citizen engagement in performance regimes has not examined the impact of engaging individuals with limited political efficacy, who are often marginalized and excluded from participating in public processes. These populations were at the center of some of the first federal efforts to engage citizens in public program planning. For those individuals who have been marginalized given their lived experience of homelessness, to better understand how their perspectives on performance measures for the supportive housing in which they reside differ from those deemed most important by professionals working in the field, two additional sub-questions support the main research question:
52


1. How do marginalized citizens and public sector professionals differ in the importance assigned to performance measures based on their impact at the individual, system, or community level?
2. How do public sector professionals and marginalized citizens understanding of performance measures (i.e., what information they can provide or what change they may indicate) differ?
Sampling and Recruitment
This research examines the perspectives of both public sector professionals and members of the marginalized community of people with experiences of homelessness. To examine the perspectives of the marginalized community members, the permanent supportive housing community of Redtail Ponds in Fort Collins, Colorado serves as the research site.
The selection of this site is both purposive and convenient (Singleton & Straits, 2010;
Teddlie & Yu, 2007). The site is convenient given the researchers knowledge of and connections to Redtail Ponds through her previous professional position of Director of Homeward 2020, a non-profit initiative that drives the implementation of Fort Collins 10 Year Plan to Make Homelessness Rare, Short-Lived, and Non-Recurring. In this role, the researcher advocated for the development of Redtail Ponds and worked daily with staff of Housing Catalyst, the entity that developed, owns, and manages Redtail Ponds. Given the working relationship between the researcher and the staff of Redtail Ponds, the process to obtain permission from Housing Catalyst to conduct research at Redtail Ponds was without complication.
The site was selected purposively given its alignment with common practices for permanent supportive housing and the federal funding awarded to support ongoing
53


operations of Redtail Ponds. While some other permanent supportive housing resources exist in Fort Collins, including housing vouchers that can be used on the private rental market with access to supportive services, Redtail Ponds was designed and developed to reflect the most up-to-date best practices for providing permanent supportive housing to people moving out of homelessness. Staff of Housing Catalyst and from other non-profit organizations providing supportive on-site services visited multiple projects throughout the country regarded as models for permanent supportive housing prior to designing and developing Redtail Ponds. These site visits and additional extensive research conducted led to intentional design considerations to best meet the needs of residents. For example, ceilings are high to provide a more open feeling in apartments, which is expected to aid the transition from people accustomed to sleeping outside to sleeping inside. Ample open, public space is available to foster a sense of community in the building. In addition, there is a single point of entry to the building, and the building is staffed 24 hours a day and seven days a week. This ensures somebody is available if a resident is in need of support or assistance at any time of day. It also serves as a buffer for some residents who may be trying to distance themselves from previous relationships, as the front door staff can refuse entry to those visitors on behalf of residents who may find it difficult to turn those acquaintances away.
As a result of these intentional design and staffing elements, Redtail Ponds has been highlighted in workshops on developing permanent supportive housing, supported by the Colorado Governors Office, the Colorado Housing and Finance Authority, and the Colorado Division of Housing. In addition to these elements that were voluntarily incorporated into Redtail Ponds and that have led the project to be acknowledged by state-level housing entities, Redtail Ponds receives an annual, renewable operating grant from the Department of
54


Housing and Urban Development (HUD) through the Continuum of Care process. This funding requires Redtail Ponds to measure and adhere to the performance measures set by HUD. For these reasons, Redtail Ponds serves as an ideal example of permanent supportive housing, selected purposively for this research.
To examine the perspectives of public administrators operating at the federal, state, and local levels, representatives from various public and non-profit agencies were also interviewed. Interviewed professionals held varying functions in the homelessness and housing field, including as administrators of permanent supportive housing, as funders of permanent supportive housing, or as capacity-builders. Capacity-builders work in public and non-profit agencies providing training, technical assistance, and other support to communities or agencies in developing permanent supportive housing or generally working to end homelessness. Professionals interviewed also represented different levels of government and different geographic purviews, working at either the local, state, or federal/national level. These interviews were conducted with the intention of collecting data from those working in the field, to serve as points of comparison to the findings from the residents interviews.
Study Participant Selection
Of the 60 apartments in Redtail Ponds, 40 are designated as permanent supportive housing, with the other 20 apartments designated as traditional affordable housing, offered for individuals at a slightly higher area median income than those units designated as permanent supportive housing. Only the permanent supportive housing residents are required to be moving out of homelessness to be eligible for tenancy at Redtail Ponds. Tenants of the permanent supportive housing units are also required to have a disability to be eligible for
55


residence. While these are requirements only for the 40 permanent supportive housing units, the majority of the first tenants in the 20 affordable housing units also had experiences of homelessness.
Most of the permanent supportive housing residents living in Redtail Ponds at the time of the interviews had been referred to the housing project by either the community mental health center, a local emergency shelter, or the Department of Veteran Affairs (for those units reserved for veterans because of the type of housing voucher associated with that unit). Therefore, most residents at Redtail Ponds are individuals who had been engaged with other service providers, had demonstrated they could comply with program rules and requirements, and were deemed suitable referrals to Redtail Ponds. When determining if someone is suitable for tenancy at Redtail Ponds, service providers may consider someones perceived stability, tendencies for violence, or ability to live in close quarters to others.
Given this tenant referral process which intentionally screened in individuals who staff felt could live successfully in a communal environment, follow rules, and generally be a good tenant, a selection bias does exist for the pool of eligible study participants. The tenant selection process to Redtail Ponds is now more objective, with new tenants being referred from one common community-wide list, with little to no consideration for tenants perceived suitability for the project. Therefore, repeating the interviews conducted in this study in the future may yield different results given the potential for a different, less engaged population to be housed at Redtail Ponds.
Given the marginalizing effects of homelessness and the performance measurements attached to the permanent supportive housing units, the 40 residents living in units designated as permanent supportive housing units were all eligible to participate in this research as
56


interview subjects. Within the pool of eligible interviewees, a convenience sample was used to select individual participants (Onwuegbuzie & Leech, 2007), as residents volunteered to be interviewed. The opportunity was advertised through publicly posted flyers. Front desk staff at Redtail Ponds managed the sign-up sheets for the interviews. Resident participants received a twenty-dollar stipend for their participation. To avoid any potential issues with affecting a residents reported income, the stipend will be paid in the form of a gift card. A total of 24 of 40 eligible residents of Redtail Ponds completed interviews, in line with other studies involving interviewing subjects who are experiencing or have experienced homelessness which have sample sizes ranging from 16 to 30 interviews (Dang & Miller, 2013; Collins & Barker; 2009; Kennedy, Grewal, Roberts, Steinauer, & Dehlendorf, 2014; Neale & Stevenson, 2014).
The sample of professionals interviewed was also selected purposively and for convenience (Singleton & Straits, 2010; Teddlie & Yu, 2007). To have a broad range of perspectives, representing both those working daily to manage permanent supportive housing as well as those with more removed positions as funders or capacity-builders, the researcher strategically targeted those who could offer these perspectives as well as variously represent those working at the local, state, and federal/national levels. Given the researchers professional position in the field, she was able to directly reach out to other professionals who fit the criteria of the research subjects she sought. A total of 14 professionals were interviewed. Five worked at the local level, four at the state level, and six at the federal/national level. Given that professional interview subjects were selected based on their availability and, often, their existing working relationship with the researcher, a sampling bias also exists for these interviews. Randomly sampling professionals working in different
57


capacities that mirror the criteria established by the researcher may yield different results. Between resident and professional respondents, a total of 38 interviews were conducted for this research.
Data Collection and Measurement Strategy
One-on-one, semi-structured interviews were the primary data collection strategy for this research, both for the residents of Redtail Ponds and for the professionals. For those residents of Redtail Ponds, one-on-one interviews provided the privacy and ability to engage in a dialogue regarding different performance measures that focus groups or surveys would not have allowed. Interviews with residents typically lasted between 30 minutes and one hour, depending on the verbosity of each individual.
Prior to conducting the interviews with the Redtail Ponds residents, the interview instrument was piloted with three residents at Redtail Ponds residing in the 20 affordable housing units. All three individuals had comparable histories and experiences with homelessness as those residents interviewed for this research. These pilot interviews provided an opportunity to refine the wording and order of questions, clarify instructions for the interview, and ensure the wording of questions would yield useful data.
Interviews with residents were all conducted at Redtail Ponds in a vacant office. Conducting all interviews at Redtail Ponds ensured all interview subjects could physically participate in the interview by eliminating the need for residents to arrange transportation to an off-site location. This location also ensured interview subjects were in a comfortable and familiar environment, which could facilitate conversation better than a strange, uncomfortable, or unknown location.
58


All interviews with residents were conducted during regular business hours in which multiple staff members were available. While no interview subjects expressed the desire or need to speak with any staff member as a result of completing the interview, all staff members were aware the interviews were taking place, and were available if this need did arise during or after the interview.
Prior to each interview with residents, the researcher first described the purpose of the interview and how responses would be used and reported. Anonymity was ensured to all participants, and residents were all informed that their participation was voluntary. Given all residents at Redtail Ponds are disabled (many with mental health or substance use disorders), during this introductory time, the researcher was also able to ask questions of the resident to gauge whether she or he was coherent and adequately understood the purpose of the research to participate. No residents were excluded from the research project due to an inability to comprehend the purpose of the study and their role in the research. Subjects were also provided the opportunity to ask the researcher any questions during this introductory time.
Every interview subject was asked to review and sign an IRB-reviewed and -approved consent form prior to beginning the interview process. Residents were also asked if the researcher could digitally record the interview. All but one resident permitted their interview to be recorded. Verbatim transcriptions of the interviews, along with any notes taken by the researcher during the interview process, serve as the data sources analyzed for this research.
Interviews followed a semi-structured format. Interviews began with the researcher gaining basic information from the residents including age, veteran status, and the length of time residents had been living at Redtail Ponds. Residents were then asked to comment and
59


reflect on those things they thought made Redtail Ponds successful, and those things they thought made Redtail Ponds less successful and could be improved upon. This provided the researcher with a general sense of how residents viewed success for the housing project. The next section of the interview involved residents ranking (on a five-point Likert scale) the importance of nine separate performance measures, pulled from relevant literature and guidance on permanent supportive housing. These performance measures were described in a way that would be understandable to a lay-person who did not have technical expertise in the field of homelessness. For each performance measure, the researcher also asked the interview subject to comment on why they assigned the performance measure the score they did and what information they thought the performance measure would provide.
The order of the performance measures was randomly generated for each interview so any consistencies in responses would not be due to the order in which the performance measures were presented. The researcher asked residents to specifically comment on the performance measures specified by HUD, asking if those measures could adequately determine whether a housing project was successful or not. Finally, residents were asked their opinions on participating in efforts to establish performance measures. The full interview tool for residents is included as Appendix B.
Interviews with the professionals occurred either in person or over the phone, depending on the location and preference of the interview subject. Interviews generally lasted between 30 minutes and one hour. Given that these interview subjects participated in these interviews in a professional capacity, they were not required to sign a consent form and no incentive was provided for their participation. Prior to beginning each interview, the researcher explained the purpose of the research and generally how findings would be
60


presented. Interview subjects were given the opportunity to ask any questions prior to beginning the interview process, and were asked if the interview could be digitally recorded. All professionals interviewed permitted recording the interview. Verbatim transcriptions of the interviews, along with any notes taken by the researcher during the interview process, serve as the data sources for this research.
Each interview began with the researcher soliciting basic information on the professionals background and experience in the fields of affordable housing and homelessness. Professionals were then asked to comment on how they and their colleagues use performance measures in their work. Following these questions, professionals were provided with the same list of nine performance measures and the same five-point Likert scale that was provided to the residents interviewed. As all professionals were working in the fields of housing or homelessness, the performance measures were presented in the technical language more commonly used in writings or guidance for professionals in the field.
As with the residents interviewed, the order of the performance measures was randomly generated for each professional so any consistencies in responses would not be due to the order in which the performance measures were presented. The interview subjects first offered their initial, quick ranking on each of the nine performance measures. Then, the researcher prompted the interview subject to revisit each performance measure and talk through their rationale behind assigning the ranking they did to the measure in question.
This process of revisiting the performance measures instead of asking for additional explanation when the professional initially ranked the measure served two purposes. First, it ensured if an interview was cut short, since these were occurring during regular work hours when other work emergencies may come up, the researcher would still have some data for all
61


performance measures with which to work. Second, it allowed for the interview subjects to have some space between their initial gut reaction to a performance measure and their more thought-through response to explain their reasoning behind each ranking. This enabled the researcher to see if any professionals may change their initial thoughts and for what reasons. Asking for a ranking and reasoning all at once may not have provided as much time for professionals to revisit and reflect on their thought process behind each ranking.
After receiving rankings and explanations for each of the nine performance measures, the researcher asked each professional if there were other performance measures the professional considered to be critical to collect and which performance measures would be the three most important to collect. The researcher also asked them to comment specifically on the HUD-required performance measures and whether those could adequately gauge a permanent supportive housing projects level of success. Interview subjects were also asked to guess which performance measures they thought the residents interviewed would consider to be the most important, providing information on how professionals assumptions regarding residents preferences may differ from or align with residents actual stated preferences. Finally, professionals interviewed were asked their opinions and advice on engaging residents in helping to establish outcomes or performance measures for permanent supportive housing. The full interview tool for professionals is included as Appendix C.
Explanation of Performance Measures
Nine performance measures related to permanent supportive housing were included in the interviews with both the residents and the professionals. Two of the performance measures discussed (retention in housing and increased income) are directly related to those outcomes HUD requires of funded permanent supportive housing projects. Retention in
62


housing measures whether people are able to stay in their housing or move on to other permanent housing. For a population that cycles frequently in and out of homelessness and has experienced homelessness for an extended period of time, simply providing housing that enables people to remain housed is considered a success by HUDs standards.
The performance measure related to increasing income has shifted over time. A commonly-accepted approach to providing services and housing to people experiencing homelessness used to center on a housing ready philosophy in which people were placed into transitional housing, would re-learn how to be housed and employed, and then graduate to permanent housing. Aligning with this framework, HUD focused on increasing individuals income primarily through employment, even while acknowledging that some individuals experiencing homelessness were living with severe disabilities that limited their ability to work. For instance, in the Annual Performance Plan for Fiscal Year 2006, HUD explicitly stated that [ejmployment is a critical step for homeless persons to achieve greater self-sufficiency (p. 121) and developed employment-related performance measures for projects (HUD, 2005).
With greater research and evaluation of homeless services and housing, the standard now centers on a housing first philosophy in which individuals are provided with access to permanent housing without preconditions, such as an income. With this shift to a housing first approach, there has also been an emphasis on serving those who are the most disabled and for whom self-sufficiency or employment may not be realistic goals. As the general framework through which HUD operates has shifted, their emphasis on employment has somewhat dissipated. Now, permanent supportive housing projects report on income through various measures, including reporting total income (from employment and public benefits) as
63


well as broken down by earned or employment income and income from benefits. The performance measure increased income used in this research aligns with HUDs general emphasis on individuals accessing, maintaining, and ideally growing that income while in HUD-funded housing or services.
The remaining seven performance measures come from relevant literature and current best practices in the field of housing and homeless services. The performance measures selected also reflect a variety of impact levels. Some performance measures are geared more towards measuring individual-level change. Others best measure change occurring within the housing project or within the homeless services system in a community. Still others measure change at the community level. A summary of each of the nine performance measures selected, level of impact, and general description are included in Table 1.
Table 1. Overview of Performance Measures
Performance Measure Description Level of Impact
Voluntary participation in case management or other services Measures the degree to which residents voluntarily participate in services provided Individual
Increased incomes of the residents Measures whether residents can increase their income either through employment or benefits Individual
Resident satisfaction Measures the satisfaction level of current residents Individual
Retention in housing Measures whether the project retains most residents in that housing or assists them in finding other permanent housing System
Chronicity and/or vulnerability of new residents Measures whether the project is serving those who qualify as chronically homeless or who are the most vulnerable, based on an identified set of criteria System
Reduced shelter bed days Measures whether the project alleviates the need for emergency shelter for residents, with the assumption that providing enough housing will reduce the overall need for shelter System
64


Table 1. Overview of Performance Measures contd
Performance Measure Description Level of Impact
Reduced jail bed days Measures whether by providing permanent housing, the frequency of jail stays is reduced for those residents Community
Reduced emergency department (ED) visits Measures whether by providing permanent housing, the frequency of emergency department visits is reduced for those residents Community
General community satisfaction Measures the satisfaction level of the surrounding community or neighborhood in which the project is situated Community
Analysis Strategy
This research aims to answer broad research questions related to the preferences for performance measures of marginalized citizens and professional working in the fields of homelessness or housing. Interviews with current residents of permanent supportive housing as well as professionals served as the primary data sources. A major portion of each interview involved the interview subject reflecting on and responding to questions about nine identified performance measures for permanent supportive housing. All interview subjects were also asked to rate each performance measure on a scale from one to five based on how important the interview subject felt the performance data would be to collect. In addition, interview subjects were asked to select the top three most critical performance measures from those provided. These rankings and the request for interview subjects to identify their top performance measures allowed the researcher to tabulate frequencies and mean scores on each performance measure, providing a general sense of the importance the interview subjects assigned to each performance measure.
The researcher then relied on constant comparison analysis (Leech & Onwuegbuzie, 2008) to identify other themes emerging from the verbatim transcriptions of the interviews.
65


While often used for grounded theory research, this technique was appropriate given the intent of the research to discover anew, themes and trends from the interviews. The researcher coded each interview, with each identifiable theme uniquely coded. Once all interviews within each subject group (residents or professionals) were coded, the researcher reviewed each interviews coding to ensure no themes that emerged in later interviews were overlooked in the initial interviews. This process also enabled the researcher to identify codes that were similar enough to merge into a single overarching theme. In addition, those portions of the interviews in which respondents were speaking specifically to one of the nine performance measures were coded as such to allow for a quicker review of all responses and comments related to each performance measure. A codebook of those themes that emerged is included as Appendix D.
When analyzing all frequencies, means, and codes for the interviews, the researcher referenced those characteristics that may have impacted responses, such as residents veteran status or gender, or professionals length of time in the field or experience with direct service. Given the sample size, little variation was identified when considering these characteristics. Where viable, these variations are noted in the research findings.
In addition to the interviews, the researcher reviewed selected documents published by HUD related to performance goals and outcomes for the department. The researcher coded segments of these documents related to HUDs description of housing and services for people experiencing homelessness, with a specific focus on how HUD described success for individuals served in these funded projects and how success was defined for HUDs funding priorities. For example, the documents were reviewed to determine if HUD described success for individuals and funded projects based on residents ability to be self-sufficient,
66


indicating HUD expects individuals to reduce their reliance on housing or other government assistance; or on residents ability to be stable, indicating HUD finds value in simply having residents maintain housing and a consistent quality of life, without an expectation that they will reduce their reliance on government assistance. The findings from this document review are intended to serve as a point of reference for and comparison to the statements made by interview subjects, particularly professionals who were asked in greater detail about their understanding or knowledge of how any HUD-required performance measures have been developed or why they believe HUD has adopted the performance measures currently in use.
67


CHAPTER IV
RESULTS
Characteristics of Respondents
This research is informed by interviews from 24 residents living in permanent supportive housing at Redtail Ponds in Fort Collins, CO and from 14 professionals with knowledge, expertise, and experience around housing and homelessness. All interviews with residents at Redtail Ponds were conducted between February and April 2015. Interviews with professionals were conducted between December 2015 and October 2016. Among the residents of supportive housing interviewed, the majority identified as male, with only six of the 24 respondents identifying as female. Six residents self-identified as veterans; one resident opted not to disclose his veteran status. This corresponds to approximately one-quarter of all residents surveyed who have served in the military. Overall, at least 38% of all residents in Redtail Ponds supportive housing units at any time are veterans as 15 of the 40 units are dedicated to veterans. The ages of respondents ranged from 31-60, with a mean age for respondents of 51 years old. The length of time residents had lived at Redtail Ponds ranged from one week to over a year, with a mean length of time of around eight and a half months.
Residents varied considerably on the length of time they had experienced homelessness prior to moving into Redtail Ponds. The shortest length of time of homeless reported was around three months, with the longest length of time reported of 19 years. The average length of time homeless among residents prior to moving into Redtail Ponds was over three years. Most residents were able to recall the approximate length of time they had spent homeless, however many residents who had had multiple bouts of homelessness or had
68


moved frequently in and out of housing had more difficulty recalling with much certainty the length of time they had experienced homelessness overall. In addition, some peoples experiences were limited to those times they were staying outside or in shelters, aligning with HUDs definition of literal homelessness, while others included stays with friends or family. Given the varying definitions people used to define their own homeless experiences and the limitation of relying on self-reporting, the three-year average length of time is only an approximation.
The housing professionals interviewed worked at varying levels in the public sector (e.g., local, state, or federal) and in different capacities. Table 2 provides breakdowns of the characteristics of the professionals interviewed.
Table 2. Number of Housing Professionals Interviewed, by Level of Authority and Role
Administrator Funder Capacity-Builder
Local 3 2 0
State 0 2 2
N ational/F ederal 0 1 4
Those professionals working as administrators are those directly responsible for overseeing, managing, and staffing supportive housing. All administrators are employed at a local level by Housing Catalyst, the owner and manager of Redtail Ponds. The funders are those representing agencies that regularly fund supportive housing, including both development costs and the ongoing operations and services costs. Capacity-builders are individuals responsible for supporting and building capacity in the state and in local communities to develop and manage supportive housing. Three of the four capacity-builders operating at the national/federal levels work for organizations with a formal relationship with HUD, providing contracted technical assistance and training to communities receiving HUD funding for housing and services dedicated to people experiencing homelessness.
69


Three of the housing professionals interviewed work for non-profit organizations (all falling into the national capacity-builder category). The three local administrators all work for a public housing authority, a quasi-governmental agency. All remaining housing professionals interviewed work for public agencies at the local, state, or federal levels.
A total of six out of the 14 professionals have had direct service experience at some time in their career, meaning they have worked directly with clients who are experiencing or have experienced homelessness, often as a case manager or program manager. Three out of those six with direct service experience are currently working directly with clients as local administrators of supportive housing. The majority of the housing professionals interviewed have worked at least 15 years in the fields of affordable housing or homelessness services. Only two out of the 14 professionals interviewed have worked less than five years in the field.
Summary of Performance Measure Rankings for Residents
Generally, residents were able to assign scores along a Likert scale reflecting their perceived importance of collecting those performance data. The Likert scale provided was a five-point scale, with a one described as not at all important, and a five described as absolutely important or critical. Three respondents provided feedback on what they thought of each performance measure and generally whether it made sense to collect or not, but opted not to use the Likert scale. In addition, occasionally other respondents would avoid assigning a score to a specific performance measure, largely those in which they did not see the relevance in collecting those data. A summary of the scores assigned by residents is included as Table 3.
70


Table 3. Residents Scores on Nine Performance Measures
Performance Measure Mean Score # Residents Scoring a 5 on the Scale # Residents Ranking in Top Three Minimum Score Awarded Maximum Score Awarded n
Retention in Housing 4.53 12 14 3 5 20
Chronicity/ Vulnerability 4.23 8 11 2 5 20
Resident Satisfaction 4.11 7 9 2 5 21
Reduced Jail Bed Days 4.00 6 7 1.5 5 21
Increased Incomes of the Residents 3.96 7 4 2 5 21
Reduced Shelter Bed Days 3.78 4 3 1 5 20
Reduced Emergency Dept. Visits 3.73 7 3 1 5 20
Community Satisfaction 3.71 7 7 1 5 21
Voluntary Participation in Services 3.67 6 4 1.5 5 21
Housing retention
Residents ranked retention in housing high and generally considered it an important outcome to collect. Twelve out of 20 residents assigning a score to this outcome measure ranked it as a five, absolutely important or critical. In addition, 14 of the resident respondents included retention in housing in their top three performance measures to collect.
71


This performance measure also had the narrowest range, with no resident ranking the measure less than a three on the Likert scale (fairly important).
The most prevalent theme among residents related to the performance measure of housing retention related to the core purpose of permanent supportive housing: providing a long-term or permanent place to live. Some connected this to the basic principle that it would allow people to leave homelessness and avoid having to return to that life. One resident also mentioned that this was important because it signified a whole different way of looking at [homelessness] by providing access to permanent housing without preconditions, in line with housing first practices.
Residents also mentioned that knowing people were choosing to stay could indicate
they were happy, comfortable, or felt safe in their apartment and in the building:
There's so many fears in a person that's homeless. And to be able to establish a safe residence kind of cuts all that out. You feel safe, you know. You know where your next meal's going to come from. You know you're going to wake up in the morning and have a nice hot shower. You know, all those things fall right into place. So, I'd say that's the most important.
Having a stable place to live could also allow people to work on what they needed to and get the help they needed. One resident described the difficulty in trying to do anything other than manage the crisis of homelessness when you are without stable housing by noting that when you're homeless, your head's just not in the right place to get help. You're just too stressed trying to survive. First having a permanent apartment could provide the stability and ability for people to focus on other needs and goals. One resident reflected on his recognition of his inability to stay healthy and care for himself without his apartment at Redtail Ponds, even in the face of being uncertain about staying:
72


I've been struggling with it for like the last couple weeks. I've been packing my pack and unpacking my pack. Packing my pack and unpacking my pack. Am I going to stay? Now I have a heart problem, tore up shoulder. I've had my spine fused since I've been here. My neck I've got a cracked vertebrae in my neck. And I've been ready to walk out the door. But then I realize if I do that, that the physical condition I'm in is just going to get worse and worse. So, bite the bullet, continue to ride the experiment, and we'll see what happens.
For some, that opportunity to take care of their existing needs may also set them up to
think forward into the future and consider plans for eventually moving on from Redtail
Ponds. One resident described the differences he saw in his neighbors goals and intentions:
[F]or some people, this might be permanent. Or for some, it might be a stepping stone to getting something better. And, if those people that choose to move out, if you keep track of it or whatever, then you know that they're doing well, then you were a stepping stone to them doing well.
In discussing this performance measure, many residents also indicated this housing
may not be an appropriate fit for all. Often, issues with ongoing alcohol or drug use,
behavioral issues that affected others, or an inability to follow rules were cited in residents
reasoning that Redtail Ponds was not the right place for some. Redtail Ponds does not
prohibit alcohol use, and staff often go to extensive lengths to avoid evicting residents, in line
with best practices for housing first-oriented permanent supportive housing.
Some expressed frustration with the perceived leniency of staff, wondering for those
who continue to heavily drink, how will their life be better here than it was on the street,
other than they have a roof over their head? Others acknowledged that for those who do end
up leaving or being evicted, it simply may not have been the right living situation:
[Bjasically if somebody can't [live] here, it's either because the facility isn't up to the challenge or the resident isn't. And so that's a case where you assume that.. .were following all the HUD guidelines. At least approximating them well. And then, if that person isn't able to make it in that environment, then that obviously wasn't the right environment to cultivate that sustained period of whatever [e.g., sobriety].
73


Chronicity and vulnerability
Chronicity and vulnerability of new residents was the second highest ranked performance
measure, based on mean score awarded. Eight of the 20 residents responding rated this
measure a five, absolutely important or critical, and 11 residents placed this measure in
their top three most important measures to track. Scores for this performance measured
ranged from two to five on the five-point Likert scale.
The most commonly cited benefit stated for tracking the chronicity and vulnerability
of residents related to how this information could be used to better serve residents. If staff
knew what illnesses, disabilities, or other circumstances someone faced, they could provide
the right kind of help and assist residents in meeting their needs. One resident noted how
staff having this information had helped him when he moved in:
[W]hen I first got here.. .David (the property manager) mentioned to me that if I needed a railing in the showers, he would have it installed. Yeah, if there was someone who was handicapped or needed something like that, they would certainly need to know that.
Another common theme similarly dealt with the need for staff to know about peoples disabilities and illnesses, but more for reasons of safety and more surrounding issues of mental health and substance use disorders. If staff did not understand peoples issues and illnesses, they may be caught off guard and unable to both help the person and keep other residents safe if something went wrong. One resident described the need for staff to be aware of peoples illnesses for this purpose:
[YJou're here to help people. But at the same time, you have to look out for everyone. So, I would want to know who's got what as far as.. .like mental issues or maybe medical issues. That way you can try to explain their behavior or protect them.
Protect them and the other people that they have to live with.
74


Some residents connected the potential issues seen with people with severe disabilities, including mental illness, to suitability for the building, thinking this could be used as a filtering point to select new residents. These residents expressed a preference for a treatment-first orientation to housing in which individuals with severe mental illness or substance use disorders may be required to access treatment before being provided with permanent housing. This model is incongruent with the approach Redtail Ponds is taking (the housing first approach) in which everyone is ready for housing and anyone can access permanent housing without needing to show any evidence of treatment compliance or other forms of clinical success.
Resident Satisfaction
Resident satisfaction was the third highest-ranking performance measure based on mean score. The range of scores assigned to this performance measure ranged from two to five. Seven of 21 residents responding ranked this performance measure as a five, absolutely important or critical, and nine residents included resident satisfaction in their top three most important performance measures.
Many residents expressed the importance of people being happy, recognizing that it's kind of the goal of everyone to lead a satisfied and happy life. One resident also connected being happy to housing retention noting, they want you to be happy so you can stay here. This comment mirrors some statements made by professionals in the field who felt that poor housing retention likely reflected that people were not happy or satisfied in their housing. What happiness means to residents, though, varied from person to person. Residents described happiness as being a reflection of people having privacy, people having a voice in how their housing operated, people not being depressed, people feeling safe,
75


people feeling a sense of unity and camaraderie, and the project promoting and reflecting a healthy environment in which to live.
The perception of to what degree the housing project in which a person was living could impact their happiness differed among residents. A few residents recognized this would be a helpful performance measure to track as the housing project could change things if residents were generally unhappy in their housing. These may be programmatic changes or rules of the property. This may also be something small, however, such as how staff interacts with residents. One resident noted the impact having happy staff has on residents stating, It's all up to [the staff], I mean if they're having a bad day, then they're showing it. Then they're not going to make too many people happy that day.
Other residents acknowledged the lack of control the housing project may have to affect someones level of satisfaction, noting you cant make everybody happy and no one can make another person happy or unhappy. Other factors, including mental illness or past trauma people are dealing with, may also impact residents satisfaction or happiness and are largely out of the control of staff. One resident illustrated this point, commenting on her own past experiences and trauma:
I'm very unhappy, ok? But that has to do with my personal stuff and I realize that. But I see a lot of other residents that are just tickled pink to be here and have a chance. And me if s just, you know, like I said, working through. The tragedy that I've gone through is just pretty deep.
Reduced Jail Bed Days
The performance measure related to reducing the number of days spent in jail was ranked fourth by residents, based on mean score. Awarded scores ranged from one and one-half to five. Six of 21 residents responding ranked this performance measure as a five, and seven
76


residents placed this performance measure in their top three most important measures to collect.
The most common theme expressed by residents related to the changes that can come from someone having access to a permanent place to live. For some residents, a change in the number of jail bed days would reflect individuals changing their lifestyle or not engaging in the same behaviors as they did before, including alcohol or drug use. One resident described these changes seen in residents after moving out of homelessness, and what a reduction in jail use could indicate:
Well, it'd show rehabilitative changes. See what they were doing before if they were hanging out in Hobo Park, they were doing drugs, they were drinking. Now all of a sudden you got them into this facility, they're going to groups, they're doing mindfulness, and look at what happened. No more steel bracelets. So, and I've seen quite a few of them come in here who were real hellions on the street. Staff doesn't need to know that. They're now being real cool.
Other residents who felt a reduction in jail bed days indicated what could change in someones life from homelessness into housing, connected ones homelessness to unnecessary police contact and criminal justice involvement that results from being homelessness. One resident described his own experience with spending time in jail while homeless:
I ended up a week in jail for a disorderly conduct ticket that I probably wouldn't have gotten if I had a home. And I could've had my outburst in my own domain rather than having no place to go to de-fuse.
Another resident noted that people living on the streets get in fights and may end up in jail because theyre fighting over survival. In both of these examples, residents communicate criminal justice and legal issues that arise specifically due to ones homelessness. While some simply acknowledged that it generally would be better for one to not go to jail, others connected a reduction in jail bed use to reduced costs to the community,
77


noting that every time we put someone in jail, that costs a lot of money for the city and state. This cost argument was also a common theme among professionals discussing this performance measure. While many of these residents pointed to circumstances beyond ones control, two residents stressed that ultimately behaviors and choices are up to the individual. A change in the number of visits to jail would therefore reflect a persons choice to change their behavior.
Others felt this would be a useful performance measure to collect and track due to safety reasons. Knowing who had been in jail and still was going to jail would allow staff to identify the criminal element in the building or determine who should not be permitted to live in the building at all. One resident described this concern for housing someone with a criminal history:
I don't want those people doing whatever they do here and that not be known because they could hurt somebody else and.. .they could be bringing other people in here and I guess as an apartment manager, a property manager, I wouldn't want those kind of people around.
While there were no overriding themes related to why people placed less importance on this measure than others, two residents did comment on issues of privacy. These residents felt you dont need everybody knowing what youre doing, and keeping track of peoples visits to jail would violate residents privacy. One resident also placed less importance on this performance measure given not all calls to police resulting in an arrest are legitimate but may instead reflect a he said/she said situation.
Increased Income
The HUD-related performance measure related to income was not perceived as favorably as the housing retention measure, the other HUD-required measure. Scores for increases in income ranged from two to five. Based on the average score for each
78


performance measure, the income measure ranked fifth out of nine possible measures. Only four of 21 residents responding placed increases in income in their list of top three performance measures, coming in sixth and tied with participation in services.
The most commonly stated reason for residents that this performance measure is less important relates to how having income and increasing ones income demonstrates ones stability and the corresponding ability for that person to move forward with their life. Rather than being stagnant and content in their current state, having income and working towards increasing income also demonstrates a sense of productivity and can indicate people are learning to live and budget more responsibly.
Some residents also connected their income (or lack thereof) back to personal feelings and emotions. For example, one resident noted having income was important to him because it helped him know I got power. Having income and knowing one can pay for the items they need to sustain, including their housing, can be a source of positive self-esteem and pride. By having an income and paying for rent, one resident also felt this helped her show her appreciation for her housing.
One resident with limited income who was unable at the time of the interview to work towards increasing her income mentioned the emotions that come with having enough money to pay for your expenses and do the things that make life more enjoyable. Referencing her neighbors with more income, she stated, Like, socially, emotionally, they seem happier. I'm kind of jealous of them, because they always have money and I don't. They always get to go do things, and I'm just like, Uh, I can't do that.
Two residents mentioned how income is tied to stability, and saw an increase in income not as an indicator of the individuals stability but as an illustration of what was
79


possible when one has the stability of permanent housing. It permitted people to focus on and be better equipped to seek and retain employment. While the increase in income was important, these residents framed this performance measure as a residual effect of sustained housing retention.
Another reason cited by multiple residents for tracking peoples incomes related to issues around scamming and playing the system. Residents in Redtail Ponds are expected to pay one-third of their income to rent. The remainder of the rent is paid by a Housing Choice Voucher. While some expressed issues of fairness and dishonesty if people did not disclose their full income, one resident also tied this back to the ability of the building to sustain and the effect on other residents if not all were forthcoming with their income amounts and sources:
They got a scale that they use. And people could get around it real easy and that wouldn't be fair to the other tenants. I think it's 30% of your income. And it'd be very easy to hide that income and not pay their way. You know, they don't pay their way then we're going to start losing amenities and the help we get and the maintenance of the building. Just so many things involved in that. Somebody making $2,000 a month and only pays $50 a month rent. You know, they gotta track that. Make sure he's not stashing money somewhere.
Residents stated a few reasons why measuring income and increases in income would not be the most useful or important measure to track. Several residents mentioned issues people may have with budgeting. Simply having more income would make little difference if people did not know how to use it well or spent it on items that were not necessities.
One resident also mentioned that income was not a driving factor in his experience of becoming homeless. If one became homeless only because of financial reasons, that could be remedied by getting a job or getting assistance. For this resident and others, there were other situations and factors contributing to their homelessness. If the intention was to help
80


someone move out of homelessness and into a place of greater stability, those other factors and not income should be addressed and tracked.
One resident did rate this performance measure lower for reasons that echoed those of the professionals interviewed. Many residents have disabilities or other barriers that may limit how much income they can earn. Tracking changes in income, then, may be of little value:
There's some of these folks that are never really going to have a private, you know, independent income again. Some of these guys that are here are strictly going to have benefits. They'll establish disability income if they haven't already and are going to have Social Security and Section 8 housing, or whatever is going to assist them with housing. And, they'll have their benefits, and that's it. So, as far as tracking income then, I'm not sure what purpose that serves.
In these cases, measuring changes in income is less indicative of whether the project is successful and more indicative of the need for supportive, affordable housing for people with limited income.
Reduced Shelter Bed Days
Scores assigned to the performance measure reduced shelter bed days ranged from one to five. The average score awarded to this measure was 3.78, placing it sixth out of nine measures by mean score. Four of 20 residents responding assigned this measure a five, as an absolutely important or critical measurement to collect, and three residents placed reduced shelter bed days in their top three most important performance measures.
This performance measure caused noticeable confusion and questions from residents. The most common theme expressed by residents was that it did not make sense to measure a reduction in shelter use because nobody should be returning to a shelter once they had permanent housing. One residents comments on this performance measure reflected others questions and confusion well:
81


I never thought they would use the shelter after they got an apartment. That one I don't know. I'd imagine they'd stay here as much as they could. Cause it's an apartment. I'd never heard of anybody going back to the shelter, though.
While most did not offer specific explanations of why returning to the shelter would be bad,
other than it did not make sense if someone had their own apartment to stay in, others
brought up poor conditions in the shelter. The shelter was described by residents as crowded,
as dirty, as contributing to illness, as a place to contract bedbugs, and as unsafe or potentially
violent.
Two additional reasons were provided for why this performance measure carries less importance than others. Issues of privacy were raised, with one resident noting that tracking where people go outside of this building is really [nobodys] business. One resident also noted that looking at shelter usage may not make sense given many people would choose to sleep outside. There would be no impact to shelters, even if the person did opt to not stay in their apartment.
The residents expressing confusion or questioning the validity of this performance
measure connected shelter use to an individuals circumstances and choices once in housing.
They did not discuss the collective impact to a shelter that may come from a large number of
individuals moving into permanent housing and reducing the overall need for shelter in a
community. A couple of residents, though, did make a connection between an individual
accessing and staying in permanent housing and the overall capacity of the communitys
system to serve people experiencing homelessness:
[I]f you can get a place like this and fall into a way of life where you get back into the mainstream of life and you're doing great, it frees up a bed for somebody else that really needs it.
82


Another resident connected this performance measure to issues of retention, noting it would be important to know that people were staying here and not having to go and use a shelter anymore or be homeless.
In discussing this performance measure, several residents noted the difficult transition
from homelessness to housing, citing this as a reason some people may choose to stay in a
shelter instead of their apartment. If people were opting to stay in shelters, it would show
that a habit had been developed. This may help staff understand how they could step in and
help someone to keep moving forward instead of backwards.
Reduced Emergency Department (ED) Visits
Residents rated the performance measure reduced emergency department visits
seventh overall, based on mean score. Scores assigned ranged from one to five, with seven of
20 residents responding scoring it as a five. Three residents placed this performance measure
in their top three most important measures to collect.
Residents most commonly communicated that a reduction in emergency department
visits would signify that people were able to deal with their medical and behavioral health
issues better once in housing as compared to when they were homeless. By having a
permanent place to live, people may see their health improve and be less prone to getting
sick. One resident, who had seen the toll that homelessness had taken on the health of those
on the street described the importance of housing as it related to improved health:
How many fewer times they went to the emergency department, though, that's huge for me. Because with my health situation and with the people I saw die on the streets.. .1 mean, if people are dying on the streets. If people have emergency health issues in the shelters. And if people are compromised to the point that their lives are in jeopardy. And you can provide them housing, and then suddenly those kinds of emergencies seem to go away or are minimized. Well, right there alone is the success of the program. I mean, good Lord. If you save one life, right?
83


Some residents stressed that not only was ones physical health compromised when homeless, but that the street takes a toll on you and will greatly impact your mental health as well.
Some residents focused on the use of emergency rooms as a place to keep warm during the winter months. One resident described how people experiencing homelessness may drink alcohol excessively with the purpose of being transported to the emergency room where they could stay warm for the night:
And they get really hammered sometime. And every night, the ambulance shows up and takes someone away. A lot of times, they'll do it on purpose because it's really cold out. And, they want to go to the hospital. Poudre Valley (Hospital) is wise to that. And they only keep them in there as long as is necessary. And they usually let them out about one or two in the morning when it's really cold. So, what they'll do, I mean if they're really desperate, they'll go out there and get hammered again and then have the ambulance pick them up again. They end up back in the hospital twice a night. Here, totally different situation. You have shelter here... Well, here you are, you got a room. You got no reason to go to the hospital.
Going to the emergency department less frequently, whether because one was
healthier or because one had another place to be warm during colder months, was generally
seen as a positive outcome. Two residents specifically mentioned the costs savings that
would come from fewer emergency department visits. People would cost the government one
way or another. Rather than paying for expensive hospital visits, helping to subsidize
peoples rent would be a better use of funding.
Those who ranked this performance measure lower pointed to one of two arguments.
First, issues of privacy were noted. Knowing whether someone visited the emergency
department should only be important to the person seeking medical treatment. Second,
some residents commented on the necessity of still receiving medical care, which may
involve a visit to the emergency department. For this reason, it did not seem appropriate to
84


make a goal out of reducing the number of trips to the emergency department that someone may take. This sentiment was also communicated by a professional currently working as an administrator in permanent supportive housing. Residents should feel confident staff will help them access medical care or call an ambulance if needed. If a goal was set to reduce the number of visits to an emergency department, staff may question or try to dissuade someone from calling an ambulance or seeking medical care in an emergency situation.
Community Satisfaction
Responses to the performance measure community satisfaction were varied, with residents generally feeling it was either very important or critical, or not at all important. Overall, the performance measure had the second lowest mean score awarded, and scores ranged from one to five. Seven residents (out of 21 resident respondents), however, ranked this performance measure as a five. Seven residents also placed this performance measure in their top three most important measures, a pattern more common of performance measures in the top half of performance measures discussed.
The most common theme emerging from residents centered on the stigma they felt that comes with being homeless or formerly homeless. Residents described the looks and attitudes from others in the neighborhood, noting they are viewed as a different species, as leeches, and as if were not even people. This stigma then often translated into a Not In My Backyard attitude toward the permanent supportive housing project. Multiple residents commented on how the neighborhood had fought Redtail Ponds being developed in its current location.
Although the majority of residents similarly communicated issues of stigma and community perception around homelessness, there was a split among residents with how
85


much importance this performance measure should carry given that stigma. For some,
because the neighbors unfairly looked down upon the residents, being concerned with their
satisfaction levels was not important. One resident explained why he felt paying attention to
the surrounding communitys perceptions and satisfaction levels was not important:
[W]hen they hear about a building like this, they're going to have preconceptions. Oh, this is going to be bad for business. This is going to be bad for the neighborhood.. .1 mean, this place does not bring down property values. Honestly, I think some people need to kind of just shut up. What other people think about the surrounding building,
I don't think it's that important.
For other residents, Redtail Ponds provides an opportunity for residents to help combat some of the stigma surrounding homelessness. One resident explained the importance of having a well-run supportive housing community by noting that we may be handicapped, and we may be using their tax dollars, but we can still contribute something back to the community that supports us.
Some residents also pointed to the importance of Redtail Ponds being successful and
the surrounding neighborhood being satisfied in helping to lay the groundwork for
developing other permanent supportive housing projects. As one resident noted, If it turns
out that [Redtail Ponds] wasn't what they thought it was going to be or as bad as they thought
it was going to be there will be less resistance in the future projects. Several residents also
noted things they and their neighbors do regularly to help show they can be a good neighbor
and to help reduce the negative perceptions of Redtail Ponds:
You've gotta get along with your neighbors. Even though they say they don't like the place or the people you know, a bunch of homeless people. I say, I'm not homeless. I got an apartment now. But I try to a couple of us go around, picking up trash. I don't know if they see that.
86


Another group of residents connected with the satisfaction of the surrounding neighborhood to their own personal safety, and for those reasons felt this performance measure was important. For some residents, the safety component of the project was inherent with the location and general characteristics of the neighborhood. Others noted that due to neighbor concerns, several safety provisions were put into place prior to the building opening including secured entrances, security cameras, and a 24/7 staffing pattern. While these may be frustrating for some residents, they help others feel safer in their housing.
While much of the conversation regarding this performance measure centered on the neighbors and concerns for having Redtail Ponds in their neighborhood, several residents did note that the building is surrounded by office and business buildings. There is some distance between Redtail Ponds and the nearest residential neighborhood. Because of the location, perceived as being isolated from the community for some, this performance measure was not important. It may possibly matter for another project that more directly impacted the neighborhood in which it was located, but the location of this project made it an inapplicable performance measure to track.
Voluntary Participation in Services
This performance measure had the lowest mean score of the nine performance measures discussed. Scores ranged from one and one-half to five. Six of 21 residents responding scored this performance measure as a five, absolutely important or critical, and four residents placed the performance measure in their top three most important performance measures.
The most common theme related to this performance measure centered on how services can be important in helping people to progress. Tracking participation in those
87


services offered on-site in permanent supportive housing can assist the program staff to see what's working and who shows up and how effective that is for helping people move forward. Although several residents discussed the services available in how they could be helpful, some residents pointed to others reluctance to participate, connecting this to a lack of desire to receive help and move forward. For example, one resident noted how people have changed since moving in, claiming, They came in. They got comfortable. They sat down, and theyve stopped working. Another resident described the services as a critical part of permanent supportive housing, particularly for those who choose to engage in services, stating, This is supportive housing. The idea of this is to help people that want help, that want to be more stable in their life financially, mentally, and physically. Theres a lot of people.. .that will just take a free ride.
For those who disclosed they are not regularly engaging in services, they described issues with anxiety, a desire to keep to myself, and a lack of interest in some of the classes or services provided. Two residents also expressed there may be other and better ways to help residents rather than the type of services or classes provided. Those who personally did not regularly participate in services or find value in what was offered also tended to see less importance for tracking this performance measure.
Two residents also specifically mentioned the financial component of providing services on-site. While some classes and services are provided by community members at no cost to the participants or the permanent supportive housing project, these residents assumed there was some cost to every service provided on-site. For these residents, tracking participation in services would allow staff to determine whether those services were being used and if the benefit warranted the cost. Another resident also commented on the
88


relationship between the cost of providing services and the resulting benefits, but instead
focused on the difficulty of measuring the actual impact of services:
[SJometimes participation may not be as big as they would like. They may think, "Oh, we thought 40% of the residents would show up for this. Only 10% showed up." But the 10% that showed up got so much from the program. It could change somebody's life.
Summary of Performance Measure Rankings for Professionals
Fourteen professionals responded to nine measures often used to gauge the performance and success of permanent supportive housing programs. Two of these measures (incomes of the residents and retention in permanent housing) are required outcomes for HUD-funded programs to report in annual grant reports and funding applications. The other seven performance measures are derived from guidance and recommendations from leaders in the field, including HUD, the United States Interagency Council on Homelessness, and non-profit technical assistance and training providers. A summary of the scores given by professionals for each performance measure is included as Table 4. Charts comparing the rankings of professional and resident respondents are included in Appendix A.
Table 4. Professionals Scores on Nine Performance Measures
Performance Measure Mean Score # Prof. Scoring a 5 on the Scale # Prof. Ranking in Top Three Minimum Score Awarded Maximum Score Awarded n
Retention in Housing 4.93 13 14 4 5 14
Chronicity/ Vulnerability 4.32 7 5 2 5 14
89


Table 4. Professionals Scores on Nine Performance Measures contd
Performance Measure Mean Score # Prof. Scoring a 5 on the Scale # Prof. Ranking in Top Three Minimum Score Awarded Maximum Score Awarded n
Voluntary Participation in Services 4.11 4 5 3 5 14
Reduced Emergency Dept. Visits 4.04 3 6 3 5 14
Reduced Jail Bed Days 3.93 2 4 3 5 14
Resident Satisfaction 3.79 5 4 1 5 14
Reduced Shelter Bed Days 3.68 2 2 2 5 14
General Community Satisfaction 3.39 1 1 2 5 14
Increased Incomes of Residents 3.18 1 2 2 5 14
Housing Retention
The retention in housing performance measure was consistently ranked as being highly important by professionals. Every professional but one ranked this measure as a five, absolutely important or critical, and all professionals included retention in housing as one of their top three performance measures to collect. In addition, this measure had the overall highest mean score of the nine performance measures to which administrators responded.
The high ratings for this performance measure corresponded with statements from professionals expressing how this measure reflected the ultimate intention of permanent supportive housing: to keep people housed, including those who have previously been
90


unsuccessful in housing. Thirteen of the 14 professionals interviewed framed this performance measure as being the whole main reason, the overall goal, the ultimate goal, and the whole key to permanent supportive housing. One professional working in a capacity-building function summarized the importance and weight this performance measure carries:
I think this is really the bottom line. For me about supportive housing is if it doesn't do anything else under the sun, it has to keep people housed... And the biggest way to tell if a project is not doing a good job is if they have high turnover. That means they're not doing whatever they can do to keep people housed. So I think this is the most important outcome that we should be measuring.
Given the vulnerable nature of those who are typically housed through permanent supportive housing, simply keeping residents in permanent housing may be a significant success, as it shows signs of increasing stability for the person. Aside from the weight this performance measure carries on its own, it is also a prerequisite of achieving success with any other performance measure. As stated by one professional interviewed, [I]ts the root of every other outcome. You cant do any of the other things unless theyre housed.
Chronicity and Vulnerability
Chronicity and vulnerability of residents was the second highest-ranking performance measure, based on the mean score awarded. Professionals cited the necessity to target limited resources to those who need them most. By prioritizing people who are experiencing chronic homelessness or are deemed to be extremely vulnerable, the permanent supportive housing units, which are often limited in number and expensive to operate, can be used as efficiently as possible. As one professional noted, The need is so great and the resources are so scare that youve got to be able to prioritize who you take in. One professional also noted that the individuals who are highly vulnerable and often experiencing chronic homelessness also use
91


significant community resources, such as healthcare or crisis mental health care. By prioritizing those individuals for access to permanent housing where they can receive ongoing care, other community resources can be freed up and used by others.
One professional, an administrator currently operating and managing permanent supportive housing, also noted that information about chronicity and vulnerability is helpful for program management purposes. By understanding what people have been through and what disabilities or other vulnerabilities they may be living with, administrators can better determine what supports and services need to be provided. This comment most closely aligned to residents perceptions on the utility of this measure in that by having more information about someones history and current health issues, others can better know how to support them.
While most professionals focused on the utility of this measure to the target and prioritize these units to those most in need, a couple of professionals also noted this performance measure is important because of its alignment with HUDs priorities. A program that states it will serve a certain percentage of people verified to be experiencing chronic homelessness can apply for different funding than without that criterion in place. If a program, such as Redtail Ponds, is already receiving certain streams of HUD funding for permanent supportive housing, then the need to serve individuals who are chronically homeless simply becomes the reality of operating that program. To ensure the program can continue to access this funding, a focus on the chronicity of new residents is necessary.
Perspectives on this performance measure were not uniformly positive. Two professionals noted that the measure is lacking given the data around chronicity and vulnerability can be fuzzy or the methods used to calculate who is chronically homeless or
92


Full Text

PAGE 1

PERFORMANCE MEASURES, MARGINALIZED POPULATIONS, AND DEMOCRATIC PARTICIPATION: THE CASE OF PERMANENT SUPPORTIVE HOUSING by VANESSA M. FENLEY B.S., Missouri State University, 2004 M.A., University of Kansas, 2006 A thesis submitted to the Faculty of the Graduate School of the University of Colorado in partial fulfillment of the requirements for the degree of Doctor of Philosophy Public Affairs Program 2017

PAGE 2

ii This thesis for the Doctor of Philosophy degree by Vanessa M. Fenley has been approved for the Public Affairs Program by Todd Ely, Chair Mary E. Guy, Advisor Christine Martell Donald Klingner Date: December 16, 2017

PAGE 3

iii Fenley, Vanessa M. (PhD, School of Public Affairs) Performance Measures, Marginalized Populations, and Democratic Participation: The Case of Permanent Supportive Housing Thesis directed by Professor Mary E. Guy ABSTRACT Democratic participation has been integrated in public administration theory and practice since the mid 20th century. It is a strategy used by public entities to enhance social equity by engaging citizens, including those who are marginalized and lack poli tical efficacy. Most efforts to engage the public focus on program or policy planning rather than developing and interpreting performance measures. Even more limited are efforts to engage marginalized citizens in this work. This dissertation explores how p references for performance measures differ between public sector representatives at the federal, state, and local levels and the marginalized citizens served by the public program of focus. Permanent supportive housing, or housing designed specifically for individuals moving out of homelessness, serves as a case in point to explore this research question. Research findings indicate that marginalized citizens differ frequently from public sector representatives in regards to both the importance assigned to s elected performance measures and to their perceptions of what specific performance measures may indicate. Findings contribute to the literature on performance measurement, democratic participation, and social equity, and have practical applications for pub lic administrators seeking to engage citizens in public processes. The form and content of this abstract are approved. I recommend its publication. Approved: Mary E. Guy

PAGE 4

iv ACKNOWLEDGEMENTS Throughout my doctoral studies, Mary Guy has served as an incompar able mentor and advisor. She kept me motivated, and helped me to see the true potential of my research during this dissertation process. Christine Martell also provided tremendous support, particularly during the beginning phases of my dissertation when my research ideas were just beginning to take shape. My other committee members, Todd Ely and Don Klingner, both were supportive and engaged committee members, offering advice and recommendations that strengthened this dissertation and my research agenda. I am appreciative of their support an d the time they took to serve on my committee. The staff at Housing Catalyst have been exceptional colleagues. Without their willingness to work with me and assist me in arranging to conduct my research at Redtail Ponds, I would not have been able to complete this dissertation. In addition, I appreciate all they do everyday to serve the people of Fort Collins, providing homes for people who otherwise may not have a roof over their head.

PAGE 5

v TABLE OF CONTENTS CHAPTER I. ST ATEMENT OF THE PROBLEM 1 Participatory Democracy as a Critical Component of Public Administration 2 Participatory Democracy and Performance Regimes 6 Homelessness as a Margina lizing Experience 8 Homelessness in Fort Collins, Colorado 11 Research Question 14 II. LITERATURE REVIEW 17 Participatory Democracy and Citizen Engagement 17 A Brief History of Citizen Participation in the Public Sector 17 The Rationale Behind Citizen Engagement 23 Considerations in Engaging the Public in Decision Making 25 Special Considerations for Marginalized Populations 28 Performance Measurements and Performance Regimes 31 Citizen Participation and Performance Regimes 33 Findings from Current Research on Citiz en Engagement in Performance Regimes 37 Special Considerations for Marginalized Populations 40 Housing and Homeless Assistance in the United States 41 Permanent Supportive Housing in Fort Collins, Colorado 45 III. RESEARCH DESIGN 52 Sampling and Recruitment 53

PAGE 6

vi Study Participant Selection 55 Data Collection and Measurement Strategy 58 Explanation of Performance Measures 62 Analysis Strategy 65 IV. RESULTS 68 Characteristics of Respondents 68 Summary of Performance Measure Rankings for Residents 70 Housing Retention 71 Chronicity and Vulnerability 74 Resident Satisfaction 75 Reduced Jail Bed Days 76 Increased Income 78 Reduced Shelter Bed Days 81 Reduced Emergency Department (ED) Visits 83 Community Satisfaction 85 Voluntary Participation in Services 87 Summa ry of Performance Measure Rankings for Professionals 89 Housing Retention 90 Chronicity and Vulnerability 91 Voluntary Participation in Services 93 Reduced Emergency Department (ED) Visits 95 Reduced Jail Bed Days 97 Resident Satisfaction 99

PAGE 7

vii Reduced Shelter Bed Days 101 Community Satisfaction 102 Increased Income 104 Oth er Themes Among Respondents 108 Alignment of Preferences for Performance Measures Between Residents and Professionals 115 Resident Involvement in Establishing Performance Regimes 121 V. DISCUSSION AND CONCLUSIONS 125 Impacts for Theory and Practice 131 Limitations 138 Future Research 141 REFERENCES 144 APPENDIX A. Comparisons of Resident and Professional Responses 152 B. Interview Tool for Residents 155 C. Interview Tool for Professionals 157 D. Codebook 160

PAGE 8

viii LIST OF TABLE S TABLE 1. Overview of performance m easures 64 2. Number of housing professionals interviewed, by level of authority and role 69 3. Residents' scores on nine performance measures 71 4. Professionals' scores on nine performance measures 89 5. Overview of findings 116

PAGE 9

1 C HAPTER I STATEMENT OF THE PROBLEM E ngaging citizens in performance measurement efforts has not been as strategic as efforts to incorporate participatory practices in other aspects of government work, such as program planning Engaging citizens that are marginalized or in the minority has be en given even less attention, both in research and in an applied context. As public entities strive to address performance gaps, particular attention should be given to the insights provided by those who are marginalized and often excluded from public discourse and decision making. Intentionally incorporating these diverse perspectives may allow communities to develop more comprehensive, thorough, and nuanced performance measurements that better address the varied outcomes an d goals that citizens as well as public managers desire Individuals experiencing homelessness are often stigmatized and marginalized. Although structural factors, such as a lack of affordable housing or high unemployment rates, contribute to homelessness, blame is often placed on the individual's behaviors or choices (Belcher & DeForge, 2012). This creates a perception and norm in communities that homelessness is a choice, the result of poor decisions, and the fault of the individual who is homeless. With blame shouldered by the victims they are seen as no longer useful or capable of contributing to a community (Belcher & DeForge, 2012) I nstead they are negatively constructed as powerless users or deviants (Schneider & Ingram, 1993) T heir input to commu nity programs and processes is rarely sought or is seen as un merited This failure to incorporate marginalized populations' voices leaves significant gaps in our knowledge on how to best develop performance protocols

PAGE 10

2 Participatory Democracy as a Critical Component of Public Administration The American state was developed to stand in stark contrast to the monarchy with which the country's founders were most familiar. While the intention was to create a system that would serve the people rather than a single ruler these processes were exclusively reserved for wealthy, well positioned men. For example, the d rafting of the constitution, the document serving as the cornerstone of the federal government, was closed to the public ( Elkins, Ginsburg, & Blount 2008). Subsequent ratification involved more individuals, but was still limited to a group of elite men Mosher (1982) labeled these early years the period of "government by gentlemen" given the st atus of those appointed to federal positions Mos t of the political elites (positions comparable to today's presidential appointees and other leaders in the executive branch) had upper c lass backgrounds, with their fathers coming from high ranking professions such as merchants and landowners. These gentleme n were skeptical of the capabilities of the average citizen to make decisions regarding the governance of the state. James Madison is often cited as one of the most fervent advocates for opting for a representative republic instead of a democracy w ith direct participation from citizens (deLeon, 1997). Madison outlined a series of checks and balances among branches of government, which were eventually adopted into the Constitution and became the framework for the current American state. This system c ould simultaneously limit and temper citizens' untamed passions while also helping to preve nt one faction from establishing themselves as the majority ruler, with no option for the minority factions to also be represented. The inauguration of Andrew Jacks on in 1829 brought a shift in how citizens' were perceived within the context of the American state. This era, an era of "government by the

PAGE 11

3 common man" (Mosher, 1982), brought overarching themes of pluralism and egalitarianism. While the intention was to o pen up governing to those with merit and skill, the resulting system was still fairly exclusive. This approach ushered in the spoils system, awarding political position to those close to elected officials. Government, during this era, was no more accessibl e to the "common man" and no more responsive to the will of the people. Subsequent reform effo rts addressed the spoils system and set the path for a professionalized civil service, but these changes failed to address how citizens were represented in the p ublic sphere. Mosher (1982) describes two forms of representation: active and passive. Active representation refers to administrators or elected officials who intentionally advocate on behalf of those they represent. Mosher (1982) saw this form of represen tation as "a major threat to orderly democratic government" (p. 15) since the unequal distribution of power that resides in the different interest groups represented would lead to an inequitable state. Passive representation generally refers to the make up of administrators elected officials and whether they demographically mirror the communities they represent. It is assumed that individuals with similar backgrounds will also share values and beliefs that will be translated into policies responding to th e needs of citizens from compa rable backgrounds (Lim, 2006). Passive representation can be problematic given elected positions are not as accessible to certain minority or marginalized groups Through passive representation, not all populations or interest s can be represented. Portions of the American population are unaccounted for or underrepresented among policymakers, making the opportunity for passive representation to influence policy decisions unlikely.

PAGE 12

4 Even without a perfectly representative elected body, officials from minority groups can still effect change (Lim, 2006). First, minority members may check members of the majority, bringing attention and disapproval of actions of the majority that are biased against minority groups. Majority members may respond to the potential to be "checked" by acting with greater restraint than if they were surrounded only by others in the majority. The adjustments of actions and behaviors that result from those in the minority checking those in the majority and the m ajority restraining their own biased actions can result in eventual re socialization wherein the values and beliefs of the majority shift. These options outline an ongoing issue with the current structure for American governance, established by Madison and other federalists. The core system was established to limit participation to those who owned property and even with a more diverse body of public officials today, the process of re socializing individuals to adhere to the values and beliefs of those in minority or marginalized groups is tedious and lengthy. Given these issues, other strategies are needed to ensure that perspectives of the diverse American citizenry are incorporated into the deve lopment and implementation of policies and public programs. Participatory democracy, in which citizens are actively engaged in decision making, extends an individual's ability to impact public processes beyond the more passive option of aligning with a par ticular political party or candidate. Participatory democracy is fundamental to public processes for multiple reasons. First, it is seen as contributing to a more socially equitable state. The concepts of equality and equity were woven into the founding of the American state, but were not addressed in administrative practices until more recently Grounded in the idea of a social contract and emphasizing equality among citizens of a state, the concept of social equity that is common

PAGE 13

5 in public administratio n today did not gain traction until the 1960's as racial and social tensions heightened (Guy & McCandless, 2012). The Minnowbrook Conference of 1968 and Frederickson's (1971) subsequent theoretical developments outlined in New Public Administration emphas ized the need to consider equity alongside efficiency and effectiveness in assessing government processes. As participatory democracy seeks to engage perspectives of all those imp acted by government processes, participation is a strategy now commonly used to enhance social equity. Participation by citizens can also increase accountability and transparency in the public sector. Accountability for the practices and outcomes of government shifted from residing in the elected official who could be held responsib le for missteps (Dubnick, 2005) to residing with bureaucrat s and those responsible for implementing public processes and carrying out the work of the public sector (Barberis, 1998). This shift came in tandem with other r eform effort s in the 1980's, under the umbrella of New Public Management, which promoted efficiency and incorporat ed private sector practices in public work (Hood, 1991). Alongside these efficiency driven changes in administrative practices grew public mistrust in gov ernment Engaging the public throu gh participatory processes can enable citizens to directly experience how government work was being condu cted, increasing the accessibility and transparency of public processes, and ultimately enabling the citizenry to hol d public officials accountable for their actions. In addition, strategically involving members of certain communities (either geographical or demographic) can help public officials understand the needs of that community and the potential repercussions of a ny policies or programs implemented. This "in" into the community is also beneficial if the outcomes of any decisions are not as positive

PAGE 14

6 as intended. With members of the community agreeing to and reinforcing decisions made by elected officials or administ rators by participating in decision making processes, the results of those decisions cannot be seen as solely the responsibility of public officials. Rather, the responsibility is shared among all participants of public processes constructing and implement ing the program or policy. Participatory Democracy and Performance Regimes Participatory democracy is a critical component of modern administrative practices, which are seen as advancing social equity, enhancing accountability, and providing access into c ommunities not commonly represented among public officials. One component of the public sector sometimes overlooked in regard to participatory practices is performance measurement and management. While governments commonly establish performance measures a nd follow through on data collection, citizens are largely absent from the process (Spray Kinney, 2008; Heikkila & Roussin Isett, 2007). Performance measurement involves collecting both outputs and outcomes from public programs, policies, and processes O utputs are those measurements that quantify services or products and generally communicate how much has been accomplished. Examples of outputs include the number of individuals served, the number of miles of streets paved, or the number of tons of recyclab les processed. Outcomes are measurements that describe the impact of a service or product. Outcomes could measure impacts such as academic growth in a classroom, recidivism rates among participants in a criminal justice intervention, or improvement in air or water quality. P erformance management involves using these outputs and outcomes to change or reinforce management practices (Hatry, 2010). The term "performance management" still

PAGE 15

7 does not encompass the breadth of uses of performance information for some scholars. Moynihan, et al (2011) use the term "performance regimes" to "refer not just to the practices of measuring and managing performance indicators but also to capture the embedded nature of these practices in almost all aspects of contemporary g overnance" (p. i141). This definition extends the use of performance information beyond management practices and into virtually all aspects of the public sector including how resources are allocated, how values of a public entity are formed and expressed, and how government structures its work and relationships with outside entities To accommodate the widespread uses of performance information in the public sector, this paper utilizes the term "performance regimes" when discussing the application of perfo rmance information to effect change within the public sector. Performance regimes, in both practice and study, have gained prominence in public administration starting primar ily in the 1980 s when New Public Management and its emphasis on increasing effici ency in the public sector increased in popularity and prominence (Hood, 1991) Performance regimes are tools for measuring government effectiveness, and subsequently affecting program im plementation and policy design. Given the widespread use of performanc e regimes in theory and practice, they must develop alongside and incorporate other trends in public administration to remain useful and be in alignment with other values of the field. With the emergence of New Public Management, much of the emphasis on s ocial equity that had been prominent under New Public Administration was buried under attempts to increase efficiencies and incorporate private sector practices into the public sector. Rather than seen as a critical and foundational component of the admini strative state, social equity

PAGE 16

8 would only have a place in New Public Management if it were incorporated as a performance goal or indicator (Hood, 1991). The use of performance regimes persists, and i ncorporating participatory democracy practices into these regimes can create a system that simultaneously works to achieve the diverse values of efficiency and social equity. At the most fundamental level, utilizing participatory democracy strategies in performance regimes can have utility because the perspective s of citizens can differ from those of government officials tasked with creating performance measurements and making decisions based on those performance measurements. Citizens can provide a context regarding other factors that may be impacting the intende d outputs or outcomes a program With the knowledge of both administrators and citizens involved, those developing and finalizing performance regimes can have more complete and nuanced information to inform decisions. The process of involving citizens come s with costs of additional time, effort, and resources spent. Without this engagement, though, administrators may be unaware of what information they are missing, limiting their ability to assess whether citizens' participation yields different or improved outcomes from what would have been developed otherwise. In addition, the process of engaging citizens upholds the value of social equity critical to public administration. Homelessness as a Marginalizing Experience Participatory democracy enhance s social equity in part because of its fo cus on engaging marginalized voices. Marginalization occurs when a particular grouping of people (by race, gender, social c lass, disability status, or an other identifying quality) collectively have limited political efficacy Political efficacy is compo sed of three interdependent components: a norm, a disposition, and a behavior (Easton & Dennis, 1967). As a norm,

PAGE 17

9 political efficacy describes the assumption that within a democracy, all members will be able to particip ate in and influence their government. As a disposition, political efficacy reflects community members' feelings about their ability to fulfill the norm of political participation and influence. As a behavior, political efficacy reflects community members' actual actions to participate in and influence the public sphere, regardless of their perceived ability to act efficaciously. If any of these three comp onents of political efficacy is diminished, a group of community members can be considered marginalized Depending on the group, marginalization may occur for several reasons. Political efficacy may be limited as a result of direct, intentional discrimination against the group, restricting its access to political decision making It may also result from stigma and fear of others' differentness This stigma can limit opportunities for participation because of the individual's perceived lack of ability to or worth of participating in public processes. Marginalization also can occur as a result of the statu s quo, with p rocesses and rules that limit marginalized community members' ability to participate in public processes institutionalized into standing systems. The degree of mar ginalization will differ among groups and can change over time. This may occur because of changes in cultural acceptance of certain groups that allow for individuals representing those groups to be invited in to political decision making processes Laws and policies that push for greater equity in access to power and decision making processes will also shift the degree of marginalization for different groups. Often, both the cultural acceptance of certain groups and the policies impacting their participation influence one another and lead to greater access to public participation.

PAGE 18

10 On e group of community members that qualif ies as marginalized is individuals experiencing homelessness. Homelessness is a stigmatizing experience (Belcher & DeForge, 2012; Phelan, Link, Moore, & Stueve 1997). Stigma is not an internally driven perception, but rather is reflected in social processes that shift non normative traits to negative traits (Goffman, 1963). In a society rooted in a sense of rugged individualism, someone who utilizes assistance from public or non profit entities may be seen as inadequate, lazy, or a drain on others (Belcher & DeForge, 2012; Jo, 2013). Given these perceptions, individuals experiencing homelessness may be seen as less capable or less worthy to participat e in public processes. Stigma also impacts the individual's perception of him or herself. This can in turn affect people's feelings a bout their ability to influence public decisions and their competence to participate in public processes. In addition, given the ext raordinarily low incomes of those who are homeless, many may not have the time or resources to participate in public processes T he costs of missing work and finding transportation are prohibitive Homelessness, including both the stigma attached to it and the difficulties navigating day to day functions may limit public participation T herefore it affects all components of political efficacy norms, dispositions, and behaviors and individuals experiencing homelessness are marginalized in public decisi on making processes To examin e participatory practices within performance regimes, homelessness serves as a relevant case for study The first federal efforts to incorporate the public into decision making processes began with efforts to engage communities that were marginalized given their economic status (a component of the marginalization of homelessnes s). Engaging community members wit h homeless experiences in studies of participatory practices builds

PAGE 19

11 on research and theory that was developed in communities with a similar basis of marginalization. Homelessness has also been given greater attention in recent years at federal, state, and local levels It is a timely issue to study and the outcomes of this research have the potential to impact practices in homelessness programs and policies. In 2010 the Obama administration released Opening Doors ( USICH, 2010) the first federal plan to end and prevent homelessness. This plan advocates for research based solutions and guides federal funding to align with those strategies. Permanent s upportive housing is i ncluded in th e strategies outlined in Opening Doors This type of housing is designed for individuals for whom significant barriers exist t o access housing on their own These may include disabilities or a n extensive history of ho melessness. Permanent supportive housing provides time unlimited housing support, c oupled with supportive services that are often offered on site. As described in the following section, a permanent supportive housing program in Fort Collins, Colorado, funded in part by federal dollars specified in Opening Doors serves as the setting for this research. Homelessness in Fort Collins, Colorado Fort Collins Colorado sits in the foothills of the Rocky Mountains in Northern Colorado approximately one hour north of Denver Serving as the county seat of Larimer County, the town has a total pop ulation of approximately 164 ,000, excluding the student population of Colorado State University, situated in the center of Fort Collins. The downtown area, or "Old Town," extending slightly north is home to many non profit homeless service providers inc lud ing two overnight shelters ( each provide daily meals), three day shelters, the food bank, Larimer County Human Services, the community mental health

PAGE 20

12 center, a Federally Qualified Health Center, and others. Homelessness in Fort Collins is often most visible in Old Town. In 2006, as conversations of development and preservation in Old Town and along the Poudre River corridor just north of Old Town began to take shape, local homeless advocate Sister Mary Alice Murphy called on a small group of local citizens t o consider the needs of people experiencing homelessness when crafting plans for the Old Town area. In 2010, after four years' work to better identify the current status of homelessness in Fort Collins, to research best practices, and to understand federal and state trends in ending homelessness, Fort Collins launched a local 10 Year Plan to Make Homelessness Rare, Short Lived, and Non Recurring. The Plan aligned closely with the strategies outlined in Opening Doors, released that same year. Homeward 2020, a community based non profit initiative, was formed alongside the Plan to drive its implementation. Leaders from Fort Collins' Plan also participated in the development of Pathways Home Colorado, a statewide initiative to ensure all Coloradoans have a pla ce to call home (Pathways Home, 2012). In a few years time, efforts and strategies for addressing homelessness at the local, state, and federal levels began to better align, with overall goals focused on reducing homelessness permanently through housing ra ther than managing the effects of homelessness through temporary shelter and services. To remain in compliance with federal funding requirements and to better understand the current state of homelessness, Fort Collins administers an a nnual Point in Time S urvey that counts the number of people experiencing homelessness on a single night in January. Based on the 2017 Po int in Time Survey conducted on the night of January 2 4, 2017, at least 33 1 individuals experience literal homelessness on a single night in Fort Collins. Those

PAGE 21

13 canvassed as part of the Point in Time survey are only those staying in emergency shelters, in transitional housing, or outside i n unsheltered areas including sleeping in cars, sleeping in sheds or abandoned buildings or camping. Aroun d 77 % of all respondents stayed in a shelter the night of the survey. Men comprised 65% of survey respondents. Around 81% of respondents were single adults Another 11% of respondents were unaccompanied children or youth under the age of 25. The remaining 9% of respondents were families with at least one child under the age of 18. Almost half of respondents self identified as having a disabling condition, i ncluding a mental health disorder, substance abuse disorder, traumatic brain injury, developmental disability, or a chronic physical illness or disability. In addition, 28% of respondents qualify as chronically homeless, meaning they had been homeless for a year or more or had four or more episodes of homelessness within the past three years, and had a long term disabling condition. Individuals and families experiencing chronic homelessness face some of the highest barriers to being able to access and to re tain housing I t is generally recommended that chronic homelessness is solved through permanent supportive housing (USICH, 2010) In March of 2015 new housing opened in Fort Collins to provide a permanent residence for individuals with disabilities who ar e also experiencing homelessness. Redtail Ponds is a 60 unit apartment complex in south Fort Collins, developed by Housing Catalyst, the public housing authority for Fort Collins, using a combination of low income housing tax credits federal, state, and l ocal funds and private donations. Forty of the 60 units are dedicated as permanent supportive housing with the remaining 20 serving as general affordable housing Ongoing operations are supported by a grant from the U.S. Department of Housing and Urban De velopment (HUD), renewed annually. Funding for this grant comes

PAGE 22

14 through the Colorado Balance of State Continuum of Care, a 56 county region that collectively applies to HUD for funding to support housing and services for people experiencing homelessness in Colorado. To qualify to live at Redtail Ponds, residents must be experiencing literal homelessness as d efined by HUD This means that they must meet the same qualifications as those surveyed by the Point in Time Survey (i.e., staying outside, in emergenc y shelters, or in transitional housing) In addition, to live in the permanent supportive housing units at Redtail Ponds, residents must have an income that is 30 % or less of area median income and have a disability. Qualifying disabilities include physica l disabilities, chronic physical illnesses, mental health disorders, substance use disorders, brain injuries, or post traumatic stress disorder. Although not a requirement for residence, most residents also qualify as chronically homeless. All residents a t Redtail Ponds are required to pay 30% of their monthly income (whether acquired through employment or cash benefits, such as disability or veteran benefits) towards r ent. Each resident is provided with a private one bedroom apartment (or two bedroom for those requiring additional space for medical equipm ent, live in care or partner ), multiple community space s the security and support of 24/7 staff coverage, and access to on site supportive services including behavioral health care group sessions, social activities, and case management. Research Question Participatory democracy is grounded in the idea of providing opportunities to those commonly excluded from political decision making, with limited efficacy and access to decision making processes. This research builds on previous studies of participatory

PAGE 23

15 democracy that similarly examine participatory practices as one option for enhancing social equity It expand s the literature by examining a component of public administration that infrequently utilizes citizen participation: performance regimes. The case of supportive housing for people moving out of homelessness in Fort Collins, Colorado provides an opportunity to conduct research into how marginalized voices c an impact performance protocols To begin to understand this impact and to contribute to the literature on citizen participation and performance regimes, this study focuses on the following research question: How do preferences for performance measures differ between public sector representatives at the federal, state, and local levels, and the marginalized public served? Exploring this research question matters for three primary reasons. First, if the preferences of individuals who are marginalized, wi th less political efficacy, differ from administrators and the general public but are not incorporated, the resulting performance regimes lack the breadth of perspective to fully gauge, report, and utilize performance information in a complete and nuanced manner. Second, the perspectives of marginalized communities are critical to further enhance social equity in the public sector. While relevant in any public work, engaging individuals from marginalized communities when it relates to policies and programs directly serving that population better ensures the program is not inequitably designed, implemented, and assessed for effectiveness without consideration of the viewpoints of those serv ed. Third, examining the differences in preferences for performance re gimes between those representing the public sector can contribute to a more robust understanding of where, when, and in what ways preferences align with the marginalized public served.

PAGE 24

16 This study offers both theoretical and applied contributions by expan d ing the minimal research conducted on citizen input to performance regimes It address es the specific gap of how input from marginalized populations impact s the development of performance me asures and performance regimes by examining marginalized citizens preferences for different performance measures. This research will also contribute to applied practices around performance regimes, particularly in permanent supportive housing programs. It provides initial evidence on what residents in permanent supportive housing programs deem most important in measuring the success of supportive housing programs, as well as how residents interpret commonly used performance measures. Continued research and modifications to applied prac tices that evolve from this line of research can ultimately inform more robust and inclusive performance regimes in supportive housing programs and other programs serving marginalized populations

PAGE 25

17 CHAPTER II LITERATURE REVIEW This chapter reviews the following three streams of literature: participatory democracy, performance regimes, and homelessness and housing policy. Research questions are identified at the intersection of these streams Participatory Democracy and Citizen Engagement Citizen participation is a key democratic ideal of the American public sector (Day, 1997). In today's public sector, the concept of citizen participation allows for a substantive role that extends beyond that of a passive voter Opportunities exist for cit izens to serve as collaborators and experts in developing public strategies that will respond effectively to their needs. Engaging the public gives legitimacy to public action and allows for decisions to be made beyond a small group of elite policymakers. This provides a n avenue to enhance social equity and to promote a sense of justice reinforcing the concept of public action as a tactic for building a strong democratic state (Fung, 2006). With expected outcomes and benefits for both the individual part icipants and the public sector, participatory practices have been used across jurisdictional levels and for a variety of purposes since the first large scale engagement efforts were implemented in the 20 th century. A B rief H istory of C itizen P articipation in the P ublic S ector Until recently, citizens informed public decisions most frequently through non engaged methods of voting and party involvement. A model of public participation relying on discourse and with more equitable opportunities for non policymakers to actively participate in decision making is a recent development (Webler & Tuler, 2000).

PAGE 26

18 Demonstration projects implemented in the mid 20 th century seeking to alleviate poverty and improve edu cation al outcomes in poor schools sought to e ngage local residents as leaders for community action (Rubin, 1969). These projects provided a foundation for modern public participation efforts. The f irst widespread efforts to in sert participatory practices into public programs occurred with the Housing Act of 1954, in which communities were required or strongly encouraged to involve citizens affected by the urban ren ewal projects funded by the Act In this case, citizen participation was sought less to ensure equity in public processes and more to co opt community members so they would not object to the projects once development and rehabilitation began (Johnstone, 1958). In the late 1950 s, a round the same time as the Housing Act of 1954 was being implemented Mobilization for Youth, Inc. was founded with the intention of improving issues of juvenile delinquency in the Lower East Side of New York (Moynihan, 1970). This effort, spearheaded by private philanthropists and notable academics with community members at the helm, conceived of community change occurring through community action. In 1961, a request for funding from the National Institute of Mental Health outlined the structure of this community based program. The details of the program, including the use of community action and language used to describe it aligned with the Economic Opportunity Act of 1964 presented two years later (Moynihan, 19 70 ). The 1964 Economic Opportunity Act established the Community Action Progra m as a strategy in President Johnson's War on Poverty. The legislation mandated "maximum feasible participation" among local communities, specifically among those citizens living in poverty and frequently politically isolated with the intention of buildin g community and

PAGE 27

19 improving service delivery through better coordination (Adler, 2012). The clause, given almost no attention in legislative discussions, was broadly criticized. Some insisted it would encourage revolution, local officials claimed it impinged on their authority, and welfare work ers asserted that people living in poverty would not participate as collaborative decision makers (Rubin, 1969). Although broad community participation was encouraged, individual organizations that were already prepar ed to mobilize and serve within the community could seek federal funding (Adler, 2012). Wi th funding secured, there was little incentive to work through conflict and intentionally engage stakeholders with divergent backgrounds and perspectives in order to achieve "maximum feasible participation." As Adler (2012) summarizes, "Given the social, political, and racial tensions of the sixties, in addition to the animosity many civil rights groups and the poor felt toward municipal political machines, in many cit ies around the countryconsensus was more of an ideal goal than a political possibility" (p. 553). At the same time these federal legislative efforts and demonstration projects were developed and implemented, the field of public administration was respond ing to citizens' changing attitudes toward government in attempts to remain relevant. Amid a sense of public unrest, scholars in the 1960 s began highlighting the role of democracy and the importance of social equity in the public sector (Kim, O'Leary, Van Slyke, Frederickson, & Lambright, 2010). These themes were core to the first Minnowbrook Conference in 1968, in which scholars met to discuss, debate, and wrangle with the complex social issues confronting the public sector at the time. New Public Adminis tration (NPA) arose o ut of the Minnowbrook Conference and the on goin g debates on how to resolve the disconnect between administration and democracy

PAGE 28

20 (O'Toole, 1977 ). Frederickson (1971) argued that administration had become more about the institution of go vernment than problems faced by the public; NPA could refocus the field on resolving those complex social problems. To do this, the field would need to re examine those values to which it held itself accountable. Rather than being dominated by the bureaucr atic values of efficiency and productivity, Frederickson (1976) noted that public administration needed to adhere to and be dominated by another set of values: "a concept of new public administration would have to begin with the argument that a differe nt (and certainly not new) set of values should predominate. These values would be humanistic and would be realized in decentralized, democratic organizations which distribute public service equitably. New public administration, therefore, would be the att empt to organize, describe, design, or make operative organizations that further these norms" (p. 167). Creating opportunities for involvement in public decision making processes with the intention of having stronger outcomes as a result as the Economi c Opportunity Act sought to do, reflects this shift in values. In administrative reform movements that have developed since NPA the role of the citizen has maintained a strong presence while shifting in the expectations of what that role should involve Beginning in the 1980s and 1990 s, as practitioners and scholars began to look toward the private market for models of how the public sector could be more effective, efficient, and economical, the citizen took on the role of the government's customer (Thoma s, 2012). For example, the Reinventing Government movement, sparked by Osborne and Gaebler's (1992) book by the same name, emphasizes competition and customer service both of which are common themes in the private sector New Public Management (NPM) sha ring enough tene ts with the reinvention movement that they are sometimes treated interchangeably, similarly emphasizes a market approach to public service delivery (Hood, 1991). NPM relies on the principle that i f

PAGE 29

21 government competes with the private marke t to provide certain services, it must remain responsive to the needs of its citizen customers Considering citizens as customers reduces the likelihood that a government can go about its work adhering to the rules and processes with less attention to the effects on the public. Considering citizens as customers has been questioned and criticized for several reasons. First, the choices that arise when entities compete to provide public services may not have a profound effect on citizens' choice for services or goods Instead more choices are provided to public administrators as they choose between potential providers to serve the citizens. While the process pulls from private sector practices the resulting scenario involves contracting out for services rat her than citizens selecting from among multiple choices of public services or goods they prefer (Thompson & Riccuci, 1998). Second, citizens are encouraged to make those decisions that are best for themselves or their family, responding as they would in a competitive market as a rational human being As Frederickson (1996) notes, "[t]he values of individual satisfaction are judged to be more important than the values of achieving collective democratic responsiveness" (p. 265). This tendency undermines the democratic ideals of the public sector and does not account for the irrationalities expressed in human decision making and human behavior. Finally, this approach relies primarily on the possibility for customers to leave a particular organization if it is not performing well While this may be effective motivation to improve services or products in the private sector, citizens cannot easily exit participation in their gov ernment. If a citizen is discontent with the national defense strategy and its implemen tation, there is no way for that citizen to opt out of receiving protection from national defense agencies. Government relies on a process of citizens claiming ownership

PAGE 30

22 and working to change and improve its functionality in order to improve performance (H irschman, 1970). Collaborative governance, another concept for understanding administrative funct ioning that emerged in the 1990 s, is also grounded in the ideal of participation of multiple stakeholders which can include citizens Ansell and Gash (2008) define collaborative governance as "[a] governing arrangement where one or more public agencies directly engage non state stakeholders in a collective decision making process that is formal, consensus oriented, and deliberative and that aims to make or im plement public policy or manage programs or assets" (p. 544). Others argue that collaborative governance practices can extend beyond those formal interactions between state and non state partners, and involve other arrangements such as public private partn erships (Emerson, Nabatchi, & Balogh, 2011). Now, citizens are viewed as not only collaboratively participating in the public sector, but also in partnering with government to co produce services (Thomas, 2012). For example, producing public education req uires students to show up and learn and parents to pre pare their children for school; producing public waste services requires residents to take their trash to the curb for collectors to pick up; and, producing the public safety services of the police requ ires citizens to report potential criminal activity. In addition to co producing these and other services, the public also has a role in producing public values (Rosenbloom & Gong, 2013; Thomas, 2012 ). Citizens choose to uphold or challenge certain polici es, rules, and programs through their actions, reinforcing or redefining the community's conception of what is important. For example, a citizen calling and reporting criminal activity is co producing public safety services and is also co

PAGE 31

23 producing the value of public safety (Rosenbloom & Gong, 2013; Thomas, 2012). Citizens in this role reinforce their desire for a safe community and express which behaviors are acceptable or not. For instance, citizens may be more inclined to report a robbery, which is s een as threatening to the safety of others, than to report someone jaywalking. The action of reporting crime and the discretion of what crimes will be commonly reported define the value of public safety through citizen co production. From co production o f services and values to engaged p articipation in public decision making processes to passive participation via voting and part isan identification t he concept of citizen participation in government has seen multiple iterations. Several public administrati on and public management theories and frameworks have incorporated public participation as a key concept for an effective government. While in theory, citizens should and do have multiple opportuni ties for participation, engaging the public in meaningful d ecision making processes in practice can have costs and can prove difficult for practitioners. The R ationale B ehind C itizen E ngagement A lthough the founding fathers wanted to establish a government directed and controlled by "the people," the state was never intended to allow for a widespread, active citizenry. Common citizens were seen as incapable of making the right choice according to politi cal leaders James Madison, in particular, is noted for advocating for a state that could temper the irrational or hasty interests of the common citizens by limiting their direct involvement and input in public policy (deLeon, 1997). As a result of these i nitial designs, our public sector has unfolded as one that seeks to involve citizens in politics while simultaneously not allowing for too much active public participation (King, Feltey, & Susel,

PAGE 32

24 1998). This foundational paradox illuminates why participato ry strategies can be troublesome and difficult to implement. Most see an engaged public as being more positive than one that is passive (King, Feltey, & Susel, 1998; Putnam, 2000). In addition to enhancing social equity in public processes, public entitie s engage citizens for many other reasons. C itizen participation is often cited as encouraging collaboration between the community, policymakers, and administrators This results in more effective and higher quality policies and programs that are more respo nsive to citizen needs (Bryson, Quick, Slotterback, & Crosby, 2012; Newman, Barnes, Sullivan, & Knops, 2004). E ngaging citizens allows for citizens and policymakers to learn from and have the opportunity to inform the other (Bryson et al, 2012; Irvin & Sta nsbury, 2004). For government partners, this educational process may help mollify fears or hesitations in the community around a proposed policy Particularly if a policy or program is controversial, a lack of communication may be perceived as a disingenuo us attempt to make u nfavorable or damaging changes. By being informed early on of a proposed change, citizens may instead perceive the government as being more transparent and more accountable to the residents of that community (Leroux, 2009). In addition, public officials may also see public participation as an opportunity to educate citizens on the limitations of government, enforcing why the bureaucracy works as it does and why certain changes cannot happen (Newman, Barnes, Sullivan, & Knops, 2004). Atte mpts to engage citizens are not equally effective in every community and for every issue. Generally, when the community or the multiple groups of stakeholders are diverse, co nsensus is difficult to reach, even with extensive community deliberation (Irvin & Stansbury, 2004). Rather than achieving a consensus, the result may instead be a decision

PAGE 33

25 reflective of only a few people's ideas. Stakeholder s may leave the process feeling ignored rather than engaged. The decision making process also becomes lengthier a nd more costly with greater citizen input, which leaves less funds for implementation. If the issue is not one that is likely to draw much controversy or potential litigation, extensive community engagement may be more detrimental than effective (Irvin & Stansbury, 2004). In addition, if the input provided by citizens is not incorporated into the final decisions, the public may develop greater resentment and distrust of the government (Irvin & Stansbury, 2004). While the rationale for engaging citizens in public processes is strong, in some cases it is not necessary or helpful given the costs of greater and more diverse public participation. Considerations in E ngaging the P ublic in D ecision M aking Public participation is not a singular concept but rather i nvolves three elements: the scope of participa tion, the methods of interaction and communication, and the level of authority given to citizen participants (Fung, 2006). The role of the citizen and the methods used to engage citizens will vary depending on the intended outcomes of the project. Other factors, including the timeline, funding available, intended participants, and requirements from any funding or regulatory agenc ies responsible for the program will also influence the methods used. Common forms of public participation include public comment periods, advisory committees, feedback surveys, and various forms of public deliberation (Leroux, 2009; Thomas, 2012). To accommodate multiple outcomes and work through multiple barriers, successful citizen pa rticipation strategies may utilize several methods to engage citizen stakeholders (Yang & Pandey, 2011).

PAGE 34

26 The scope of participation in participatory democracy processes is related to the mode of selection for citizen participants. This dimension encompasse s who is involved in the public process and how they are selected (Fung, 2006). Some projects may call for open participation from the designated community and participants self select. Depending upon the target community, this can yield a large and divers e participant body, with varying interests and perspectives. Although inclusivity may be an objective of engaging the public, this make up may cause difficulty when attempting to reach consensus on a topic. Collaborative decision making has benefits, but i t is most successful in small, homogen e ous groups rather than large, diverse communities (Ostrom, 1990). One limitation of a self selection process is that while some select in, some will select out. Rarely does a self selection process yield an equitable and representative grou p of participants For projects needing the input and participation of specific groups or specific individuals within a community, targeted recruitment may yield a more exclusive, but higher functioning group to tackle the designate d project. The desired participants may be engaged through strategic recruitment or by representing organized public interest groups (Thomas, 2012). Even if a specific group particularly a marginalized or minority group of the population is invited to participate, the perspectives they offer may not be held in the same regard as the majority of the community. Public officials generally see citizens as capable of contri buting to public processes, but as Fung (2006) notes, some populations may not be perc eived in the same manner. For example, those who receiv e welfare benefits wield little power and influence and are thought of more negatively (Schneider & Ingram, 1993). These constructions may affect the interactions within decision making bodies or affec t the ways in

PAGE 35

27 which policymakers choose to use the information gathered from those citizen groups. Additional research is needed to fully understand the manner and degree to which this may happen. In addition to the influence that social constructions play in citizen participation, very real barriers will also exist for some communities. Language barriers, lack of access to public processes (because of lack of transportation, childcare, or technology), or a general discomfort or feeling of exclusion can al l hinder the ability of people to participate in public processes. Although appearing neutral and allowing for equal participation, public processes will fav or those who have the skills, knowledge and resources to participate in bureaucratic processes. Wi thout these ski lls, knowledge, and resources, political efficacy is limited. Public participation will be as much about acclimating these individuals to existing processes as to examining how the public processes may better adapt to suit the needs of a div erse publi c (Newman et al, 2004). The methods of communication and decision making to allow for public participation also vary Participation in public processes can range from fairly passive, with community members serving as little more than spectators, to more active and engaged d uring which citizens participate in developing a true, collaborative outcome (Fung, 2006). In person forums with smaller groups may provide the level of interaction that allows for quality citizen participation (Thomas, 2012) Officials can strategically invite members of specific populations which allow s for a more diverse representation than a self selected process T his allows for a more collaborative process in which citizens and government officials learn from one anothe r. Though the perspectives of the multiple stakeholders and participants involved may not initially align, face to face discussion allows for a process of

PAGE 36

28 bargaining, deliberation, and negotiation to arrive at a final outcome that more effectively incorpor ates the viewpoints of participants (Fung, 2006). Online forums, posting regulations and information on websit es, and other technology based methods allow for greater participation in public processes, which may or may not be beneficial. In an analysis of online comments submitted to regulations.gov Bryer (2013) found the majority of comments were based on emotion and commenters largely failed to es tablish any credibility in having informed knowledge of the subject matter at hand. Technology increases the volum e of citizen participation, but the majority of participation is low quality, providing policymakers and administrators with little useable an d credi ble information (Bryer, 2013). Special C onsiderations for M arginalized P opulations T he public is often defined in one of two ways: as the white, pr ivileged, and able bodied group or as th e minority, disadvantaged group (Newman, Barnes, Sullivan, & Knops, 2004). Each of these groups is socially constructed (Schneider & Ingram, 1993; Barnes, Newman, Knops, & Sullivan, 2003), impacting their perceived ability to participate in public processes. The perception of whether or not a group is capable of ef fective citizen participation limits their political efficacy, marginalizing that population in the community. Engaging minority disadvantaged or marginalized groups is seen as beneficial for the opportunity to educate or empower participants (Newman, Ba rnes, Sullivan, & Knops, 2004). With limited political efficacy, d isseminating information can enhance citizen competence on the issue at hand or on ways individuals can participate in and influence the political process. Increasing individuals' knowledge and advocacy skills improve s the

PAGE 37

29 outcomes of those engagement strategies as individuals learn how to participate in public processes given the established forums (Yang & Pandey, 2011; Leroux, 2009). Improving knowledge and skills so that marginalized individuals can participate in established public processes allows for more equitable outcomes, though this process is still less than ideal. A pp roaching citizen engagement among marginalized communities with only the intention of assimilating those individuals to existing processes reinforces the power differential already in place These goals for citizen engagement emphasize public pro cesses that exclude marginalized por tions of the public (Barnes et al, 2003; Newman et al, 2004). As partici patory processes are generally meant to build consensus and represent the public broadly, relying on processes known to exclude portions of the population only highlights the differences between participating and non participating community members (Barnes Newman, & Sullivan, 2006). In addition to enhancing political efficacy through building skills and knowledge of marginalized communities, public officials can also examine altering public participation processes to better accommodate those individuals. There is no singu lar process or set of methods established as best practice for engaging marginalized populations. Communities or organizations develop engagement strategies based on participant profile, purpose of engagement, resources available, or the history between the engaged community and the engaging organization (Bryson, Quick, Slotterback, & Crosby, 2012; Thomson, 2012). Bryson, Quick, Slotterback, & Crosby (2012) identified guidelines for public participation, constituting an iterative process t hat encourages evaluation and redesign of the participation process. An effective participation process, including one that engages marginalized communities, should be designed to (a) fit the problem at hand and the purpose of engagement; (b) foster effect ive leadership from the

PAGE 38

30 engaged community; (c) engage diverse participation through inclusive processes; (d) use multiple technologies to heighten participation; (e) manage existing power differentials; and, (f) continue to generate resources for participa tion processes (Bryson, Quick, Slotterback, & Crosby, 2012) When marginalized communities are successfully engaged in public processes, the effects are varied. Different perspectives are offered, which may impact the design or implementation of the program or policy discussed. Participati on also impact s the individual. Gupte (2010) notes that public participation reinforces one's identity as a member of a certain group. With time, this identification strengthens that community and its advocacy power as more individuals develop close r ties to others. Stronger and better organized, that group is better positioned to advocate for changes needed to improve their community and generally promote equity. Public participation of marginalized communities may not only have an immediate impact on the public process at hand, but also can have rippling effects. Citizen participation creates a different conversation, framing the needs of the community and the responses by the state in a way different than what woul d occur with only elected officials or administrators partici pating in decision making. The inte ractions between ci tizens and public institutions serve to reshape those institutions and community values, influencing later responses from the public sector a s well as influencing how citizens react to and interact with the public sector (Stivers, 1990). The outcomes of citizen participation, then, may have a more long lasting effect than what is immediately noticeable. Including marginalized groups allows for a more diverse and robust dialogue that will shape not only policy, but also the long standing values and institutions in the community

PAGE 39

31 (Gupte, 2010). People who have recently experienced homele ssness are often marginalized, and t heir voices are frequentl y missing from public processes. They are even less frequently engaged in developing performance protocols given the limited use of participatory practices in general when establishing performance measurement and reporting processes. To improve performance processes and promote democratic participation, people experiencing homelessness have a role to play in developing performance regimes. Performance Measurement s and Performance Regimes Measuring outcomes in the public sector began in the 1960 s with a federal effort to connect program effectiveness to budget items ( Newcomer, Forthcoming ). This concept of "effectiveness" evolved into a focus on "results" and "outcom es eventually developing into an emphasis on performance measurement and performance mana gement Performance measurement generally describes collecting performance information and data while performance management requires the use of that performance information to make management decisions. Performance information may have even broader uses, influencing many aspects of the public sector. For this reason, Moynihan, et al (2011) coined the term "performance regimes" to capture the extensive impact that performance information has on cur rent government practices. Performance management and perfo rmance regimes emerged in the 1980 s, incorporated into the doctrines of New Public Management (Van Dooren, 2011) Using a market driven approach to governing, New Public Management advocates for the use of measurements to determine what is being accomplish ed and how well. This information can be used to make budgeting decisions and ultimately create a more efficient public sector.

PAGE 40

32 While used in some situations prior, the emphasis on performance measurement gained a prominent position in federal administrat ive processes beginning in 1993 with the passage of the Government Performance and Results Act (GPRA) (Lavertu & Moynihan, 2012). GPRA was grounded in the findings that a lack of clearly stated goals and related performance information diminished public t rust for government, limited administrators in implementing effective and efficient public programs, and impeded policymaking and program oversight by legislative bodies (GPRA, 1993). By implementing performance regimes that included setting goals, develop ing performance indicators, identifying data collection methods, and creating opportunities for administrators and policymakers to utilize this information, public programs could function more effectively and could enhance public trust. Since GPRA was ini tially implemented, researchers have developed a more comprehensive understanding of the uses of performance indicators and performance regimes beyond improving the effectiveness and efficiency of service delivery in public sector. First, as anticipated by the stated purposes of the GPRA, performance regimes enhance accountability and transparency of the public sector (Moynihan, 2008; Grosso & Van Ryzin, 2011). The publ ic distrust arising in the 1960 s fueled the desire for the government to be able to show what it was doing and how effective it was at its work. By aligning programs with outputs and outcomes, measuring th em and communicating results, the public ideally has more knowledge of how their government works and how their public dollars are spent. P erformance information is also used to celebrate and promote accomplishments of the agency and to motivate staff and stakeholders involved in the government's work (Behn, 2003).

PAGE 41

33 While performance measurements can be beneficial if used appropriately, the reliance on performance information may encourage some managers to do the minimum work required to collect the required performance information and disseminate the findings (Moynihan, 2009). In such cases, performance information is rarely used to modify a nd improve any management practices or program activities. Rather, collecting performance indicators is viewed as little more than a mandate to compl y with policies or program rules. The emphasis on performance indicators and the pressure to perform may also lead some managers to manipulate the program or the collection of data to demonstrate outcomes. Concentrating on those aspects of the agency that are measured may lead to a perceived improvement in performance, but if other aspects of the program are correspondingly deteriorating, no actual program improvement can be expected (Van Thiel & Leeuw, 2002). These factors all result in a tendency for performance indicators to lose their power and ability over time to differentiate strong performers from weak performers (Meyer & Gupta, 1994). To ensure th at performance information is used effective ly these possible shortcomings should be considered in the design of public performance regimes. Citizen P articipation and P erformance R egimes The trends of citi zen participation and performance regimes have generally been dealt with separately, both in the literature and in practice. Public officials usually pull citizens into public processes simply to inform them. More active participation by citizens, in which they are involved in collaborative decision making processes is rare (Yetano, Royo, & Acerete, 2010). Even in communities with strong efforts to engage citizens, the public participates more frequently in planning effor ts (Epstein, Wray, & Harding, 2006). While these impact the subsequent establishment of performance regimes, citizens' preferences on goals,

PAGE 42

34 outcomes, and measurements or their knowledge of other factors that may impact performance goals, are often not re flected in final evaluation efforts. Engaging citizens in processes to define, measure, and report outcomes is critical to establish ing performance regimes that will benefit the public sector and its constituents. First, engaging citizens helps ensure the goals of the public sector are aligned with citizen preferences (Spray Kinney, 2008), potentially affecting what programs or services are prioritized, funded, and implemented to meet those citizen preferences. In addition, including citizens when establis hing performance regimes can increase support for government a ction among citizens, increase social capital in a community, and increase the capacity of the public sector to deal with complex problems (Heikkila & Roussin Isett, 2007). Through citizen engag ement, ties among community members and between citizens and their government can strengthen, all while establishing performance regimes that measure and produce outcomes that the citizenry values (Cohn Berman, 2008; Epstein, Wray, & Harding, 2006). C itize n engagement can also improve the transparency and accou ntability of the public sector, one of the primary purposes of establishing performance regimes ( Moynihan, 2008; Grosso & Van Ryzin, 2011 ). Even if actual practices or programs do not change as the re sult of engaging the public in performance regimes educating and involving citizens in the process of measuring outcomes can lead to a perception of greater trust and accountability (Halachmi & Holzer, 2010). When perceived transparency, accountability, a nd trust increase, the public sector and its work is also perceived as more legitimate (Woolum, 2011). Although perceived favorably and as a missing component of larger citizen engagement efforts, few public agencies and communities have strategically or effectively

PAGE 43

35 involved citizens in performance efforts. Research from the Government Finance Officers Association found that fewer than five percent of communities engaged citizens when developing performance measures (Spray Kinney, 2008). Similarly, an anal ysis of special districts revealed that very few districts had any citizen input in developi ng performance measures (Heik k il a & Roussin Isett, 2007). The lack of citizen voice in performance measures is often reflected in evaluation reports published by public ag encies. Government reports do not regularly address those services or attributes of an agency that citizens deem most critical (Cohn Berman, 2008; Cohn Berman, 2012) While public agencies commonly report outputs, these measures mean little to cit izens because they have no basis for comparison with other communities. Citizens prefer to focus on outcomes, as well as measures of the quality of service delivery, such as responsiveness and accessibility (Cohn Berman, 2008; Cohn Berman, 2012; Woolum, 20 11). The disconnect between which indicators governments and citizens each deem relevant to measure and report can lead to a poor perception of government and the quality of its work and general confusion and frustration among both citizens and public off icials (Cohn Berman, 2012). Citizens may also have different preferences with how information is communicated (Heikkila & Roussin Isett, 2007), opting for simple, honest, and complete accounts of how their government is performing and how their community compares to others (Cohn Berman, 2008). Citizens prefer having all information accessible, rather than only those measures that highlight positive outcomes, allowing them to make assessments about the efficacy of the work conducted (Cohn Berman, 2008).

PAGE 44

36 T echnological advances allow governments to provide easier access to performance information to citizens via websites, multiple social media channels, or emails. With these options available, government officials must weigh the advantages and costs of using various formats for information dissemination. For instance, publishing performance information online is less costly and allows for more data to be published, as compared to printed data reports. Despite these potential benefits, Des Moines, Iowa opted to mail their final performance report to every one in the city with the assumption that more citizens would receive the information if sent directly to them (Matthes, 2008). Technology not only makes disseminating aggregated performance information easier and less expensive but it is also effective in providing more data, updated in real time or more frequently than printed evaluation reports For example, in the City of Chicago, the datasets on different community indicators are made available to citizens allowing them to examine those data that are of most interest to them rather than the limited data that a public agency may determine is important to communicate (Spray Kinney, 2008). Citizens in Chicago can access information on everything from the numb er of WiFi sessions used per month at public library branches to the number of days taken to issue a building permit to the percentage of homeless shelter beds used each month (City of Chicago, 2014). Although having thousands of indicators available for citizens to explore may have utility, understanding data is often complicated and complex. Without a mutual understanding between both citizens and public employees of what constitutes success, the conclusions drawn from data may still not lead to a greate r understandin g of how effectively the government is responding to a particular issue. In addition to providing the actual data,

PAGE 45

37 effective performance regimes that are responsive to citizens' needs also provide explanations and a context for understanding the numbers (Cohn Berman, 2012). Multiple reasons have been cited for not actively engaging citizens in performance processes. First, managers may question citizens abilities to make informed, qualified decisions regarding performance measures and changes in management practices (Heikkila & Roussin Isett, 2007). Particularly those areas requiring specialized or technical expertise are difficult for most citizens to understand (Cohn Berman, 2008) T his lack of knowledge may negatively impact citizens' abili ties to effectively participate in performance related activities. Public managers also perceive that citizens generally are not interested in being involved in many engagement efforts (Heikkila & Roussin Isett, 2007; Yetano, Royo, & Acerete, 2010). Partic ularly at the most active levels of par ticipation, public agencies find it difficult to engage citizens and retain their participation throug hout the performance regime process. Lack of resources, lack of political will, and a general resistance to change also impact efforts to engage citizens in performance regimes (Yetano, Royo, & Acerete, 2010). These concerns are valid, as even communities that have strategically changed their performance processes to allow for more robust citizen engagement have had to work through the challenges of involving citizens, gaining acceptance among public managers, finding the time and resources required and collecting accurate and reliable data (Cohn Berman, 2012). Findings from C urrent R esearch on C itizen E ngagement in P erformance R egimes Although still uncommon, some local governments have made significant attempts to engage citizens in establishing performance regimes. The Center on Government

PAGE 46

38 Performance at the National Center for Civic Innovation established a Trailb lazers program in 2003 (with additional cohorts of communities participating in later years) as part of an "effort to encourage and support government management practices that inform and are informed by the public" (National Center for Civic Innovation, 2 014). Communities accepted into the Trailblazers program were already collecting data, primarily consisting of costs, revenues, workloads, and outputs. Public employees and managers typically used the information ; reports compiled were generally not const ructed for public dissemination or consumption. Trailblazer communities were asked to engage citizens in performance processes, using the book Listening to the Public: Adding the Voices of the People to Government Performance Measurement and Reporting (Coh n Berman, 2005) as a guide. Trailblazer communities used a variety of methods to engage the public, with the most common approaches being surveys and focus groups. Some communities also utilized community conversations and citizen committees. As a result of these engagement efforts, Trailblazer communities tended to sha re two common observations: 1) t he public actually is interested in their government, wants information on what is happening within the government, and wants to be involved in government eff orts ; and, 2) p ublic agencies' assumptions about what citizens want and need from their government are often incorrect (Cohn Berman, 2012). After soliciting information from citizens, Trailblazer communities implemented changes to improve their performanc e regimes. Across all participating communities, five broad categories of change occurred. First, Trailblazer communities changed their practices by adopting new measures that better aligned with citizens' preferences (Cohn Berman,

PAGE 47

39 2012). Instead of relyin g solely on the output related measures previously used, communities utilized outcomes and measures of service quality and responsiveness. Second, performance reporting improved, with reports becoming more accessible to the public (Cohn Berman, 2012). For some communities, this process led them to produce a performance report for the first time. Other communities modified their reports to be more user friendly for the public, including relying on graphs and charts to relay data, providing both quick summar ies as well as more in depth information, and including information to provide a context to many of the data C ommunities also changed how information was disseminated back into the community. In addition to publishing more comprehensive reports, Trailblaz er communities reported posting data and reports online and printing shorter performance summaries in brochures or report cards. A third major change seen as a result of the Trailblazer program was the adoption of legislation by participating governments requiring performance reporting to the citizenry (Cohn Berman, 2012). This formalizes and institutionalizes the need for effective performance regimes that engage citizens in decision making, helping to ensure these practices are sustained after the initia l champions of the Trailblazer program leave their post s Fourth, public managers and elected officials came to rely on the performance data to a greater degree than before (Cohn Berman, 2012). For many agency heads or managers, performance data became a valuable tool for evaluating current programs and making subsequent changes to improve those programs. Elected officials utilized performance data when making funding decisions or other programmatic decisions.

PAGE 48

40 Finally, as a result of the Trailblazer pro gram, governments reported their performance improved (Cohn Berman, 2012). Participating communities reported employees beginning to rely on data to look for opportunities for improvement and implement changes to enhance perform ance. All of these effects d emonstrate the potential for public agencies to ultimately serve the public more effectively when citizens' ideas and preferences inform the established performance regimes. Special C onsiderations for M arginalized P opulations Research and practice in the past decade has brought a greater understanding of the impact of engaging citizens in performance regimes, including its impact on improving public program performance and better aligning performance measures with citizen preferences ( Cohn Berman, 2012 ). For citizen partic ipation to matter, citizens may relay different perspectives and ideas from those of administrators or policymakers. The concept of "the citizen" in these efforts, though, is still fairly homogen e ous. New man, Barnes, Sullivan, and Knop s (2004) note that citizens are often described in binary terms, as either members of the able bodied, capable, majority; or as those belonging to minority groups, with a disability status, or somehow marginalized. References to the "public" in documentation or research on citizen engagement typically include few descriptors, resulting in a picture of the public that aligns with this first description. Little attention has been given to engaging groups that may possess minority, disadvantaged, or marginalized status in performance regimes. Given the limited research conducted on the influence of citizen participation on performance regimes, one topic that remains unexplored is the impact of engaging marginalized citizens in performance regimes.

PAGE 49

41 One population that is marginalized is people experiencing homelessness. Their perspect ives on the performance measures used to gauge the success and impact of the public pro grams, such as housing programs intended to serve them are important. These perspectives can pr ovide additional options for performance measures or can provide some context to the performance data collected. To accurately measure the impact of these homeless assistance and housing programs, input from those who have been homeless is needed. Housing and Homeless Assistance in the United States H ousing for low income populations has been a relevant policy issue nationwide for the past century. More recently, permanent housing for individuals experiencing homelessness has taken primacy in federal policy and funding over general homeless services or emergency shelters (McKinney Vento, 2009). With this focus, more permanent supportive housing projects have been developed in recent years, serving some of the most stigmatized and marginalized individuals in a community. Performance regimes established to measure the success of these housing communities serve as an effective case in point to explore the research question : How do preferences for performance measures differ between public sector representatives at the federal, state, and local levels, and the marginalized public served? The focus on affordable housing in the public sector began as industrializ ation and urbanization occurred concurrently during t he late 1800s and early 1900 s, bringing the need for housing for urban workers (Purdy & Kwak, 2007). Decades of conversations regarding the need for housing in American cities did not result in any significant policy change until the passage of the Housing Act of 1937, which established the United States Ho using

PAGE 50

42 Authority, a structure for disseminating federal funds for public housing development to local communities (Shester, 2011). The call of the Act to develop housing was just as much a response to the Great Depression and the desire to increase employme nt opportunities and stimulate the economy as it was a response to the growing need for urban housing (Purdy & Kwak, 2007; Stoloff, 2004). Twelve years later, the Housing Act of 1949 was passed with the intention to develop over 800,000 units of public ho using over six years. Over a decade later, less than 40 percent of the units had been developed or were in the process of development (Coulibaly, Green, & James, 1998). The Housing Act of 1949 also established subsidized housing, including guidelines on re ntal and income limits and a prioritization of very low income individuals and moved the task of developing housing for low income workers to the private sector (Stoloff, 2004) Over the following decades, as a result of income limits as well as efforts by advocates to have public housing used for those who were most disadvantaged, units more commonly became occupied by some of the poorest and most marginalized in communities (Stoloff, 2004). As reported by the National Commission on Severely Distressed i n Public Housing ( HUD, 1992), as of 1981, around 2.5 percent of all public housing residents had an income level of 10 percent of the area median income or less (a measure of extreme poverty). By 1991, almost 20 percent of public housing residents fell int o this income bracket. At the same time that public housing was serving more individuals at lower income levels, th e federal government was responding to calls from advocates and communities to recognize homelessness as a national issue requiring a federal response In 1987, Congress passed the McKinney Vento Homeless Assistance Act (titled at that time the Stewart B.

PAGE 51

43 McKinney Homeless Assistance Act). This legislation established the United States Interagency Council on Homeless ness and authorized m ultiple funding stream s and programs serving homeless individuals including programs addressing food, shelter, housing, health care, and education (N ational C oalition for the H omeless 2006). The McKinney Vento Act was amended in 1988, 1990, 1992, and 199 4, expanding and strengthening the original programs and provisions outlined in the legislation. In 2009, the original act was amended with the passage of the Homeless Emergency Assistance and Rapid Transition to Housing (HEARTH) Act. While final rules for all of the programs falling under the HEARTH Act have not been published, interim rules and additional guidance from the Department of Housing and Urban Development (HUD) point to an increased emphasis on housing rather than emergency shelter. Specificall y, funding guidelines under the McKinney Vento Act promote and prioritize permanent supportive housing, particularly for individuals who are chronically homeless, meaning they have been homeless for a year or more or have had four or more episodes of homel essness in the past three years, and have a long term disabling condition ( HUD Exchange, 2012 ). Permanent supportive housing is a general term for subsidized housing with no time limits that is accompanied by supportive services that includ e medical care, mental health and substance use disorder treatment, case management, employment assistance and job training, general life skills, and any other services that may be needed for individual s to retain their housing. Permanent supportive housing can either be developed as a single site building, with all individuals residing there being formerly homeless individuals in need of this level of care; as a mixed use building in which some individuals are formerly homeless and others do not meet this definition; or as a scattered site project in which individuals

PAGE 52

44 reside in apartments rented on the private market (typically with the help of housing vouchers or other rental assistance) and supportive services are offered to that individual to help them retain their hou sing. While the HEARTH Act has formalized a response to homelessness driven by permanent housing options, additional guidance on implementation o f the HEARTH Act and Opening Doors have advocated for communities to use a Housing First approach in homeless assistance efforts. Housing First describes an approach that provides access to safe, affordable, permanent housing for people experiencing homeless ness first, then offers services and resources to residents to aid in retaining their housing (NAEH, 2006). As opposed to "housing ready" programs requiring sobr iety, compliance with services or other evidence of programmatic or clinical success prior to being housed, Housing First relies on the assumption that individuals moving out of homelessness should be h eld to comparable standards as any other housed person as outlined in a standard lease agreement. Supportive services provided to those in housing largely involve taking steps to avoid any issues that may jeopardize one's ability to uphold the lease. Housi ng First programs have been shown to be as or more effective in helping individuals who are chronically homeless or who are homeless with mental health and/or substance use disorders in retaining housing as other housing programs (Groton, 2013). Housing Fi rst programs have also demonstrated significant cost savings to public systems when used to house individuals with high emergency service utilization. A Denver based Housing First program reduced costs in emergency services including detoxification service s, emergency department visits, incarceration costs, inpatient and outpatient medical care, and shelter usage by 73% in two years among program participants totaling an average net

PAGE 53

45 savings per person of $31,545 annually (Colorado Coalition for the Homeless, 2006). These outcomes, for both participating individuals and communities, have helped promote the use of Housing First strategies to address homelessness. Permanent S upportive H ousing in Fort Collins, Colorado I n 20 10, a group of community leaders, prompted by local advocate Sister Mary Alice Murphy, launched a ten year strategic plan to make homelessness rare, short lived, and non recurring. Among other strategies, the plan called for an increase in permanent suppor tive housing, particularly to serve residents experiencing chronic homelessness or with barriers severe enough to warrant long term housing assistance and services. The plan also advocates for the community to utilize a Housing First philosophy in plan ning and service implementation, aligning with the prioritizations of the federal government outlined in the HEARTH Act and Opening Doors. In Fort Collins, the public housing authority ( Housing Catalyst ) has the largest portfolio of affordable housing units o f any housing provider. In addition, Housing Catalyst manages most voucher programs serving the area, including the Housing Choice Voucher (formerly Section 8 voucher) program, the HUD Veterans Aff airs Supportive Housing (VASH) v oucher program, Tenant Base d Rental Assistance program, and Family Unification Program (FUP) vouchers. Housing Catal yst also develops new affordable housing throughout Fort Collins for a variety of income levels In 2011 new federal funding became available in the form of bonus fun ding through the Continuum of Care process which is a HUD homeless assistance funding stream that allocates dollars to communities in alignment with the strategies outlined in the HEARTH Act. Housing Catalyst applied for funds that would provide ongoing operations support for a

PAGE 54

46 new permanent supportive housing community. Additional grant m onies were secured, along with low income housing tax c redits, to develop a 60 unit apartment complex, with 40 of those units dedicated as permanent supportive housing. This apartment complex, Redtail Ponds, secured its Certificate of Occupancy in Spring 2015, and residents moved in beginning in March 2015. Individuals occupying the 40 permanent supportive housing apartments all moved directly out of homelessness with so me previously staying primarily in emergency shelters some primarily staying outside in unsheltered areas, and others staying in buildings unfit for human habitation. All residents in the permanent supportive housing units at Redtail Ponds have a diagnose d disabling condition whether physical, medical, mental, or behavioral The building is staffed 24 hours a day, seven days a week with, at minimum, a security person. Additional full time staff include front desk staff, a property manager, a program manag er, a case manager funded through the local community mental health center ( SummitStone Health Partners ), and a case manager for HUD VASH clients The full time staff work closely with tenants to identify and arrange for additional services including medic al check ups, job training and employment coaching, or behavioral health treatment. Redtail Ponds integrates many Housing First practices in its design and operations. The lease generally mirrors those used in other traditional affordable housing units, w ith expectations for tenancy revolving around paying rent on time, maintaining one's unit, and generally being a good neighbor. In line with Housing First, sobriety is not required for residents to stay in compliance with their lease. For the first two ye ars of operations, any individual meeting the qualifications of the building (for example, inc ome falling between zero and 30 percent of area median income,

PAGE 55

47 currently homeless and living with a disability) could apply to be a resident P rioritization how ever, was given to those applicants referred from partnering service provider s including the com munity mental health agency, a local emergency shelter, and the Department of Veteran Affairs Beginning in Summer 2017, Redtail Ponds began receiving referrals through a community wide process called Coordinated Entry. Rather than every housing project maintaining their own waitlist and referral process, Coordinated Entry consolidates those separate lists into a single community w ide list on which individuals are organized not on a first come, first served basis, but instead by vulnerability. Those with the most severe disabilities or the longest length of time homeless are now referred first into any available housing units. Housi ng projects funded with Continuum of Care funding such as Redtail Ponds, are required to accept referrals from the Coordinated Entry process existing in a community beginning in early 2018. Those residents interviewed for this study were selected as tena nts at Redtail Ponds prior to the Coordinated Entry process. Therefore, every tenant had been referred from a partnering service provider Particularly for the first group of tenants moving in, Housing Catalyst favored selecting residents who remained in g ood standing with the referring service provider Accepting tenants with positive referrals from these partne r service providers was intended to increase the likelihood of success of Redtail Ponds specifically in those first months as staff were discoveri ng how best to manage the property and the community was keeping a close eye on how well the project would blend in to the surrounding neighborhoods As the first true permanent supportive housing project in Fort Collins, success at Redtail Ponds can help t o build community and political will for future permanent supportive housing developments.

PAGE 56

48 Redtail Ponds provides an ideal environment for exploring the research question outlined in this thesis for three reasons. First, given the stigmatizing nature of h omelessness and of mental health and substance use disorders, residents of Redtail Ponds will have direct experience of living in one of the most marginalizing situations one can experience. Second, while anyone who has been homeless may be able to recount the experience and the perspectives it provided, residents at Redtail Ponds will have recently moved out of homelessness. Those experiences will be fresh in mind, but with being housed currently they will also have the experience to comment on the evalua tion and performance measurements of the housing project of which they are now residents and know well Third, because of the federal funding supporting operations of the building Redta il Ponds operates in a way that aligns with common best practices prom oted by HUD and collects data for federally established performance measures for permanent supportive housing. HUD requires three standard performance measures that each funded permanent supportive housing project must report annual ly The first performance measure relates to housing retention for the project overall, wi th projects expected to keep 80 percent of residents housed for at least one year. The second and third performance measures relate to changes in income, with the second performanc e measure focused on overall income (including benefits) and the third performance measure focused on earned income. These performance measures are narrow in scope and may not capture the breadth of measures that residents in permanent supportive housing perceive as capturing success. In addition to each housing project funded by HUD collecting and report ing on specific performance measures, data from the region as a whole receiving fu nding (the Continuum of Care) are also collected and submitted to HUD. T he highest performing

PAGE 57

49 Continuums of Care are more likely to have all projects within that region funded each year. While the income related measures for permanent supportive housing deem either maintained or increased income of residents a success, the sys tem level performance measures a Continuum of Care must collect focus on employment and income growth. As a whole, all of the funded housing projects in a Continuum of Care should be working towards increasing residents' income and working with residents t o secure employment. Housing Catalyst must collect and report the performance measures outlined by HUD to remain in compliance with grant funding expectations. In addition to these performance measures, Housing Catalyst also has specified the following br oad descriptors of success for Redtail Ponds: 1) "A project the whole community can be proud of" and, 2) "A safe and beautiful housing community" ( Housing Catalyst 2013a). These reflect Housing Catalyst's general commitment to good neighbor practices incl uding developing and managing attractive, safe housing communities; holding residents accountable to certain standards of conduct respectful of the surrounding neighborhoods and neighbors; and, communicating openly with area residents regarding any relevan t development or management concerns ( Housing Catalyst 2015). To create the expectations for operating as a "good neighbor" and creating a safe and beautiful housing community that the entire community can be proud of, Housing Catalyst developed a Good Ne ighbor Statement of Operations for Redtail Ponds in conjunction with area neighbors. This document outlines general management practices for the building, tenant selection processes, and measures taken to ensure safety and security of residents and surroun ding neighborhoods ( Housing Catalyst 2013b). In addition to the HUD required and locally developed performance measures and goals, more research has been conducted in recent years on the effects of permanent

PAGE 58

50 supportive housing and the best practices for m anagement. Supportive housing projects have been able to demonstrate a reduction in the use of more expensive emergency services, including hospitals, jails, and shelters, once individuals are living in permanent housing. In addition, having robust support ive services available voluntarily is a standard best practice. Successful supportive housing projects are able to find creative ways to engage residents who may be reluctant to participate in any traditional case management or therapy. There is also an in creasing emphasis on permanent supportive housing projects to accept those residents who are highly vulnerable and chronically homeless upon move in. All of these components point to the following profile for a "successful" permanent supportive housing pro ject: A project which accepts highly vulnerable and chronically homeless residents; which creatively engages residents in services while not requiring participation in case management, therapy, or other services; which reduces the impact to emergency rooms jails, and other emergency services; which keeps residents housed in permanent housing for at least one year; and, which assists residents in, at minimum, maintaining their income if not increasing their income. Redtail Ponds also holds itself to the sta ndard of being a good neighbor and a beautiful and safe housing community. Previous research has shown that the preferences for performance measures established by public officials differ from what the citizenry deem s important and relevant. No research has specifically examined how these preferences differ when comparing public officials to a marginalized citizenry. To begin to explore these differences, this research centers on the following question: How do preferen ces for performance measures differ between public sector representatives at the federal, state, and local levels, and the

PAGE 59

51 marginalized public served? To inform this research question, the following sub questions are also included in this research: 1. How do marginalized citizens and public sector professionals differ in the importance assigned to performance measures based on their impact a t either the individual level, the system level, or the community level? 2. How do public sector professionals' and m arginalized citizens' understanding of performance measures (i.e., what information they can provide or what change they may indicate) differ? Two populations were involved in this research study. People living in permanent supportive housing at Redtail Po nds represented the marginalized public served. Each person moving into permanent supportive housing has experienced homelessness and been politically marginalized. Professionals working in the field of affordable housing or homelessness, with familiarity of the goals, intentions, and basic program design of permanent supportive housing represent the public sector professionals. Focusing specifically on individuals living in permanent supportive housing and the professionals who work closely with these hous ing projects also has the additional benefit of relating to the well defined performance measures established by federal funding streams supporting the development and operations of permanent supportive housing.

PAGE 60

52 CHAPTER III RESEARCH DESIGN As outlined in the previous chapter this dissertation addresses the following research question: How do preferences for performance measures differ between public sector representatives at the federal, state, and local levels, and the marginalized public served? The prem ise behind this research question extends from findings within the limited literature on citizen engagement efforts in performance regimes. When citizens are called to the table, most communities find that the goals and values outlined by c itizens differ from those outlined by employees and officials within the government (Cohn Berman, 2008; Cohn Berman, 2012; Spr ay Kinney, 2008; Woolum, 2011). The public generally prefers those measures related to outcomes rather than basic outputs, and val ues measures of service delivery, such as responsiveness. The minimal research that has been conducted on citizen engagement in performance regimes has not examined the impact of engaging individuals with limited po litical efficacy, who are often marginali zed and excluded from participating in public processes. These populations were at the center of some of the first federal efforts to engage citizens in public program planning. For those individuals who have been marginalized given their lived experience of homelessness, t o better understand how their perspectives on performance measures for the supportive housing in which they reside differ from those deemed most important by professionals working in the field, two additional sub questions support the mai n research question:

PAGE 61

53 1. How do marginalized citizens and public sector professionals differ in the importance assigned to performance measures based on their impact at the individual, system, or community level? 2. How do public sector professionals' and ma rginalized citizens' understanding of performance measures (i.e., what information they can provide or what change they may indicate) differ? Sampling and Recruitment This research examines the perspectives of both public sector professionals and members of the marginalized community of people with experiences of homelessness. To examine the perspectives of the marginalized community members, t he permanent supportive housing community of Redtail Ponds in Fort Collins, Colorado serves as the res earch site The selection of this site is both purposive and convenient ( Singleton & Straits, 2010; Teddlie & Yu, 2007) The site is convenient given t he researcher 's knowledge of and connections to Redtail Ponds through her previous professional position of Director of Homeward 2020, a non profit initiative that drives the implementation of Fort Collins' 10 Year Plan to Make Homelessness Rare, Short Lived, and Non Recurring. In this role, the researcher advocated for the development of Redtail Ponds and wo rked daily with staff of Housing Catalyst the entity that developed, owns, and manages Redtail Ponds. Given the working relationship between the researcher and the staff of Redtail Ponds, the process to obtain permission from Housing Catalyst to c onduct research at Redtail Ponds was without complication The site was selected purposively given its alignment with common practices for permanent supportive housing and the federal funding awarded to support ongoing

PAGE 62

54 operations of Redtail Ponds. While s ome other permanent supportive housing resources exist in Fort Collins, including housing vouchers that can be used on the private rental market with access to supportive services, Redtail Ponds was designed and developed to reflect the most up to date bes t practices for providing permanent supportive housing to people moving out of homelessness. Staff of Housing Catalyst and from other non profit organizations providing supportive on site services visited multiple projects throughout the country regarded a s models for permanent supportive housing prior to designing and developing Redtail Ponds These site visits and additional extensive research conducted led to intentional design considerations to best meet the needs of residents. For example, ceilings are high to provide a more open feeling in apartments, which is expected to aid the transition from people accustomed to sleeping outside to sleeping inside. Ample open, public space is available to foster a sense of community in the building. In addition, t h ere is a single point of entry to the building, and the building is staffed 24 hours a day and seven days a week This ensures somebody is available if a resident is in need of support or assistance at any time of day. It also serves as a buffer for some r esidents who may be trying to distance themselves from previous relationships, as the front door staff can refuse entry to those visitors on behalf of residents who may find it difficult to turn those acquaintances away. As a result of these intentional d esign and staffing elements, Redtail Ponds has been highligh ted in workshops on developing p ermanent supportive h ousing, supported by the Colorado Governor's Office, the Colorado Housing and Finance Authority, and the Colorado Division of Housing. In addit ion to these elements that were voluntarily incorporated into Redtail Ponds and that have led the project to be acknowledged by state level housing entities, Redtail Ponds receives an annual, renewable operating grant from the Department of

PAGE 63

55 Housing and Urb an Development (HUD) through the Continuum of Care process. This funding requires Redtail Ponds to measure and adhere to the performance measures set by HUD. For these reasons, Redtail Ponds serves as an ideal example of permanent supportive housing, selec ted purposively for this research To examine the perspectives of public administrators operating at the federal, state, and local levels, representatives from various public and non profit agencies were also interviewed. Interviewed professionals held varying functions in the homelessness and housing field, including as administrator s of permanent supportive housing, as funder s of permanent supportive housing, or as capacity builde rs Capacity builders work in public and n on profit agencies providing training, technical assistance, and other support to communities or agencies in developing permanent supportive housing or generally working to end homelessness. Professionals interviewed also represented different levels of go vernm ent and different geographic purviews, working at either the local, state, or federal/national level. These interviews were conducted with the intention of collecting data from those working in the field, to serve as points of comparison to the findin gs from the residents' interviews. Study Participant S election Of the 60 apartments in Redtail Ponds, 40 are designated as permanent supp ortive housing, with the other 2 0 apartments designated as traditional affordable housing offered for individuals at a slightly higher area median income than those units designated as permanent supportive housing Only the permanent supportive housing residents are required to be moving out of homelessness to be eligible for tenancy at Redtail Ponds Tenant s of the permanent supportive housing units are also required to have a disability to be eligible for

PAGE 64

56 residence. While these are requirements only for the 40 permanent supportive housing units, the majority of the first tenants in the 20 affordable housing units al so had experiences of homelessness. Most of the permanent supportive housing residents living in Redtail Ponds at the time of the interviews had been referred to the housing project by either the c ommunity mental health center, a local emergency shelter o r the Department of Veteran Affairs (for those units reserved for veterans because of the type of housing voucher associated with that unit) Therefore, most residents at Redtail Ponds are individuals who had been engaged with other service providers, had demonstrated they could comply with program rules an d requirements, and were deemed suitable referrals to Redtail Ponds When determining if someone is suitable for tenancy at Redtail Ponds, service providers may consider someone's perceived stability, ten dencies for violence, or ability to live in close quarters to others. Given this tenant referral process which intentionally screened in individuals who staff felt could live successfully in a communal environment, follow rules, and generally be a good ten ant, a selection bias does exist for the pool of eligible study participants. The tenant selection process to Redtail Ponds is now more objective, with new tenants being referred from one common community wide list, with little to no consideration for tena nts' perceived suitability for the project Therefore, r epeating the interviews conducted in this study in the future may yield different results given the potential for a different, less engaged population to be housed at Redtail Ponds. Given the margin a l izing effects of homelessness and the performance measurements attached to the permanent supportive housing units, the 40 residents living in units designated as permanent supportive housing units were all eligible t o participate in this research as

PAGE 65

57 inte rview subjects. Within the pool of eligible interviewees, a convenience sample was used to select individual participants (Onwuegbuzie & Leech, 2007), as residents volunteered to be interviewed. The opportunity was advertised through publicly posted flyers Front desk staff at Redtail Ponds managed the sign up sheets for the interviews. Resident participants received a twenty dollar stipend for their participation. To avoid any potential issues with affecting a resident's reported income, the stipend will b e paid in the form of a gift card. A total of 24 of 40 eligible residents of Redtail Ponds completed interviews, in line with other s tudies involving interviewing subjects who are experiencing or have experienced homelessness which have sample sizes ranging from 16 to 30 interviews (Dang & Miller, 2013; Collins & Barker; 2009; Kennedy, Grewal, Rober t s Steinauer, & Dehlendorf, 2014 ; Neale & Stevenson, 2014). The sample of professionals interviewed was also selected purposively and for convenience ( Singleton & Straits, 2010; Teddlie & Yu, 2007). To have a broad range of perspectives, representing both those working daily to manage permanent supportive housing as well as those with more removed positions as funders or capacity builders, the researcher strategically targeted those who could offer these perspectives as well as variously represent those working at the local, state, and federal/national levels. Given the researcher's professional position in the field, she was able to directly reach out to other professionals who fit the criteria of the research subjects she sought. A total of 14 professionals were interviewed. Five worked at the local level, four at the state level, and six at the federal/national level. Given that professional interview s ubjects were selected based on their availability and, often, their existing working relationship with the researcher, a sampling bias also exists for these interviews. Randomly sampling professionals working in different

PAGE 66

58 capacities that mirror the criteri a established by the researcher may yield different results. Between resident and professional respondents, a total of 38 interviews were conducted for this research. Data Collection and Measurement Strategy One on one semi structured i nterviews were the primary data collection strategy for this research both for the residents of Redtail Ponds and for the professionals For those residents of Redtail Ponds, one on one interviews provided the privacy and ability to engage in a dialogue regarding different performance measures that focus groups or surveys would not have allowed. Interviews with residents typically lasted between 30 minutes and one hour, depending on the verbosity of each individual. Prior to conducting the interviews with the Redtail Ponds residents, the interview instrument was piloted with three residents at Redtail Ponds residing in the 20 affordable housing units. All three individuals had comparable histories and experiences with homelessness as those residents interviewed for this res earch. These pilot interviews provided an opportunity to refine the wording and order of questions, clarify instructions for the interview, and ensure the wording of questions would yield useful data. Interviews with residents were all conducted at Redtai l Ponds in a vacant office Conducting all interviews at Redtail Ponds ensured all interview subjects could physically participate in the interview by eliminating the need for residents to arrange transportation to an off site location This location also ensured interview subjects were in a comfortable and familiar e nvironment, which could f acilitate conversation better than a strange, uncomfortable or unknown location.

PAGE 67

59 All interviews with residents were conducted during regular business hours in which mu ltiple staff members were available. While no interview subjects expressed the desire or need to speak with any staff member as a result of completing the interview, all staff members were aware the interviews were taking place, and were available if this need did arise during or after the interview Prior to each interview with residents the researcher first described the purpose of the interview and how responses would be used and reported Anonymity was ensured to all participants, and residents were all informed that their participation was voluntary. Given all residents at Redtail Ponds are disabled (many with mental health or substance use disorders) during this introductory time, the r esearcher was also able to ask questions of the resident to gauge whether she or he was coherent and adequately understood the purpose of the research to participate. No residents were excluded from the research project due to an inability to comprehend th e purpose of the study and their role in the research. Subjects were also provided the opportunity to ask the researcher any questions during this introductory time Every interview subject was asked to review and sign a n IRB reviewed and approved consent form prior to beginning the interview process. Residents were also asked if the researcher could digitally record the interview. All but one resident permitted their interview to be recorded. Verbatim t ranscriptions of the interviews along with a ny notes taken by the researcher during the interview process, serve as the data sources analyzed for this research Interviews followed a semi structured format Interviews began with the researcher gaining basic information from the residents including a ge, veteran status, and the length of time residents had been living at Redtail Pon d s. Residents were then asked to comment and

PAGE 68

60 reflect on those things they thought made Redtail Ponds successful, and those things they thought made Redtail Ponds less succes sful and could be improved upon. This provided the researcher with a general sense of how residents viewed success for the housing project. The next section of the interview involved residents ranking (on a five point Likert scale) the importance of nine s eparate performance measures, pulled from relevant literature and guidance on permanent supportive housing. These performance measures were described in a way that would be understandable to a lay person who did not have technical expertise in the field of homelessness. For each performance measure, the researcher also asked the interview subject to comment on why they assigned the performance measure the score they did and what information they thought the performance measure would provide. The order of t he performance measures was randomly generated for each interview so any consistencies in responses would not be due to the order in which the performance measures were presented. The researcher asked residents to specifically comment on the performance me asures specified by HUD, asking if those measures could adequately determine whether a housing project was successful or not. Finally, residents were asked their opinions on participating in efforts to establish performance measures. The full interview too l for residents is included as Appendix B Interviews with the professionals occurred either in person or over the phone depending on the location and preference of the interview subject Interviews generally lasted between 30 minutes and one hour. Given that these interview subjects participated in these interviews in a professional capacity, they were not required to sign a consent form and no incentive was provided for their participation. Prior to beginning each interview, the researcher explaine d the purpose of the research and generally how findings would be

PAGE 69

61 presented. Interview subjects were given the opportunity to ask any questions prior to beginning the interview process, and were asked if the interview could be digitally recorded. All profe ssionals interviewed permitted recording the interview. Verbatim t ranscriptions of the interviews along with any notes taken by the researcher during the interview process, serve as the data sources for this research Each interview began with the researc her soliciting basic information on the professional's background and experience in the fields of affordable housing and homelessness. Professional s were then asked to comment on how they and their colleagues use performance measures in their work. Followi ng these questions, professionals were provided with the same list of nine performance measures and the same f ive point Likert scale that was provided to the residents interviewed. As all professionals were working in the fields of housing or homelessness, the performance measures were presented in the technical language more commonly used in writings or guidance for professionals in the field. As with the residents interviewed, the order of the performance measures was randomly generated for each professi onal so any consistencies in responses would not be due to the order in which the performance measures were presented. The interview subjects first offered their initial, quick ranking on each of the nine performance measures. Then, the researcher prompted the interview subject to revisit each performance measure and talk through their rationale behind assigning the ranking they did to the measure in question. This process of revisiting the performance measures instead of asking for additional explanation when the professional initially ranked the measure served two purposes. First, it ensured if an interview was cut short, since these were occurring during regular work hours when other work emergencies may come up, the researcher would still have some data for all

PAGE 70

62 performance measures with which to work. Second, it allowed for the interview subjects to have some space between their initial gut reaction to a performance measure and their more thought through response to explain their reasoning behind each ra nking. This enabled the researcher to see if any professionals may change their initial thoughts and for what reasons. Asking for a ranking and reasoning all at once may not have provided as much time for professionals to revisit and reflect on their thoug ht process behind each ranking. After receiving rankings and explanations for each of the nine performance measure s the researcher asked each professional if there were other performance measure s the professional considered to be critical to collect and which performance measures would be the three most important to collect. The researcher also asked them to comment specifically on the HUD required performance measures and whether those could adequately gauge a permanent supportive housing project's level of success. Interview subjects were also asked to guess which performance measures they thought the residents interviewed would co nsider to be the most important, providing information on how professionals' assumptions regarding residents' preferences may differ from or align with residents' actual stated preferences. Finally, professionals interviewed were asked their opinions and advice on engaging residents in helping to establish outcomes or performance measures for permanent supportive housing. The full interview tool for professionals is included as Appendix C E xplanation of Performance Measures Nine performance measures related to permanent supportive housing were included in the interviews with both the residents and the professionals. Two of the performance measures discussed (retention in housing and increas ed income) are directly related to those outcomes HUD requires of funded permanent supportive housing projects. Retention in

PAGE 71

63 housing measures whether people are able to stay in their housing or move on to other permanent housing. For a population that cycl es frequently in and out of homelessness and has experienced homelessness for an extended period of time, simply providing housing that enables people to remain housed is considered a success by HUD's standards The performance measure related to increas ing income has shifted over time. A commonly accepted approach to providing services and housing to people experiencing homelessness used to center on a "housing ready" philosophy in which people were placed into transitional housing, would re learn how to be housed and employed, and then graduate to permanent housing. Aligning with this framework, HUD focused on increasing individuals' income primarily through employment, even while acknowledging that some individuals experiencing homelessness were living with severe disabilities that limited their ability to work. For instance, in the Annual Performance Plan for Fiscal Year 2006, HUD explicitly stated that "[e]mployment is a critical step for homeless persons to achieve greater self sufficiency" (p.121) an d developed employment related performance measures for projects (HUD, 2005). With greater research and evaluation of homeless services and housing, the standard now centers on a housing first philosophy in which individuals are provided with access to pe rmanent housing without preconditions, such as an income. With this shift to a housing first approach, there has also been an emphasis on serving those who are the most disabled and for whom self sufficiency or employment may not be realistic goals. As the general framework through which HUD operates has shifted, their emphasis on employment has somewhat dissipated. Now, permanent supportive housing projects report on income through various measures, including reporting total income (from employment and pub lic benefits) as

PAGE 72

64 well as broken down by earned or employment income and income from benefits. The performance measure "increased income" used in this research aligns with HUD's general emphasis on individuals accessing, maintaining, and ideally growing tha t income while in HUD funded housing or services. The remaining seven performance measures come from relevant literature and current best practices in the field of housing and homeless services. The performance measures selected also reflect a variety of i mpact levels. Some performance measures are geared more towards measuring individual level change O thers best measure change occurring with in the housing project or within the homeless services system in a community. Still others measure change at the community level. A summary of each of the nine performance measures selected, level of impact, and general descriptio n are included in Table 1 T able 1. Overview of Performance Measures Performance Measure Description Level of Impact Voluntary participation in case management or other services Measures the degree to which residents voluntarily participate in services provided Individual Increased incomes of the residents Measures whether residents can increase their income either through employ ment or benefits Individual Resident satisfaction Measures the satisfaction level of current residents Individual Retention in housing Measures whether the project retains most residents in that housing or assists them in finding other permanent housing System Chronicity and/or vulnerability of new residents Measures whether the project is serving those who qualify as chronically homeless or who are the most vulnerable, based on an identified set of criteria System Reduced shelter bed days Measures whether the project alleviates the need for emergency shelter for residents, with the assumption that providing enough housing will reduce the overall need for shelter System

PAGE 73

65 Table 1. Overview of Performance Measures cont'd Performance Measure Description Level of Impact Reduced jail bed days Measures whether by providing permanent housing, the frequ ency of jail stays is reduced for those residents Community Reduced emergency department (ED) visits Measures whether by providing permanent housing, the frequency of emergency department visits is reduced for those residents Community General community satisfaction Measures the satisfaction level of the surrounding community or neighborhood in which the project is situated Community A nalysis Strategy This research aims to answer broad research questions related to the preferences for performance measures of marginalized citizens a nd professional working in the fields of homelessness or housing Interviews with current residents of per manent supportive housing as well as professionals served as the primary data sources. A major portion of each interview involved the interview subject reflecting on and responding to questions about nine identified performance measures for permanent suppo rtive housing. All interview subjects were also asked to rate each performance measure on a scale from one to five based on how important the interview subject felt the performance data would be to collect. In addition, interview subjects were asked to sel ect the top three most critical performance measures from those provided. These rankings and the request for interview subjects to identify their top performance measures allowed the researcher to tabulate frequencies and mean scores on each performance me asure, providing a general sense of the importance the interview subjects assigned to each performance measure. The researcher then relied on constant comparison analysis (Leech & Onwuegbuzie, 2008) to identify other themes emerging from the verbatim transcriptions of the interview s

PAGE 74

66 While often used for grounded theory research, this technique was appropriate given the intent of the research to discover a new themes and trends from the interviews The researcher coded each interview, with ea ch identifiable theme uniquely coded. Once all interviews within each subject group (residents or professionals) were coded, the researcher reviewed each interview's coding to ensure no themes that emerged in later inter views were overlooked in the initial interviews. This process also enabled the researcher to identify codes that were similar enough to merge into a single overarching theme. In addition, those portions of the interviews in which respondents were speaking specifically to one of the nine perf ormance measures were coded as such to allow for a quicker review of all responses and comments related to each pe rformance measure. A codebook of those themes that emerged is included as Appendix D When analyzing all frequencies, means, and codes for th e interviews, the researcher referenced those characteristics that may have impacted responses, such as residents' veteran status or gender, or professionals' length of time in the field or experience with direct service. Given the sample size, little vari ation was identified when considering these characteristics. Where viable, these variations are noted in the research findings. In addition to the interviews, the researcher reviewed selected documents published by HUD related to performance goals and outc omes for the department. The researcher coded segments of these documents related to HUD's description of housing and services for people experiencing homelessness, with a specific focus on how HUD described success for individuals served in these funded p rojects and how success was define d for HUD's funding priorities. For example, the documents were reviewed to determine if HUD described success for individuals and funded projects based on residents' ability to be "self sufficient,"

PAGE 75

67 indicating HUD expects individuals to reduce their reliance on housing or other government assistance; or on residents' ability to be "stable, indicating HUD finds value in simply having residents maintain housing and a consistent quality of life, without an expectation that t hey will reduce their reliance on government assistance. The findings from this document review are intended to serve as a point of reference for and comparison to the statements made by interview subjects, particularly professionals who were asked in grea ter detail about their understanding or knowledge of how any HUD required performance measures have been developed or why they believe HUD has adopted the performance measures currently in use.

PAGE 76

68 CHAPTER IV RESULTS Characteristics of Respondents This research is informed by interviews from 2 4 residents living in permanent supportive housing at Redtail Ponds in Fort Collins, CO and from 14 professionals with knowledge, expertise, and experienc e around housing and homelessness All interviews with reside nts at Redtail Ponds were conducted between February and April 2015. Interviews with professionals were conducted between December 2015 and October 2016. Among the residents of supportive housing interviewed, the majority identi fied as male, with only six of the 24 respondents identifying as female. Six residents self identified as veterans; one resident opted not to disclose his veteran status. This corresponds to approximately one quarter of all residents surveyed who have served in the military. Overall, a t least 38% of all residents in Redtail Ponds' supportive housing units at any time are veterans as 15 of the 40 units are dedicated to veterans. The ages of respondents ranged from 31 60, with a mean age for respondents of 51 years old. The length of time residents had lived at Redtail Ponds ranged from one week to over a year, with a mean length of time of around eight and a half months. Residents varied considerably on the length of time they had experienced homelessness prior to moving into Redtail Ponds. The shortes t length of time of homeless reported was around three months, with the longest length of time reported of 19 years. The average length of time homeless among residents prior to moving into Redtail Ponds was over three years. Most reside nts were able to recall the approximate length of time they had spent homeless, however many residents who had had multiple bouts of homelessness or had

PAGE 77

69 moved frequently in and out of housing had more difficulty recalling with much certainty the length of time they had experienced homelessness overall In addition, some people's experiences were limited to those times they were staying outside or in shelters, aligning with HUD's definition of "literal" homelessness, while others included stays with friends or family. Given the varying definitions people used to define their own homeless experiences and the limitation of relying on self reporting, the three year average length of time is only an approximation. The housing professionals interviewed worked at varying levels in the public sector (e.g., local, state, or federal) and in different capacities. Table 2 provides breakdowns of the characteristics of the professionals interviewed. Table 2 Number of Housing Professionals Interviewed, by Level of Autho rity and Role Administrator Funder Capacity Builder Local 3 2 0 State 0 2 2 National/Federal 0 1 4 Those professionals working as "administrators" are those directly responsible for overseeing, managing, and staffing supporti ve housing. All administrators a re employed at a local level by Housing Catalyst, the owner and manager of Redtail Ponds. The "funders" are those representing agencies that regularly fund supportive housing, including both development costs and the ongoing operations and services costs. "Capacity builders" are individuals responsible for supporting and building capacity in the state and in local communities to develop and manage supportive housing. Three of the four capacity builders operating at the national/federal level s work for organizations with a formal relationship with HUD, providing contracted technical assistance and training to communities receiving HUD funding for housing and services dedicated to people experiencing homelessness.

PAGE 78

70 Three of the housing profess ionals interviewed work for non profit organizations (all falling into the national capacity builder category). The three local administrators all work for a public housing authority, a quasi governmental agency. All remaining housing professionals intervi ewed work for public agencies at the local, state, or federal levels. A total of six out of the 14 professionals have had direct service experience at some time in their career, meaning they have worked directly with clients who are experiencing or have e xperienced homelessness, often as a case manager or program manager. Three out of those six with direct service experience are currently working directly with clients as local administrators of supportive housing. The majority of the housing professionals interviewed have worked at least 15 years in the fields of affordable housing or homelessness services. Only two out of the 14 professionals interviewed have worked less than five years in the field. Summary of Performance Measure Rankings for Residents Gener ally, residents were able to assign scores along a Likert scale reflecting their perceived importance of collecting those performance data. The Likert scale provided was a five point scale, with a "one" described as "not at all important," and a "five described as "absolutely important or critical." Three respondents provided feedback on what they thought of each performance measure and generally whether it made sense to collect or not, but opted not to use the Likert scale. In addition, occasionally other respondents would avoid assigning a score to a specific performance measure, largely those in which they did not see the relevance in collecting those data. A summary of the scores assigned by residents is included a s Table 3

PAGE 79

71 T able 3 Resident s' Scores on Nine Performance Measures Performance Measure Mean Score # Residents Scoring a 5 on the Scale # Residents Ranking in Top Three Minimum Score Awarded Maximum Score Awarded n Retention in Housing 4.53 12 14 3 5 20 Chronicity/ Vulnerability 4.23 8 11 2 5 20 Resident Satisfaction 4.11 7 9 2 5 21 Reduced Jail Bed Days 4.00 6 7 1.5 5 21 Increased Incomes of the Residents 3.96 7 4 2 5 21 Reduced Shelter Bed Days 3.78 4 3 1 5 20 Reduced Emergency Dept. Visits 3.73 7 3 1 5 20 Community Satisfaction 3.71 7 7 1 5 21 Voluntary Participation in Services 3.67 6 4 1.5 5 21 Housing retention Residents ranked "retention in housing" high and generally considered it an important outcome to collect. Twelve out of 20 residents assigning a score to this outcome measure ranked it as a five "absolutely important or critical." In addition, 14 of the resident r espondents included retention in housing in their top three performance measures to collect.

PAGE 80

72 This performance measure also had the narrowest range, with no resident ranking the measure less than a three on the Likert scale ("fairly important"). The most prevalent theme among residents related to the performance measure of housing retention related to the core purpose of permanent supportive housing: providing a long term or permanent place to live. Some connected this to the basic principle that it would allow people to leave homelessness and avoid having to return to that life. One resident also mentioned that this was important because it signified a "whole different way of looking at [homelessness]" by providing access to permanent housing with out preconditions, i n line with housing f irst practices. Residents also mentioned that knowing people were choosing to stay could indicate they were happy comfortable, or felt safe in their apartment and in the building: There's so many fears in a person that's homeless. And to be able to establish a safe r esidence kind of cuts all that out. You feel safe, you know. You know where your next meal's going to come from. You know you're going to wake up in the morning and have a nice hot shower. You know, all those things fall right into place. So, I'd say that's the most important. Having a stable place to live could also allow people to work on what they needed to and get the help they needed. One resident described the difficulty in trying to do anythi ng other than manage the crisis of homelessness when you are without stable housing by noting that w hen you're homeless, your head's just not in the right place to get help. You're just too stressed trying to survive. First having a permanent apartment could provide the st ability and ability for people to focus on other needs and goals. One resident reflected on his recognition of his inability to stay healthy and care for himself without his apartment at Redtail Pond s, even in the face of being uncertain about staying:

PAGE 81

73 I've been struggling with it for like the last couple weeks. I've been packing my pack and unpacking my pack. Packing my pack and unpacking my pack. Am I going to stay? Now I have a heart problem, tore up shoulder. I've had my spine fused since I've been here. My neck -I've got a cracked vertebrae in my neck. And I've been ready to walk out the door. But then I realize if I do that, that the physical condition I'm in is just going to get worse and wor se. So, bite the bullet, c ontinue to ride the experiment, and we'll see what happens. For some, that opportunity to take care of their existing needs may also set them up to think forward into the future and consider plans for eventually moving on from Re dtail Ponds. One resident described the differences he saw in his neighbors' goals and intentions: [F] or some people, this might be permanent. Or for some, it might be a stepping stone to getting something better. And, if those people that choose to move out, if you keep track of it or whatever, then you know that they're doing well, then you were a stepping stone to them doing well. In discussing this performance measure, many residents also indicated this housing may not be an appropriate fit for all. O ften, issues with ongoing alcohol or drug use, behavioral issues that affected others or an inability to follow rules were cited in residents' reasoning that Redtail Ponds was not the right place for some. Redtail Ponds does not prohibit alcohol use, and staff often go to extensive lengths to avoid evicting residents, i n line with best practices for housing f irst oriented permanent supportive housing. Some expressed frustration with the perceived leniency of staff, wondering for those who continue to heavily drink, how will their life be better here than it was on the street, other than they have a roof over their head ?" Others acknowledged that for th ose who do end up leaving or being evicted, it simply may not have been the right living situation: [B]asically if somebody can't [live] here, it's either because the facility isn't up to the challenge or the resident isn't. And so that's a case where you assume that we're following all the HUD guidelines. At least ap proximating them well. And then, if that person isn't able to make it in that environm ent, then that obviously wasn't the right environment to cultivate th at sustained period of whatever [e.g., sobriety].

PAGE 82

74 C hronicity and vulnerability Chronicity and vulnera bility of new residents was the second highest ranked performance measure, based on mean score awarded. Eight of the 20 residents responding rated this measure a five, absolutely important or critical, and 11 residents placed this measure in their top three most important measures to track. Scores for this performance measured ranged from two to five on the five point Likert scale. The most commonly cited benefit stated for track ing the chronicity and vulnerability of residents related to how this information could be used to better serve residents. If staff knew what illnesses, disabilities, or other circumstances someone faced, they could provide the right kind of help and assis t residents in meet ing their needs. One resident noted how staff having this information had helped him when he moved in: [W] hen I first got here David (the property manager) mentioned to me that if I needed a railing in the showers, he would have it inst alled. Yeah, if there was someone who was handicapped or needed something like that, they would certainly need to know that. Another common theme similarly dealt with the need for staff to know about people's disabilities and illnesses, but more for reas ons of safety and more surrounding issues of mental health and substance use disorders. If staff did not understand people's issues and illnesses, they may be "caught off guard" and unable to both help the person and keep other residents safe if something went wrong. One resident described the need for staff to be aware of people's illnesses for this purpose: [Y] ou're here to help people. But at the same time, you have to look out for everyone. So, I would want to know who's got what as far as like mental issues or maybe medical issues. That way you can try to explain t heir behavior or protect them. P rotect them and the other people that they have to live with.

PAGE 83

75 Some residents connected the potential issues seen with people with severe disabilities, incl uding mental illness, to suitability for the building, thinking this could be used as a "filtering point" to select new residents. These residents express ed a preference for a treatment first orientation to housing in which individuals with severe mental i llness or substance use disorders may be required to access treatment before being provided with permanent housing. This model is incongruent with the approach Redtail Ponds is taking (the housing first approach) in which everyone is ready for housing and anyone can access permanent housing without needing to show any evidence of treatment compliance or other forms of clinical success. Resident S atisfaction Resident satisfaction was the third highest ranking performance measure based on mean score. The range of scores assigned to this performance measure ranged from two to five. Seven of 21 residents responding ranked this performance measure as a five, absolutely important or critical, and nine residents included "resident satisfaction" in their top three most important performance measures. Many residents expressed the importance of people being happy, recognizing that "it's kind of the goal of everyone to lead a satisfied and happy life ." One resident also connected being happy to housing retentio n noting, "they want you to be happy so you can stay here." This comment mirrors some statements made by professionals in the field who felt that poor housing retention likely reflected that people were not happy or satisfied in their housing. What happine ss means to residents though, varied from person to person. Residents described happiness as being a reflection of people having privacy, people having a voice in how their housing operated, people not being depressed, people feeling safe,

PAGE 84

76 people feeling a sense of unity and camaraderie, and the project promot ing and reflecting a "healthy environment" in which to live. The perception of to what degree the housing project in which a person was living could impact their happiness differed among residents. A few residents recognized this would be a helpful performance measure to track as the housing project could change things if residents were generally unhappy in their housing. These may be programmatic changes or rules of the property. This may also be something small, however, such as how staff interacts with residents. One resident noted the impact having happy staf f has on residents stating, "It's al l up to [the staff]. I mean if they're having a bad day, then they're showing it. Then they're not going to make too many people happy that day. Other residents acknowledged the lack of control the housing project may have to affect someone's level of sat isfaction, noting "you can't make everybody happy" and "no one can make another person happy or unhappy." Other factors, including mental illness or past trauma people are dealing with, may also impact residents' satisfaction or happiness and are largely o ut of the control of staff. One resident illustrated this point, commenting on her own past experiences and trauma: I'm very unhappy, ok? But that has to do with my personal stuff and I realize that. But I see a lot of other residents that are just tickled pink to be here and have a chance. And me it's just, you know, like I said, working through. The tragedy that I've gone through is just pretty deep. Reduced Jail Bed D ays The performance measure relat ed to reducing the number of days spent in jail was ranked fourth by residents, based on mean score. Awarded scores ranged from one and one half to five. Six of 21 residents responding ranked this performance measure as a five, and seven

PAGE 85

77 residents placed t his performance measure in their top three most important measures to collect. The most common theme expressed by residents related to the changes that can come from someone having access to a permanent place to live. For some residents, a change in the n umber of jail bed days would reflect individuals changing their lifestyle or not engaging in the sam e behaviors as they did before, including alcohol or drug use. One resident described these changes seen in residents after moving out of homelessness, and what a reduction in jail use could indicate: Well, it'd show rehabilitative changes. See what they were doing before -if they were hanging out in Hobo Park, they were doing drugs, they were drinking. Now all of a sudden you got them into this facility, they're going to groups, they're doing mindfulness, and look at what happened. No more steel bracelets. So, and I've seen quite a few of them come in here who were real hellions on the street. Staff doesn't need to know that. They're now being real cool. Other residents who felt a reduction in jail bed days indicated what could change in someone's life from homelessness into housing, connected one's homelessness to "unnecessary" police contact and criminal justice involvement that results from being homelessness One resident described his own experience with spending time in jail while homeless: I ended up a week in jail for a disorderly conduct ticket that I probably wouldn't have gotten if I had a home. And I could've had my outburst in my own dom ain rather than having no place to go to d e f use. Another resident noted that people living on the streets get in fights and may end up in jail "because they're fighting over survival." In both of these examples, residents communicate criminal justice a nd legal issues that arise specifically due to one's homelessness. While some simply acknowledged that it generally would be better for one to not go to jail, others connected a reduction i n jail bed use to reduced costs to the community

PAGE 86

78 noting that "ever y time we put someone in jail, that costs a lot of money for the city and state." This cost argument was also a common theme among professionals discussing this performance measure. While many of these residents pointed to circumstances beyond one's contro l, two residents stressed that ultimately behaviors and choices are up to the individual. A change in the number of visits to jail would therefore reflect a person's choice to change their behavior. Others felt this would be a useful performance measure t o collect and track due to safety reasons. Knowing who had been in jail and still was going to jail would allow staff to identify the "criminal element" in the building or determine who should not be permitted to live in the building at all. One resident d escribed this concern for housing someone with a criminal history: I don't want those people doing whatever they do here and that not be known because they could hurt somebody else and they could be bringing other people in here and I guess as an apartment manager, a property manager, I wouldn't want those kind of people around. While there were no overriding themes related to why people placed less importance on this measure than others, two residents did comment on issues of privacy. Th ese residents felt you "don't need everybody knowing what you're doing and keeping track of people's visits to jail would violate residents' privacy. One resident also placed less importance on this performance measure given not all calls to police resulting in an arrest are "legitimate" but may instead reflect a "he said/she said" situation. Increased Income The HUD related performance measure related to income was not perceived as favorably as the housing retention measure the other HUD required measure Scores for "increases in income" ranged from two to five. Based on the average score for each

PAGE 87

79 performance measure, the income measure ranked fifth out of nine possible measures. Only four of 21 residents responding placed "increases in income" in their list of top three performance measures, coming in sixth and tied with "participation in services." The most commonly stated reason for residents that this performance measure is less important relates to how having income and increasing one's income demonstrates one's stability and the corresponding ability for that person to "move forward" with their life. Rather than being stagnant and content in their current state, having income and working towards increasing income also demonstrates a sense of p roductivity and can indicate people are learning to live and budget more responsibly. Some residents also connected their income (or lack thereof) back to personal feelings and emotions. For example, one resident noted having income was important to him b ecause it helped him know "I got power." Having income and knowing one can pay for the items they need to sustain, including their housing, can be a source of positive self esteem and pride. By having an income and paying for rent, one resident also felt t his helped her show her appreciation for her housing. One resident with limited income who was unable at the time of the interview to work towards increasing her income mentioned the emotions that come with having enough money to pay for your expenses and do the things that make life more enjoyable. Referencing her neighbors with more income, she stated, Like, socially, emotionally, they seem happier. I'm kind of jealous of them, because they always have money and I don't They always get to go do things, and I'm just like, Uh, I can't do that.'" Two residents mentioned how income is tied to stability, and saw an increase in income not as an indicator of the individual's stability but as an illustration of what was

PAGE 88

80 possible when one has the stability of permanent housing. It permitted people to focus on and be better equipped to seek and retain employment. While the increase in income was important, these residents framed this performance measure as a residual effect of sustained housing retention. Anothe r reason cited by multiple residents for tracking people's incomes related to issues around scamming and playing the system. Residents in Redtail Ponds are expected to pay one third of their income to rent. The remainder of the rent is paid by a Housing Ch oice Voucher. While some expressed issues of fairness and dishonesty if people did not disclose their full income, one resident also tied this back to the ability of the building to sustain and the effect on other residents if not all were forthcoming with their income amounts and sources: They got a scale that they use. And people could get around it real easy and that wouldn't be fair to the other tenants I think it's 30% of your income. And it'd be very easy to hide that income and not pay their way. Yo u know, they don't pay their way then we're going to start losing amenities and the help we get and the maintenance of the building. Just so many things involved in that. Somebody making $2 000 a month and only pays $50 a month rent. You know, they gotta t rack that. Make sure he's not stashing money somewhere. Residents stated a few reasons why measuring income and increases in income would not be the most useful or important measure to track. Several residents mentioned issues people may have with budget ing. Simply having more income would make little difference if people did not know how to use it well or spent it on items that were not necessities. One resident also mentioned that income was not a driving factor in his experience of becoming homeless. If one became homeless only because of financial reasons, that could be remedied by "getting a job or getting assistance." For this resident and others, there were other situations and factors contributing to their homelessness. If the intention was to hel p

PAGE 89

81 someone move out of homelessness and into a place of greater stability, those other factors and not income should be addressed and tracked. One resident did rate this performance measure lower for reasons that echoed those of the professionals interview ed. Many residents have disabilities or other barriers that may limit how much income they can earn. Tracking changes in income, then, may be of little value: There's some of these folks that are never really going to have a private, you know, independent income again. Some of these guys that are here are stric tly going to have benefits. They'll establish disability income if they haven't already and are going to have Social S ecurity and Section 8 housing, or whatever is going to assist them with housin g. And, they'll have their benefits, and that's it. So, as far as tracking income then, I'm not sure what purpose that serves. In these cases, measuring changes in income is less indicative of whether the project is successful and more indicative of the need for supportive, affordable housing for people with limited income. Reduced Shelter Bed D ays Scores assigned to the performance measure "reduced shelter bed days" ranged from one to five. The average score awarded to this measure was 3.78, placing it sixth out of nine measures by mean score. Four of 20 residents responding assigned this measure a five, as an absolutely important or critical measurement to collect, and three residents placed reduced shelter bed days in their top three most important pe rformance measures. This performance measure caused noticeable confusion and questions from residents. The most common theme expressed by residents was that it did not make sense to measure a reduction in shelter use because nobody should be returning to a shelter once they had permanent housing. One resident's comments on this performance measure reflected others' questions and confusion well:

PAGE 90

82 I never thought they would use the shelter after they got an apartment. That one I don't know. I'd imagine they'd stay here as much as they could. Cause it's an apartment. I'd never heard of anybody going back to the shelter, though. While most did not offer specific explanations of why returning to the shelter would be bad, other than it did not make sense if someone had their own apartment to stay in, others brought up poor conditions in the shelter. The shelter was described by residents as crowded, as dirty, as contributing to illness, as a place to contract bedbugs, and as unsafe or potentially violent. Two additional reasons were provided for why this performance measure carries less importance than others Issues of privacy were raised, with one resident noting that "tracking where people go outside of this building is really [nobody's] business." One resi dent also noted that looking at shelter usage may not make sense given many people would choose to sleep outside. There would be no impact to shelters, even if the person did opt to not stay in their apartment. The residents expressing confusion or questio ning the validity of this performance measure connected shelter use to an individual's circumstances and choices once in housing They did not discuss the collective impact to a shelter that may come from a large number of individuals moving into permanent housing and reducing the overall need for shelter in a community. A couple of residents though, did make a connection between an individual accessing and staying in permanent housing and the overall capacity of the community's system to serve pe ople expe riencing homelessness: [I] f you can get a place like this and fall into a way of life where you get back into the mainstream of life and you're doing great, it frees up a bed for somebody else that really needs it.

PAGE 91

83 Another resident connected this performan ce measure to issues of retention, noting it would be important to know that people were "staying here and not having to go and use a shelter anymore or be homeless. In discussing this performance measure, several residents noted the difficult transition from homelessness to housing, citing this as a reason some people may choose to stay in a shelter instead of their apartment. If people were opting to stay in shelters, it would "show that a habit had been developed." This may help staff understand how th ey could step in and help someone to keep moving forward instead of backwards. Reduced Emergency D epartment (ED) V isits Residents rated the performance measure "reduced emergency department visits" seventh overall, based on mean score. Scores assigned ranged from one to five, with seven of 20 residents responding scoring it as a five. Three residents placed this performance measure in their top three most important measures to collect. Residents most commonly communicated that a reduction in emergency department visits would signify that people were able to deal with their medical and behavioral health issues better once in housing as compared to when they were homeless. By having a permanent place to live, people may see their health improve and be les s prone to getting sick. One resident, who had seen the toll that homelessness had taken on the health of those on the street described the importance of housing as it related to improved health: How many fewer times they went to the emergency department, though, that's huge for me. Because with my health situation and with the people I saw die on the streets I mean, if people are dying on the streets. If people have emergency health issues in the shelters. And if people are compromised to the point that th eir lives are in jeopardy. And you can provide them housing, and then suddenly those kinds of emergencies seem to go away or are minimized. Well, right there alone is the success of the program. I mean, go od Lord. If you save one life, r ight?

PAGE 92

84 Some resident s stressed that not only was one's physical health compromised when homeless, but that "the street takes a toll on you" and will greatly impact your mental health as well. Some residents focused on the use of emergency rooms as a place to keep warm during the winter months One resident described how people experiencing homelessness may drink alcohol excessively with the purpose of being transported to the emergency room where they could stay warm for the night: And they get really hammered sometime. And e very night, the ambulance shows up and takes someone away. A lot of times, they'll do it on purpose because it's really cold out. And, they want to go to the hospital. Poudre Valley (Hospital) is wise to that. And they only keep them in there as long as is necessary. And t hey usually let them out about one or two in the morning when it's really cold. So, what they'll do, I mean if they're really desperate, they'll go out there and get hammered again and then have the ambulance pick them up again. They end u p back in the hospital twice a night. Here, totally different situation. You have shelter hereW ell, here you are, you got a room. You got no reason to go to the hospital. Going to the emergency department less frequently, whether because one was healthi er or because one had another place to be warm during colder months, was generally seen as a positive outcome T wo residents specifically mentioned the costs savings that would come from fewer emergency department visits. People would cost the government one way or another. Rather than paying for expensive hospital visits, helping to subsidize people's rent would be a better use of funding. Those who ranked this performance measure lower pointed to one of two arguments. First, issues of privacy were noted Knowing whether someone visited the emergency department should only be "important to the person seeking medical treatment." Second, some residents commented on the necessity of still receiving medical care, which may involve a visit to the emergency dep artment. For this reason, it did not seem appropriate to

PAGE 93

85 "make a goal out of" reducing the number of trips to the emergency department that someone may take. This sentiment was also communicated by a professional currently working as an administrator in pe rmanent supportive housing. Residents should feel confident staff will help them access medical care or call an ambulance if needed. If a goal was set to reduce the number of visits to an emergency department, staff may question or try to dissuade someone from calling an ambulance or seeking medical care in an emergency situation. Community S atisfaction Responses to the performance measure "community satisfaction" were varied, with residents generally feeling it was either very important or critical, or no t at all important. Overall, the performance measure had the second lowest mean score awarded, and scores ranged from one to five. Seven residents (out of 21 resident respondents) however, ranked this per formance measure as a five. S even residents also pl aced this performance measure in their top three most important measures, a pattern more common of performance measures in the top half of performance measures discussed. The most common theme emerging from residents centered on the stigma they felt that c omes with being homeless or formerly homeless. Residents described the looks and attitudes from others in the neighborhood, noting they are viewed as a "different species," as "leeches," and as if "we're not even people." This stigma then often translated into a "Not In My Backyard" attitude toward the permanent supportive housing project. Multiple residents commented on how the neighborhood had fought Redtail Ponds being developed in its current location. Although the majority of residents similarly commun icated issues of stigma and community perception around homelessness, there was a split among residents with how

PAGE 94

86 much importance this performance measure should carry given that stigma. For some, because the neighbors unfairly looked down upon the resident s, being concerned with their satisfaction levels was not important. One resident explained why he felt paying attention to the surrounding community's perceptions and satisfaction levels was not important: [W] hen they hear about a building like this, the y're going to have preconceptions. Oh, this is going to be bad for business. This is going to be bad for the neighborhood I mean, this place does not bring down property values. Honestl y, I think some people need to kind of just shut up. What other people think about the surrounding building, I don't think it's that important. For other residents, Redtail Ponds provides an opportunity for residents to help combat some of the stigma surrounding homelessness. One resident explained the importance of having a well run supportive housing community by noting that "we may be handicapped, and we may be using thei r tax dollars, but we can still contribute something back to the community that supports us." Some residents also pointed to the importance of Redtail Ponds being successful and the surrounding neighborhood being satisfied in helping to lay the groundwork for developing other permanent supportive housing projects. As one resident noted, "I f it turns out that [Redtail Ponds] wasn't what they thought it was going to be or as bad as they thought it was going to be there will be less re sistance in the future p rojects ." Several residents also noted things they and their neighbors do regularly to help show they can be a good neighbor and to help reduce the negativ e perceptions of Redtail Ponds: You've gotta get along with your neighbors. Even though they say they don't like the place or t he people -you know, a bunch of homeless people. I say, I'm not homeless. I got an apartment now. But I try to -a couple of us go around, picking up trash. I don't know if they see that.

PAGE 95

87 Another group of residents connected w ith the satisfaction of the surrounding neighborhood to their own personal safety, and for those reasons felt this performance measure was important. For some residents, the safety component of the project was inherent with the location and general charact eristics of the neighborhood. Others noted that due to neighbor concerns, several safety provisions were put into place prior to the building opening including secured entrances, security cameras, and a 24/7 staffing pattern. While these may be frustrating for some residents, they help others feel safer in their housing. While much of the conversation regarding this performance measure centered on the neighbors and concerns for having Redtail Ponds in their neighborhood, several residents did note that the building is s urrounded by office and business buildings. There is some distance between Redtail Ponds and the nearest residential neighborhood. Because of the location, perceived as being isolated from the community for some, this performance measure was n ot important. It may possibly matter for another project that more directly impacted the neighborhood in which it was located, but the location of this project made it an inapplicable performance measure to track. Voluntary Participation in S ervices This performance measure had the lowest mean score of the nine performance measures discussed. Scores ranged from one and one half to five. Six of 21 residents responding scored this performance measure as a five, absolutely important or critical, and four residents placed the performance measure in their top three most important performance measures. The most common theme related to this performance measure centered on how services can be important in helping people to progress. Tracking participation in those

PAGE 96

88 services offered on site in permanent supportive housing can assist the program staff to see "what's working and who shows up and how effective that is for helping people move forward." Although several residents discussed the services available in h ow they could be helpful, some residents pointed to others' reluctance to participate, connecting this to a lack of desire to receive help and move forward. For example, one resident noted how people have changed since moving in, claiming, "They came in. T hey got comfortable. They sat down, and they've stopped working." Another resident described the services as a critical part of permanent supportive housing, particularly for those w ho choose to engage in services, stating, This is supportive housing. The idea of this is to help people that want help, that want to be more stable in their life financially, mentally, and physically. There's a lot of peoplet hat will just take a free ride." For those who disclosed they are not regularly engaging in services, they described issues with anxiety, a desire to "keep to myself," and a lack of interest in some of the classes or services provided. Two residents also expressed there may be other and better ways to help residents rather than the type of services or cla sses provided. Those who personally did not regularly participate in services or find value in what was offered also tended to see less importance for tracking this performance measure. Two residents also specifically mentioned the financial component of providing services on site. While some classes and services are provided by community members at no cost to the participants or the permanent supportive housing project, these residents assumed there was some cost to every service provided on site. For the se residents, tracking participation in services would allow staff to determine whether those services were being used and if the benefit warranted the cost. Another resident also commented on the

PAGE 97

89 relationship between the cost of providing services and the resulting benefits, but instead focused on the difficulty of measuring the actual impact of services: [S] ometimes participation may not be as big as they would like. They may think, "Oh, we thought 40% of the residents would show up for thi s. Only 10% sh owed up." But the 10% that showed up got so much from the program. It could change somebody's life. Summ ary of Performance Measure Rankings for Professional s Fourteen p rofessionals responded to nine measures often used to gauge the performance and success of permanent supportive housing programs. Two of these measures (incomes of the residents and retention in permanent housing) are required outcomes for HUD funded progr ams to report in annual grant reports and funding applications. The other seven performance measures are derived from guidance and recommendations from leaders in the field, including HUD, the United States Interagency Council on Homelessness, and non prof it technical assistance and training providers. A summary of the scores given by professionals for each performanc e measure is included as Table 4 Charts comparing the rankings of professi onal and resident respondents are included in Appendix A Table 4 Professional s' Scores on Nine Performance Measures Performance Measure Mean Score # Prof. Scoring a 5 on the Scale # Prof. Ranking in Top Three Minimum Score Awarded Maximum Score Awarded n Retention in Housing 4.93 13 14 4 5 14 Chronicity/ Vulnerability 4.32 7 5 2 5 14

PAGE 98

90 Table 4. Professionals' Scores on Nine Performance Measures cont'd Performance Measure Mean Score # Prof. Scoring a 5 on the Scale # Prof. Ranking in Top Three Minimum Score Awarded Maximum Score Awarded n Voluntary Participation in Services 4.11 4 5 3 5 14 Reduced Emergency Dept. Visits 4.04 3 6 3 5 14 Reduced Jail Bed Days 3.93 2 4 3 5 14 Resident Satisfaction 3.79 5 4 1 5 14 Reduced Shelter Bed Days 3.68 2 2 2 5 14 General Community Satisfaction 3.39 1 1 2 5 14 Increased Incomes of Residents 3.18 1 2 2 5 14 Housing R etention The retention in housing performance measure was consistently ranked as being highly important by professionals. Every professional but one ranked this measure as a five "absolutely important or critical," and all professionals included retention in housing as one of their top three performance measures to collect. In addition, this measure had the overall highest mean score of the nine performance measures to which admi nistrators responded The high ratings for this performance measure corresponded with statements from professionals expressing how this measure reflected the ultimate intention of permanent supportive housing: to keep people housed, including those who hav e previously been

PAGE 99

91 unsuccessful in housing. Thirteen of the 14 professionals interviewed framed this performance measure as being "the whole main reason," the "overall goal," the "ultimate goal," and the "whole key to permanent supportive housing." One prof essional working in a capacity building function summarized the importance and weight this performance measure carries: I think this is really the bottom line. For me about supportive housing is if it doesn't do anything else under the sun it has to keep people housed And the biggest way to tell if a project is not doing a good job is if they have high turnover. That means they're not doing whatever they can do to keep people housed. So I think this is the most important outcome that we should be measu ring. Given the vulnerable nature of those who are typically housed through permanent supportive housing, simply keeping residents in permanent housing may be a significant success, as it shows signs of increasing stability for the person. Aside from the weight this performance measure carries on its own, it is also a prerequisite of achieving success with any other performance measure. As stated by one professional interviewed, "[I]t's the root of every other outcome. You can't do any of the other things unless they're housed." Chronicity and V ulnerability Chronicity and vulnerability of r esidents was the second highest ranking performa nce measure, based on the mean score awarded. Professionals cited the necessity to target limited resources to those who need them most. By prioritizing people who are experiencing chronic homelessness or are deemed to be extremely vulnerable, the permanent supportive housing units, which are often limited in number and expensive to operate, can be used as efficiently as pos sible. A s one professional noted, "T he need is so great and the resources are so scare that you've got to be able to prioritize who you take in." One professional also noted that the individuals who are highly vulnerable and often experiencing chronic home lessness also use

PAGE 100

92 significant community resources, such as healthcare or crisis mental health care. By prioritizing those individuals for access to permanent housing where they can receive ongoing care, other community resources can be freed up and used by others. One professional, an administrator currently operating and managing permanent supportive housing, also noted that information about chronicity and vulnerability is helpful for program management purposes. By understanding what people have been th rough and what disabilities or other vulnerabilities they may be living with, administrators can better determine what supports and services need to be provided. This comment most closely aligned to residents' perceptions on the utility of this measure in that by having more information about someone's history and current health issues, others can better know how to support them. Whil e most professionals focused on the utility of this measure to the target and prioritize these units to those most in need, a couple of professionals also noted this performance measure is important because of its alignment with HUD's priorities. A program that states it will serve a certain percentage of people verified to be experiencing chronic homelessness can apply for diff erent funding than without that criterion in place. If a program, such as Redtail Ponds, is already receiving certain streams of HUD funding for permanent supportive housing, then the need to serve individuals who are chronically homeless simply becomes "t he reality" of operating that program. To ensure the program can continue to access this funding, a focus on the chronicity of new residents is necessary. Perspectives on this performance measure were not uniformly positive. Two professionals noted that t he measure is lacking given the data around chronicity and vulnerability can be "fuzzy" or the methods used to calculate who is chronically homeless or

PAGE 101

93 vulnerable can be faulty and incomplete One current administrator of Redtail Ponds explained this point : I think there's just people who are vulnerable in ways that the scale doesn't even measure I think, you know, people with developmental disabilities, for example, are at risk of being sort of victimized. And, we've got a guy at Redtail who is doing great, but once his brother gets out of jail, we know that's a huge risk for him because he's been a horrible influence on him and every time they get together, things fall apart [T]here's always circumstances you can't predict. In addition to questions raised regarding the objectivity, clarity, and uniformity of assessing one's chronic status or vulnerability, two administrators also noted this metric was less of a measure that could gauge a program's performance and more of a co mponent of this model of housing. If HUD requires permanent supportive housing to prioritize people experiencing chronic homelessness, then a program that does not do this is not actually operating as permanent supportive housing. Voluntary Participation i n S ervices "Voluntary participation in services" received the third highest mean score of the nine performance measures discussed. This performance measure had one of the smallest ranges of scores, with all professionals assigning a three, four, or five (o n the five point Likert scale). It also had the second highest number of professionals (seven of 14) awarding it a five. Professionals emphasized both the importance of services being offered to permanent supportive housing residents and the condition th at those services are all voluntary. The availability of services was noted as important because these could assist people in getting their needs met, having the support needed to make progress toward their individual goals, and having support, case manage ment, or other services that can aid someone in retaining

PAGE 102

94 their housing. As one current administrator of permanent supportive housing noted, the incentive to retain one's housing can also be a motivation to engage in services: We're here to help you. The services are offered. But it's up to you if you want to partake in them. So my residents I've seen in the last year have been up and down. Fortunately, many times where they're really like desperately needing help, there's a foot in the door and they'l l be willing to work with a provider because they want to retain their housing. And that's a big thing. And so that usually works successfully. Professionals stressed the importance that services were offered but not required in permanent supportive hou sing. Some noted that people are more likely to participate when services are offered as voluntary rather than mandated. Many also thought it was a key program component to have things operate on the resident s terms as "these folks have already been case managed to death Having self determination and being able to dictate how one spends their time is simply a core value of the program model. One professional also connected the voluntary aspect of participation in s ervices to an indication of how a resident was progressing and growing: I think the key word the re is voluntary. Because that to me is the demonstrated evolution of the individual. And the long term health and viable independence of the individual. Force d case management and other services typically don't work. They have to want it. They have to want to get healthy. So the fact that they are doing that is, that's a key to retention in housing. That's a key to not being in jail. That's a key to not visitin g the ED (emergency department) That's a key to not pissing off the neighborhood. I mean, it's, you can go on and on. But that to me is a pretty key indicator. Two professionals noted the relevance of this performance measure as a management tool. Both professionals have direct service experience and recognized that tracking and measuring participation in services can help a program better determine how to assist or case manage residents and how to plan and budget for those services provided. While ran ked relatively high by professionals, some pointed out this measure's flaws. One professional, for example, ranked voluntary participation in services as a four, and stated

PAGE 103

95 she refrained from ranking it as a five given that score would indicate to her, a mandate. If the purpose was to provide access to services but not require them, putting too much of an emphasis on this measure may create pressure on programs to have residents participate in services, thus removing some of the "voluntary" aspect of parti cipation. Another professional also stated that this measure is too subjective to be a reliable indicator for a program's performance. For instance, staff in permanent supportive housing programs may use casual interactions, such as an impromptu conversat ion over coffee, to engage with residents rather than a traditional, one on one case management or therapy session. Different funders or other program stakeholders may view what constitutes voluntary participation in services very differently from each oth er and from what the permanent supportive housing program determines to qualify as "participation." Reduced Emergency D epartment (ED) V isits Reducing ED utilization at hospitals fell in the top half of all performance measures when ranked by mean score. Reduced ED visits had the highest number of professionals place the performance measure in their top three most critical measures to collect, second only behind retention in housing. Two general themes emerged among professionals discussing the utility and importance of tracking ED utilization: improved quality of care and cost savings. Ten of the 14 professionals interviewed explicitly mentioned cost savi ngs or cost avoidances to hospitals and the hea lthcare system in their explanations on the importance of measuring changes in ED visits. Most professionals also connected these cost savings to better care for the individual. As explained by one professiona l working in a capacity building function:

PAGE 104

96 The other piece is that the less people are using the emergency department is an indicator that they're getting some sort of regular medical care. So that means that the program is doing a good job of hooking them up with their regular doctor, making sure they get to their appointments, making sure they're doing preventative and self care at home. Other professionals did not feel that a reduction in emergency department usage would necessarily indicate better heal thcare or health outcomes for individuals. A stronger performance measure would gauge whether there is an increase in preventative or regular healthcare usage rather than a reduction in emergency service usage. Three professionals, including two administr ators currently working in permanent supportive housing, noted there should be a consideration for the length of time someone had been in housing before relying too heavily on these data. Individuals moving into permanent supportive housing, by definition, often have severe disabilities and illnesses. If only a short period of time after moving into permanent supportive housing is examined, "it can be really misleading becausetheremay be things that were unaddressed that now are finally addressed and can actually result in an increase in utilization initially." A reduction in emergency department visits may show an impact at the community level, but will serve little function in illustrating whether an individual is better off or healthier. This perspect ive was not shared by all professionals. One professional expressed that a reduction in ED visits does have the capability of assessing how well someone is doing at an individual level. If someone is still using the emergency department frequently, program s can examine why this person is still relying on the ED heavily. Do they feel safe in their housing, or do they still feel threatened or in crisis physically or behaviorally, triggering a trip to the emergency department?

PAGE 105

97 One professional who works in a capacity building function also recognized the importance of tracking ED usage for engaging additional partners and potential funders. With hospitals' interest in reducing those costs that cannot be reimbursed or billed for through insurance, hospital admi nistrators are examining ways to provide better care and reduce utilization among those accessing the ED most frequently, a population that is often chronically homeless and disabled. By demonstrating that existing permanent supportive housing can yield th e hospital's desired outcomes, hospitals, healthcare providers, and insurance providers may be more apt to explore options for funding future permanent supportive housing options for patients. While tracking ED visits has its utility, issues still exist w ith data quality and fidelity. Accessing data directly from hospitals or insurance providers would be accurate, but programs may instead rely on their own accounts of how frequently and for what purposes residents visited the ED. The reliability of these d ata would be questionable. Reduced Jail Bed D ays Reducing the utilization of jails ranked in the middle of all nine performance measures, when ranking based on mean score assigned by professionals. Only two professionals assigned a "five" to the performa nce measure, denoting it as absolutely important or critical data to collect. Four professionals placed this performance measure in their top three measures to collect. The most consistent theme expressed by professionals when discussing the relevance of collecting data related to jail bed usage related to cost savings, similar to the reduction in emergency department visits. Eleven of 14 professionals interviewed cited cost savings or avoidances as a reason for collecting these data. One federal level pro fessional noted that

PAGE 106

98 "night by night, it's a whole lot cheaper to have someone in their own studio apartment in PSH (permanent supportive housing) than it is in a cell. Period. It's just that simple." The cost savings argument is relevant not only because of the actual potential dollars saved by providing permanent supportive housing instead of "housing" people in jails, but also because it ensures those resources in the criminal justice system are being used as efficiently and effectively as possible. One professional working in a capacity building function described recent conversations she has had with professionals in the criminal justice field who are interested in understanding what can be done to address homelessness as they a re seeing the court and justice system clogged with non criminoge n ic homeless individuals instead of focusing on those with more dangerous behaviors It has become a public safety issue for those professionals and not only a matter of reducing costs. Beyond the cost argument, pro fessionals had varying explanations for what a reduction in jail bed days may show. Some felt by having housing those individuals may not be in situations that would more likely end in police contact or arrest, such as getting in fights. Another profession al relayed stories she has heard of people not having a shelter bed to sleep in and knowingly committing a crime to get arrested and have a warm place to go. Two other professionals brought up the issue of the unfair criminalization of homelessness, or tic keting and arresting people for conducting daily living functions (such as sleeping) in public spaces. As one professional stated: [I]t's an inhumane response to homelessnessI think it depends on why people are in jail. But if they're being criminalized due to not having a place to go, we need to offer them a more humane response and reducing any time that they spend being in that situation and not being housed would beideal.

PAGE 107

99 Two professionals working as administrators of permanent supportive housing questioned the connection between this measure and the actual performance of the project. While showing a reduction in jail usage may be helpful to show on the community level, it may be difficult to effect change on the individual level. Whether or not someone recidivated into jail would largely be out of the control of the permanent supportive housing project and its staff. For these professionals, any jail usage would reflect mor e on the individual and their choices than on the success of the housing project. Resident S atisfaction Resident satisfaction had the broadest range, with rankings ranging from one (not at all important) to five (absolutely important or critical). Five p rofessionals ranked this performance measure a five, outdone only by housing retention and the chronicity and vulnerability measure. The relatively low mean score is due to two professionals' lower rankings of the measure. Measuring resident satisfaction was viewed as relevant for two primary reasons. First, it could act as a useful performance management tool, allowing permanent supportive housing projects to understand for what reasons and in what contexts residents are satisfied or not. This information would enable the housing project to modify programs, services, or amenities to better meet the needs of residents. Second, it would correlate to individuals' likelihood of staying in their housing. As one professional noted: [I]t should be about them at the end of the day. I mean, if they're not happy, if they're not getting what they need from the building, they're not going to stay. They're not going to get healthy. And what's it all about? While the professionals interviewed thought it would be idea l for residents to be happy and satisfied in their housing, several noted issues with relying too heavily on resident

PAGE 108

100 satisfaction as a gauge of project success. Two professionals mentioned the possibility of individuals acting entitled and not being thank ful for and satisfied with the housing in which they now live. Two professionals, both working as administrators of permanent supportive housing, also noted it is not the best measure of success because it can fluctuate greatly. Whether an effect of a men tal health or substance use disorder or the result of someone just having a rough day, the measure of resident satisfaction does little to inform the project of how well an individual is actually doing. Instead, it is a snapshot of how that person is feeli ng on that particular day. Another current administrator of permanent supportive housing noted it is more useful to understand why a person's le vel of satisfaction may be low and react to those complaints or concerns accordingly. For example: [I]f they're not satisfied because we don't have a pool table for them, or there's not a swimming pool, t hen that's one thing. But if they're really feeling unsafe or we're not able to meet their needs, then I think that would be a very important one. Even with co ncerns around objectivity and the need to look more specifically at why a resident may be unhappy, most professionals felt this was an important measure to consider. For one professional now working in a capacity building function and with direct service e xperience, there typically is a direct correlation between the overall success and performance of a project and the satisfaction levels of the residents: I think in some ways that is the way you tell if a housing project is being done right[O]bviously you're going to run into situations where you have because you're working with people who are vulnerable and have a lot of issues you're going to have people that get mad a lot. I know that from direct service. But overall the projects that I've seen that do a really good job, the residents are really, really happy to be living there. And I think that's in some ways the most important thing.

PAGE 109

101 Reduced Shelter Bed D ays Reduced shelter bed days" was one of the lower ranking pe rformance measures ranked by professionals. There was a range of scores between two and five on the five point Likert scale, with only two prof essionals giving the measure a five and only two placing the measure in their top three most critical performance measures. The most commonly cited explanation for this measure carrying some importance was the through flow of the community's response to homelessness demonstrated by showing a reduced use or need for shelter once someone was in housing. Taking "an enti re cohort out ofcompetition for beds" would open up shelter bed days for others and reduce the overall demand on emergency shelters in the comm unity. Three professionals connected the reduced demand for e mergency shelter to the reduced costs associated wi th placing people in permanent housing. One professional, working in a capacity building function, also focused on the need to demonstrate that the community was doing more than "warehousing" people in shelters Showing a reduced need for emergency shelter would indicate the community was working to find a better and more humane living situation for people than emergency shelters Three professionals pointed to the connection to measuring housing retention. Usi ng emergency shelters once a person was in housing would indicate they had lost their housing or were not actually staying in their apartment. Because of this reason, some felt it would be a less useful measure as it would be duplicative to measuring reten tion in housing. One current administrator of permanent supportive housing commented on the relevance of this measure when looking at the community impact of providing permanent supportive housing but the inadequacy of the measure when looking at impact at the individual level:

PAGE 110

102 [O] n a community level it's important to see that we're making that impact. That we were able to take 15 people off the Point in Time Count. That means there's 15 less people using the shelter beds. But once they're in supportive housing, that seems like a weird measurement because they're housed, so why would they use a shelter? Another current administrator of permanent supportive housing did, however, see a connection between showing a reduction in reduced shelter bed days and an impact at the individual level: [E] specially for the community it helps to show that, that again the permanent supportive housing is being successful. That you are reducing that need in the community. That you are, people are in homelessness for a s horter amount of time, which is again not only important for the community, but important for the individual. The longer somebody is homeless, the more trauma they're going to experience. The more damage to their healt h, to their emotional well being. You know, the more work it's going to take to get them stable and help them start moving forward in their lives. So, the less time that they're out on the street or in a shelter, the better for everybody. Professionals generally felt this was not one of the most critical measures that could be relied upon to show how well a permanent supportive housing project performed as there are simply other, better measures to use. A few professionals also pointed to the specific issue with who may move into and use perm anent supportive housing. If individuals who are the most vulnerable are prioritized for permanent supportive housing, many of these individuals may be sleeping outside, a more dangerous location than inside in an emergency shelter. If this is the primary sleeping location new residents are moving from, there should not be an impact to emergency shelter use. Community S atisfaction The performance measure "community satisfaction" had the second lowest mean score among the nine performance measures for profe ssionals. One professional ranked it as a five and one professional (a different individual than the professional assigning a five to this performance measure) placed it in their top three most critical performance measures.

PAGE 111

103 The most common theme emerging from professionals' discussions on this performance measure addressed NIMBY ism, or the tendency for neighborhoods to oppose the project, wishing it to be placed elsewhere and "Not In My Backyard." The need to reduce or address this NIMBY effect, creating a more positive response from the community yielded varied responses. A few comments emerged regarding the benefit of being a good neighbor just for the sake of being a positive part of the community. For one current administrator of permanent supportive housing, this was most important to address at the beginning of implement ation of a new project, with many of the concerns from neighbors dissipating over time. Six professionals tied the need to address community satisfaction and the NIMBY ism faced by pe rmanent supportive housing projects to longer term outcomes, including the ongoing financial support and sustainability of existing programs as well as the provider's ability to develop future pro jects Without a positive response from the community, the existing program would be at risk of folding and the potential to develop future properties would be dismal without being able to point to a currently successful project in the community. A few professionals also pointed to the stigma attached to being homeless, which tends to follow someone into housing, especially if they are living in a property designated for formerly homeless persons. Having a successful permanent supportive housing project can help to change people's minds and reduce the stigma att ached to people with disabilities or people who have experienced homelessness. One professional commented on this potential effect:

PAGE 112

104 [I]t's wonderful to be able to get the community like, "Oh wait, this isn't so bad. No wait, these aren't scary people. Wow. You've totally changed my thinking around people with mental illness or people who have been homeless" And you can demystify all of these things and how amazing is that? That this can be used as a tool for that. Professionals expressed several r easons why overall community satisfaction would not be as critical of a performance measure as some others. Most community members would have a lack of understanding about what permanent supportive housing is and how it operates. They simply would not have the depth of knowledge needed to have an informed opinion of whether or not a project was operating well or not. Examples from other communities have also shown that after initial fear and frustration from the community, most community members forget the project is operating or, at minimum, are not vocal about any remaining fears or concerns. Finally, permanent supportive housing is not designed to serve and accommodate the entire community. It is designed to serve people experiencing homelessness, and thi s should not be sacrificed for the sake of other community members who may fear or dislike the project As one professional working in a capacity building function stated: I think that we tend to as non profit and supportive housing provide rs, we tend to h ave a mentality that we owe something to the neighbors that an affordable housing or a for profit housing developer doesn't think they owe [W]e don't want people to be unhappy. Neighbors to be unhappy. Or the politicians to be unhappy. And at the same time I think we just got to do what we got to do, regardless if they are or not. Increased Income Professionals' opinions and preferences toward the performance measure related to increasing or maintaining residents' income were mixed. This m easure receive d scores (on the five point Likert scale) ranging from two to five, or from "a little important" up to "absolutely important or critical." "Increased income" also received the lowest average score

PAGE 113

105 of all performance measures discussed It was also one of t he least frequently cited performance measures to include in one's top three measures to collect, tied with a reduction in shelter usage and community satisfaction. Some professionals struggled to even provide a single number rating this performance measur e because of the different circumstances faced by residents in permanent supportive housing. Some may be able to be employed, and gaining employment and increasing their earned income would be a viable performance measure for those individuals. Some may mo ve into permanent supportive housing with no income, be unable to work enough to sustain themselves, and ultimately be connected to benefits, which would be a positive outcome for those individuals. Some people, though, will have access to those benefits t o which they are entitled at the time of move in, and will never be able to see an increase in income unless those pay ment formulas for benefits changed. The varying expectations for people and whether it was realistic to increase one's income or simply ma intain was a frequent topic of discussion among professionals. The idea of gaining more income was not negative in itself. Professionals recognized having more money could enable people to have their basic needs met and even have additional funds to allow them to be more involved in the community. One administrator of permanent supportive housing noted this could be a measure used to generally gauge someone's ability to live in a better circumstance: [I]f someb ody goes from having zero income to having AND (Aid to the Needy and Disabled) that's only an extra $189 a month but that makes a huge difference in the person's life in terms of being able to eat better. In terms of being able to go out and actually participate in something in the community. And then the jump from AND to getting SSI (Supplemental Security Income) is really huge. So yeah, I mean I think it definitely can be a good, hard number that can help give you some sense of at least somebody's opportunities in life.

PAGE 114

106 In addition, one professional working in a capacity building role linked having income to housing retention. One major component of adhering t o one's lease is paying rent on time. This can only happen if the person has access to income. For being able to maintain one's hou sing and have opportunities to reconnect in the community, having some income is important. This was also seen as an outcome measure that would be easy to measure and to track. The reality of whether it was an appropriate expectation of residents in perman ent supportive housing to increase income, though, was called into question. Given that individuals who are severely disabled and who have been homeless for extended periods of time are targeted for permanent supportive housing, having a performance measur e related to income seemed "incongruent with the goals of permanent supportive housing." These are individuals who are not likely going to be able to be employed. The only option for income for many is public benefits. Once someone has been connected to th ose benefits, there is no room for continued improvement and no ability for the staff to effect any change. One current administrator of permanent supportive housing relayed her frustration with being required to report on things seemingly outside of their control: I think you have to look at what they came in with. So if they came in with no benefits and you were able to get them some benefits, then that's a success. But if they're looking at earned income, then again, you've got the conflict between pe ople with disabilities and just their ability to do that is -it's not anything we can control. So in some ways it feels like an unfair measurement. It's like, give me s omething I can control. Another professional expressed concern that there could be residual effects of pushing permanent supportive housing projects to assist those who can be employed to gain more income through earned sources. If a heavy emphasis was placed on increasing income, projects may unintentionally be encouraged to select tho se people who seem the most

PAGE 115

107 employable rather than those who are deemed to be most vulnerable or most in need of this type of housing. When speaking about placing an emphasis on increasing income through employment, one professional mentioned the potential outcomes that may follow: [D] oes it inadvertently change your selection process on what tenants you get because you're like, "Well this one won't be able to get a job." But they're the ones who need the permanent supportive housing the most. And so you've got to be very careful about inadvertently creaming or going for the wrong population based on what outcomes you set. HUD has adjusted the performance measure related to income from one that previou sly focused on increasing income to one that now counts maintaining or increasing income as a positive outcome. Even with this adjustment, the professionals working as administrators of permanent supportive housing did not view this measure as the most rel evant or valuable to determine whether or not their project was a success. Income is not necessarily tied to someone's well being. One administrator illustrated this by noting that there are "some people at Redtail who haven't increased their income but ar e doing 100 times better than they were a year ago." Other people find the stability that comes with housing, work to become employed, and take on a schedule that is too much to handle at once. The same administrator described these experiences: We have so me other people who have increased their income. They felt like, "Oh, now I can get back to work." They're going back to work and they w ent a little bit too fast. And going from a few months ago sleeping under a bridge to now working 50 hours a week is too much for them. And it really is actually a detriment. Overall, professionals felt more comfortable with a performance measure that focused on maintaining or increasing income. Even with this modification, professionals generally felt this was not as critical of a measure to collect given the focus of permanent supportive housing to simply house those who previously were thought to be "unhouseable" and the lac k of a consistent correlation between one's income and their overall health and well being.

PAGE 116

108 Other Themes Among Respondents A few common themes emerged among resident respondents while discussing their experiences in Redtail Ponds and the performance measu res that are often associated with permanent supportive housing. The most common theme emerging centered on the issue of safety. Fourteen of the 24 residents mentioned safety in some context. For some, Redtail Ponds offered a safer location to live as comp ared to their time on the streets. One resident commented on the enhanced safety he felt in his new housing by stating, "Living on the street, obviously it's not a very safe place to be. But here you feel very safe. You don't lock your doors or anything. For others, issues or conflicts with other residents reduced their feelings of safety. One resident described how she does not always feel completely safe at Redtail Ponds: I still have trouble sometimes feeling safe all the time here, because of the few p eople that I don't trust or like their behavior or whatever. Cause I think when people are on drugs or really drunk, they are capable of doing just about anything. And so it's kind of scary. But, I definitely feel safe in my apartment. And really the only time I don't feel safe is like on the weekends, because the staff isn't here. I mean, sure there 's still front desk staff, but something about having the staff here just makes me feel safe. Still other comments related to safety did not mention Redtail Ponds specifically. Instead, residents were commenting on such things as their perceived levels of safety at the emergency shelters or on the general sense of safety they have while walking around the area. In total, almost 60% of the residents intervie wed mentioned safety in some sense during the course of the interview. A slightly higher percentage of women i nterviewed mentioned issues of safety as compared to men. Two thirds of the women interviewed commented on safety issues while around 55% of men b rought up this theme in their interview. In addition, a higher proportion of non veteran residents discussed issues of safety

PAGE 117

109 as compared to the veteran residents. Half of the veterans interviewed (three of six) mentioned safety issues or concerns while ar ound 61% of the non veteran interview subjects brought up the topic of safety. By comparison, just over one third of those administrators interviewed brought up safety during the course of the discussion. Of the five professionals who relayed the importa nce of residents feeling safe, three had direct service experience working with people experiencing homelessness. All five of the respondents mentioning "safety" worked at either the local or state level. Issues of safety were typically brought up in conj unction with other performance measures. For instance, one professional relayed a story of a supportive housing resident (living in a different program from Redtail Ponds) who because of serious mental illness is never satisfied and frequently calls staff as well as her elected officials to complain. As stated by the professional interviewed, "And so you might not be able to ever fully satisfy her. But are we keeping her safe? And are we improving her stability in some way? Another professional incorporate d safety into his definition of positive housing retention, stating the purpose of permanent supportive housing is to help "people stay housed in a safe place." Still another professional incorporated the concept of safety into discussions about the use of emergency services noting that there is a need to "get these folks out of our system and somewhere where they can be safe and not clog the system." While safety was not expressed as explicitly by professionals as by some of the residents interviewed, thos e professionals who did mention the concept of safety regarded it as an assumed expectation of permanent supportive housing, that residents could and should feel safe and secure in their housing.

PAGE 118

110 While not as prominent as the theme of "safety," one third of residents interviewed mentioned the concept of moving forward or progressing in their life. An expectation of federally funded permanent supportive housing is that supportive services and case management are offered but not required. Similarly, expecti ng certain outcomes from residents is not in line with the housing first philosophy underlying permanent supportive housing programs. For three residents, this concept of improving their lives or moving forward was tied to an increase in income; increasing one's income was viewed as an indication that one was making an effort to improve his life. For other residents, having a sense of stability from living in permanent housing allowed them to work on personal goals. One resident described his process of mov ing from gaining stable housing to moving forward in other areas of his life, such as rebuilding his familial relationships: Now we can start talking about doing a little old self improvement. Now we can start talking about some of the things that we need to do to get this right and get that right to get you on your way. Which I'll be glad to get back to. I've got a lot of things I'd love to do. Got a lot of unfinished business. A lot of things to do for kids and grandkids. Few of the professionals intervi ewed conveyed the theme of resident growth, improvement, or progression. Four professionals mentioned the need for residents in permanent supportive housing simply to have their basic needs met. Four professionals (not exclusive to the four expressing the need for residents' basic needs to be met) expressed an interest in having performance measures that could indicate residents were doing more than simply subsisting with their most basic needs met. One professional, currently working as an administrator of permanent supportive housing, described how this may work:

PAGE 119

111 Every resident has kind of their individual goals. And another thing we were looking at was how many residents achieve the goal that they set. Rather than saying "how many achieved this goal but just how many achieved a goal. Which means they're motivated and they're making progress and they're moving forward and they're forward thinking versus just in survival mode. One third of residents interviewed also conveyed issues around feeling a sen se of empowerment and control over their living spaces and their lives. Some residents noted their current living situation afforded them a sense of control they had not had when they were homeless and living in shelters. One resident described these feeli ngs of powerlessness: Because homeless, we start feeling like we're somebody's child. They tell us when we can eat, when we can go inside, when we can go out and play We might not want to go out and play, but we get eight hours out there whether we want i t or not I'd rather have a home of my own and my own roof over my head where I have a little bit of control of what goes on in my home. You know, being in the shelter, you have no control. Other residents, still conveying this theme around wanting a sens e of control and power in one's living situation and life, reflected instead on those factors of living in permanent supportive housing that limited their sense of control. Some residents mentioned issues around privacy that have come from having heightene d security (including security cameras and round the clock staff who are responsible for monitoring guests and visitors) and regular apartment inspections. Commenting specifically on the frequent inspections conducted when Redtail Ponds first opened, one r esident stated it "just feels too much like an invasion of privacy. Too much like control, and not enough like it's my place." Another resident described the building as feeling "slightly institutionalized" with the security measures taken and rules relate d to visitors (such as requiring visitors to check in when they arrive and requiring residents to escort their guests at all times). Five professionals mentioned the concept of residents having control or having a voice in how their housing looked. For some, this was a generic statement related to the ideal

PAGE 120

112 of giving residents a say in what happens in their own housing. For two professi onals, the concept of having a choice was communicated specifically in relation to their participation in services. As supportive services and case management are intended to be voluntary, this is one aspect of permanent housing in which resident control a nd resident voice are embedded in the program design. For professionals, resident choice may come into play with the services offered or the general management of the project; for residents, more specific issues around privacy and security were those area s that impacted their sense of power and control most frequently. One professional currently working as an administrator of permanent supportive housing recognized and commented on some residents' feelings related to their sense of control: [There are] peo ple who are actually doing really well and were doing horribly on the streets, suffering a ton on the streets. And they're saying, "I don't like it here. I feel like it's a jail. I feel like it's a mental institut ion." And you're sitting there thinking, "You can walk out the door at anytime." This clearly is not like a jail. We had one guy who was saying, "This feels like it's a gigantic Nazi experiment." And I understand his frustration, and you want to find ways to address that frustration. At t he same time, I don't remember concentration camps having birt hday parties and art classes, you know? The most frequently expressed theme among professionals related to cost savings, cost benefits, or cost avoidances by pr oviding permanent supportive housing. Every professional interviewed mentioned the costs associated with serving people experiencing homelessness at some point during their interview. Serving people in emergency departments, shelters, and jails is seen as inefficient and expensive. Providing permanent supportive housing offers an alternative intervention that can reduce the use of these systems, thereby reducing the costs to these systems and the community. While efficiency of community systems and responsi ble use of resources may be desirable goals in general, professionals interviewed connected the cost savings or

PAGE 121

113 avoidances that could be tabulated to the need to advocate for retaining existing permanent supportive housing and building additional units of housing. This is an effective argument when speaking with community members (who are taxpayers seen as subsidizing these types of housing projects ) and political leaders. One professional serving in a capacity building role summarized the utility of a cost savings or cost avoidances argument by noting the following: Cost savings, cost savings, cost savings. All the time. I mean, it's 90% how I get engaged with our leadership. And it's so nicely defined. And I think if we didn't have the cost savings argum ent so well defined, we wouldn't be as successful as we are right now. Another common theme among professionals dealt with the stability or improved health and well being of residents as a result of living in permanent supportive housing. Eleven of the fourteen professionals interviewed relayed the importance of seeing residents' lives stabilize in whatever ways are relevant to that individual. In addition to demonstrating stability in terms of one's housing, professionals spoke of stability in relation to one's ability to be more independent and healthy. One professional, for example, noted that if someone could earn some income, this would be "a massive step not towards self sufficiency, but a level of independence that is not purely program dependent." For other professionals, the connection between being "stable" and being independent was less important than a connection between being "stable" and being healthier. Measuring one's health, though, may prove difficult given the diversity of health issues and needs with which residents may be dealing. Professionals offered suggestions for helping to measure one's well being, stability, and overall health including tracking diet and nutrition, measuring one's use of preventative or primary medical care (rat her than emergency care), and tracking the management of any chronic conditions one may be living with. One professional who

PAGE 122

114 currently works as an administrator of permanent supportive housing also noted the utility of speaking with those who have known th e resident the longest to get an anecdotal sense of how residents' health and well being have improved over time: I think a lot of times people may not be able to see the forest through the trees. And even themselves, may not recognize how much better t hey're doing. But somebody who has known them for a long time. Talking to family members and friends who have known them for a really long time and may say, "Yeah, this person is doing so much better. They may not feel like they are, but man, they're doing so much better than they were a couple of years ago." For all of the value that professionals may find in knowing someone is in a more stable or healthier place due to accessing permanent supportive housing, there was no consistency in how stability may be able to be measured. Seeing an improvement in one's stability or well being was related less to hitting a pre determined benchmark and more to a sense that one's life was continuing to im prove. Profes sionals also spoke frequently to the value of residents having choices and having a voice in their housing. This theme emerged in interviews with ten professionals, primarily when speaking about residents' decisions to try to gain employment income, to participate in services provided, and in setting up personal goals In addition, measuring resident satisfaction w as seen as valuable for how this measure c ould reflect residents perceptions of having a voice and having options in their housing For these professionals, having these choices related strongly to the basic tenets of permanent supportive housing including adhering to a housing first philosophy To avoid having permanent supportive housing feel like anything less than a traditional apartment, providing choices ensures individuals have a say in how their life and their residence looks. As one professional working in a capacity building role stated, "[T] hat's our entire goal. To get folks into

PAGE 123

115 housing, have them satisfied, and have the m have some say over where their life is going now. Or have them have complete say over where their life is going now. For professionals, the idea of having choices means residents are in control of their living situation. In reference to having case man agement and participation in services be voluntary, one professional noted, "[T]hey knowthey are in control and this is not something being forced upon them." In contrast, when residents spoke of feeling in control (or conversely lacking control) it was i n relation to rules of the permanent supportive housing or of shelters or a perceived lack of privacy often resulting from security measures, such as locked front doors and cameras. While both populations interviewed valued resident choice, voice, and control, perceptions of where residents could exercise that control varied. Alignment of Preferences for Performance Measures Between Residents and Professionals Findings from the in terviews with residents and with professionals indicate each group may place an emphasis or importance on different performance measures, and each group may view each performance measure's importance for different reasons. The mean scores for all performan ce measures fell between 3.18 and 4.93 for professionals and between 3.67 and 4.53 for residents. In both groups, four performance measures received an average score of 4.0 or above, and five performance measures received an average score below 4.0. Append ix A provides charts comparing professionals' and residents' rankings of each performance measure. Table 5 below summarizes the findings for both residents and professionals and specifies whether the residents and professionals are aligned in their perspec tives on the performance measures examined. In Table 5 those performance measures with mean scores

PAGE 124

116 of 4.0 or above are marked as those receiving strong support. T hose performance measures scoring less than 4. 0 are marked as having mixed/mild support, indicating there was not consistent support for the measure across the interview group. Alignment refers to differences among the interview groups' understanding of the performance measures and what each performa nce measure indicates. Generally, there was more consistency among professionals in their rationales behind their rankings. Res idents expressed greater variation in their assumptions around what information each performance measure could provide and why th e measure is important The more frequently stated rationales expressed by r esidents are provided in Table 5 as an illustration of the most common themes and are not inclusive of every opinion offered. T able 5 Overview of Findings Residents Professional s Resident/ Professional Alignment Performance Measures Retention Strong Support Valued for measuring core function of PSH (providing a permanent place to live) Strong Support Valued for measuring core function of PSH (providing a permanent place to live and keeping people housed) Alignment Increased Income Mixed/Mild Support Valued for indicating residents' stability and progress Valued for preventing cheating or scamming Mixed/Mild Support Valued because income can be positive, but it does not guarantee progress Is valued less because it is incongruent with serving those who with most severe needs Misalignment

PAGE 125

117 Table 5 Overview of Findings cont'd Residents Professionals Resident/ Professional Alignment Performance Measures Chronicity/ Vulnerability Strong Support Valued as a management tool; staff knows how to deal with people and keep other residents safe Strong Support Valued for prioritizing and targeting scarce PSH resources Misalignment Resident Satisfaction Strong Support Valued because being happy is important, although happiness is not always within the control of the project Mixed/Mild Support Valued as a management tool; can indicate where the project can improve Valued for its connection to housi ng retention Misalignment Reduced Jail Bed Days Strong Support Valued because it indicates a change in negative behavior or indicates a reduction in unnecessary police contact Valued for reducing/avoiding costs Mixed/Mild Support Valued for reducing/avoiding costs Some alignment Reduced Shelter Bed Days Mixed/Mild Support Valued because individuals should not return to shelters if they have housing; shelters are dirty and unsafe for people Performance measure does not make sense Mixed/Mild Support Valued for indicating a through flow to the homelessness services system Misalignment

PAGE 126

118 Table 5. Overview of Findings cont'd Residents Professionals Resident/ Professional Alignment Performance Measures Reduced ED Visits Mixed/Mild Support Valued for demonstrating improved health Strong Support Valued for indicating people are accessing better and more appropriate care Valued for reducing/avoiding costs Misalignment Community Satisfaction Mixed/Mild Support Valued because project can help de stigmatize homelessness, with others feeling stigma is so entrenched that gauging others' satisfaction is useless Mixed/Mild Support Valued because the project should be a good neighbor to create support for the curren t and future project(s) Valued because project can help de stigmatize homelessness Some alignment Voluntary Participation in Services Mixed/Mild Support Valued for helping people progress Strong Support Valued for helping people meet their needs and continue to progress Alignment Of the nine performance measures discussed during interviews, professionals and residents were misaligned in the level of support assigned and/or thei r understanding of the context of the performance measure for five s eparate performance measures (increased income, chronicity or vulnerability of residents resident satisfaction, reduced shelter bed days, and reduced emergency department visits). For the performance measures "reduced jail bed days" and "community satisfa ction," there was some alignment between residents and professionals. With both of these performance measures, there was overlap in the understanding of why each measure was important, but also some variation expressed

PAGE 127

119 between groups. With the measure "red uced jail bed days," there was also a difference in the level of importance assigned to the measure, with residents rating the measure higher in importance out of the nine performance measures and professionals rating the measure lower in the ranks of the performance measures. Two performance measures (retention and voluntary participation in services) yielded alignment between residents and professionals in their understanding of the performance measure's meaning. Both groups felt retention was a useful performance measure that was helpful in measuring the core intention of permanent supportive housing (to provide permanent housing and keep people in that housing) and both expressed strong support for using this performance measure. Both groups were also similar in their understanding of the meaning of the performance measure "voluntary participation in services," specifically that it shows individuals' needs are met and they are able to make progress in their life. With this performance measure, however, professionals and residents differed in the importance assigned to the measure. Professionals expressed stronger support, ranking this performance measure third by mean score, as opposed to residents who ranked this performance measure last by mean score The common misalignment between residents and professionals in stated preferences for performance measures is also illustrated by findings from the interviews with professionals in which they were asked to guess which performance measures would be most imp ortant to residents to collect. Generally, administrators did not accurately guess those performance measures that residents found most important, with the exception of the "retention in housing" measure. Ten of the 14 administrators believed "retention in housing"

PAGE 128

120 would be frequently ranked in residents' top three performance measures list. More residents did place "retention in housing" in their top three list more than any other measure. There was more div ision between what administrators guessed reside nts' preferences would be and residents' actual reported preferences for the other performance measures. Twelve out of the 14 administrators guessed "resident satisfaction" would be in the top three measures for residents. Only nine residents interviewed a ctually placed this measure in their top three measures More residents ranked b oth "retention in housing" and the "chronicity or vulnerability of new residents" in their top three. While important to residents, no administrators assumed the residents would select "chronicity or vulnerability of new residents" as one of their top three performance measures to collect. Residents valued this performance measure because they felt it would help staff better know what issues people were dealing with so th ey could provide the right kind of support They viewed these data as being used primarily as a performance management tool, assisting staff to be prepared for the disabilities, addictions, and other issues facing residents. Administrators, however, viewed th is performance measure as a gauge on whether the project was serving those for which it was designed. HUD's most recent guidance requires permanent supportive housing projects to first house people experiencing chronic homelessness. HUD also encourages com munities to use measures of vulnerability to determine who may be most in need of the permanent supportive housing available in a community. For administrators, this measure is less about the management of the project and more about adhering to HUD require ments and demonstrating that the project is serving those with the most severe needs in the community.

PAGE 129

121 Resident Involvement in Establishing Performance Regimes If a permanent supportive housing community were to engage residents in establishing performa nce regimes, some residents would need to volunteer for these efforts. Half of the residents interviewed stated they would be willing to assist staff at Redtail Ponds if they were to ask for their help to understand what was most important to them related to the performance of the project. Four more residents were uncertain or needed more information before they would commit to participating. One resident's hesitancy tied back to his experience with some of the staff, which had tainted his enthusiasm for pa rticipating in some formal efforts of Redtail Ponds: I want to help. But at the same time I've been put in a situation at the same time where I don't feel like I want to help anymore. You know when I first came here, I wanted to help Some staff members hav e shown that they don't really care. It's just a job. So, it's like, well now, why should I care? Those residents who stated they would not be interested in participating cited various reasons including not knowing if they could be of much help to the pro cess, preferring to "stay out of the drama," and simply not preferring to engage in these types of efforts. No professional interview ed felt residents did not have a role to play in helping permanent supportive housing projects to develop performance regimes. Professionals generally expressed it was a good practice to involve those most impacted by the housing project (in other words, the residents) in decision making processes. In addition, some noted that there may be additional information gained from involving residents in those discussions. One professional currently serving as an administrator of permanent supportive housing describe d the pot ential benefit coming from resident involvement:

PAGE 130

122 I'm not sure if somebody would come up with some new performance measure that somebody else hasn't thought of before, but it definitely doesn't hurt to have people involved in that conversation. I think it can also be helpful just on a community level because while it's great that HUD has tried to identify a few performance measures and outcomes that can really be applied in any community, there may be ones that are more specific to an individual co mmunity that can really help identify whether or not a program is successful or whether or not a person is being successful. While there are perceived benefits to involving those with the lived experience of homelessness in establishing these performance measures, most professionals also offered conditions and caveats to this participation, highlighting potential issues that may arise during this proc ess. First, professionals noted that individuals should not be put in a situation that would ca u se them gre ater harm or stress. Individuals who are frequently in crisis or have more immediate needs to attend to may not be the best fit for this work. If administrators are responsible for only selecting those who appear more stable, happy, and less likely to be t riggered by any conversations, administrators run the risk of "cherry picking who's giving input so that you're basically getting people who are going to give you the same input that everybody's always gotten." Second, professionals must recognize these in dividuals are civilians and are generally uninformed about complicated regulations and rules that may exist and are tied to funding or the project development. To make the process more accessible to those residents involved, considerable work should be don e to properly prepare them for participation. One professional currently working in a capacity building role described how the organization for which she works aims to have resident participation in much of their work, including having at least one tenant or resident of permanent supportive housing sitting on the organization's board of directors at all times. She described the level of engagement and amount of effort from staff to ensure this participation is meaningful and successful:

PAGE 131

123 [W] hat we've found t o be successful and what we encourage providers to do, and do ourselves because we have a tenant on our board, is really assign a more senior level staff person to that tenant. Prep them before the meeting And give them a strong, meaningful role in board p articipation. It takes a lot of work and time. For us, we don't just concentrate it in one senior staff member. We all kind of take a turn getting to know our board member and hel ping her navigate the meetings I get to know her story, I can call on her if I'm in the board meeting and say, "Hey, don't you have a perspective on that, Dorothy, from living in supportive housing? What do you think?" In addition to this intensive work to ensure the resident can participate and will stay engaged in the process, one professional also recommended compensating residents if they participate in this type of work. For professionals in the field, they are paid for the time spent in meetings and participating in these efforts. As one professional noted, I'm getting paid to be at this meeting, so why would I expect this client to not get paid? A third issue centers around an inability for any one resident to completely represent the views of all residents in the program. For some professionals, this is simply a recognition that having a few voices at the table still will not fully capture everyone person's perspective, opinions, and ideas. One professional cur rently working as an administrator in permanent supportive housing relayed a story regarding residents' frustrations with a single person who had taken it upon himself to speak for other residents, with the intention of representing their interests: You kn ow, we have one resident who wants to be a spokesperson for everyone. The result was they are now meeting where he can't find them because they don't want him in their group. So they've gone underground. And they aren't telling him where they're going. In addition to issues that may arise from one person or a few people attempting to represent the views of many, there is also the risk of a person coming to the table thinking primarily about their own needs or successes rather than thinking of what may be b est for or applicable to the project as a whole. Professionals also noted that if any residents were involved in the

PAGE 132

124 process of making decisions for the project, it should be stressed that many people's ideas would feed into the process and be incorp orated into any final decisions. One professional described setting this expectation with residents: [T] his is a collective decisi on making process with multiple stakeholders that ultimately decide what's going happen and that you play a role in that. But that i t will be a compromise at the end of the day. And just setting that expectation. And there might be different things that they would want to have as an outcome that you don't. Overall, results from this research indicate residents and professionals under stand performance measures differently. Only one performance measure (retention in housing) had strong support from both residents and professionals and had alignment between both groups' understanding of what the performance measure would indicate. There was a lack of complete alignment in the understanding or interpretation of all other performance measures. In addition, the other themes that emerged through interviews indicate residents and professionals value different aspects of the program implementat ion and corresponding performance measures. All of these findings illustrate that professionals and marginalized citizens think differently about performance measures. I ncludin g perspectives from marginalized citizens can provide different or additional in formation when developing performance regimes.

PAGE 133

125 CHAPTER V DISCUSSION AND CONCLUSIONS This research examines the following question: How do preferences for performance measures differ between public sector representatives at the federal, state, and local levels, and the marginalized public served? To inform this main research question, the following sub questions are also included in this research: 1. How do marginalized citizens and public sector professionals differ in the importance assigned to pe rformance measures based on their impact at the individual system, or community level? 2. How do public sector professionals' and ma rginalized citizens' understanding of performance measures (i.e., what information they can provide or what change they ma y indicate) differ? In regards to sub question number one, both residents and professionals interviewed preferred performance measures focused on the system level. Both groups ranked "retention in housing" and "chronicity/vulnerability of residents" as th e top two performance measures; both of these performance measures primarily indicate change at the homeless services system level. A reduction in shelter bed days (the third performance measure impacting the system level) fell farther down in each group's rankings. Preferences for those measures reflecting change at the individual and community levels were mixed, providing no clear indication of preference for one group of measures over another in either group of interview subjects. In this regard, both re sidents and professionals expressed similar preferences for performance measures.

PAGE 134

126 Both "retention in housing" and the chronicity or vulnerability of residents" are heavily emphasized for permanent supportive housing. "Retention in housing" is a required performance measure of HUD funded permanent supportive housing projects, and newly funded permanent supportive housing projects are required to prioritize people experiencing chronic homelessness. As these performance measures reflect requirements from HUD a major funder of permanent supportive housing, professionals may have been drawn to these performance measures due to the alignment with federal requirements rather than an appeal to measure change at the system level. While a valid consideration for th e professional interviewees, this would not explain residents' preference for these performance measures as those individuals are unfamiliar with the details of HUD's requirements for permanent supportive housing. This emphasis on system level performanc e measures does not necessarily align with professionals' emphasis on cost savings especially cost savings to the community While the theme of cost savings was prominent throughout the professional interviews, those costs are typically measured at the co mmunity level, through a reduction in jail or emergency department use. The disconnect between the common theme of cost savings and the preference for system level, rather than communi ty level, indicators is reconciled when considering professionals' desig nation of "retention in housing" as a core performance measure. While cost savings may be ideal and may be what will open the door to conversations with community members, funders, and elected officials, these savings cannot be realized unless individuals are housed and retain that housing over time. Similarly, housing someone who has been on the street longer or who has more severe needs is assumed to have a greater impact on the cost savings from those emergency services used. Even if

PAGE 135

127 critical for gaining favor, cost savings occurring at the community level is impossible without first focusing on housing those most in need and keeping those individuals in housing long term. The preference among residents for system level indicators also does not align wit h the common themes that emerged from resident interviews, which primarily focused on their individual experiences related to feeling safe, feeling a sense of control, and seeing progress in one's life. Given these themes that emerged throughout the reside nt interviews, a preference for individual level performance measures would seem to better align with resident preferences. However, when asked about specific performance measures, residents leaned first towards preferring the system level measures of "ret ention in housing" and "chronicity or vulnerability of new residents." While residents' conception of what "retention in housing" could measure was consistent with how professionals as well as major funders, like HUD, define the performance measure, resid ents did differently define the "chronicity or vulnerability" measure. For residents, identifying or tracking chronicity or vulnerability of new residents would be helpful to understand the individual issues residents are facing. Residents noted this would be beneficial for program staff but several residents also felt they are able to better interact with one another once they understand the other residents' issues. In these cases, residents may be using a performance measure that is typically used to sho w system level performance to improve their daily lives. If program staff or other residents can better deal with residents with complicated issues, the daily experience of sharing a living space with those individuals may be less tumultuous.

PAGE 136

128 In relation to research sub question number one, when examining the basic rankings of the performance measures discussed during this research, professionals and residents appear aligned in their preferences for system level indicators. However, through more open ende d questioning, residents more frequently discussed factors occurring at the individual level, and professionals more frequently discussed factors reflecting community level change. Sub question number two questions whether professionals and residents have similar or different definitions for each of the performance measures discussed. With only "retention in housing" and "voluntary participation in services" did professionals and res idents communicate similar understandings or definitions of what those performance measures could indicate. With the remaining seven performance indicators discussed, there was misalignment or only partial alignment in professionals and residents definit ions of the performance measures. Even when there is clarity on how a performance measure would be measured, there are considerable differences in what conclusions professionals and residents think can be drawn based on those data. For example, a performance measure such as "reduced emergency department visits" is intuitively clear. The measure would measure how much placing someone in permanent supportive housing could reduce their reliance on the emergency dep artment. The expectations of what this could indicate are broad. A reduction in emergency department visits may indicate someone is receiving care in more primary care settings, that someone is no longer using the emergency department to stay warm or out o f the elements, or that someone's overall health has improved. For those preferring this performance measure because of its connection to a cost savings or avoidance, the reasons

PAGE 137

129 behind this reduction may be less critical. However, even for those partners, they may not be able to explain these cost savings with much specificity without having a common understanding of w hat these data may reveal or complementary measures to explain any measured reduction s in emergency department usage. Overall, findings fro m this research indicate that professionals and residents are rarely in agreement on what performance measures indicate, which can impact the importance assigned to those performance measures by each group. Even within each group of interview subjects, the re may be variety in how individuals may read the intention of certain performance measures. As many of these performance measures are not required by any funder or other entity, there is not always a consistent definition or explanation of what the data c an reveal. Even when there is another entity, such as a funder, in play, the intention of the performance measure may still not be clear. When discussing those performance measures required by HUD for permanent supportive housing communities, professional s were asked if they had any knowledge of how those measures ("retention in housing" and "increases in income") came to be the primary indicators for this type of housing. No professional, including those working in the federal government, had any knowledg e of how the se performance measure s came to be or why HUD preferred these performance measure s over others. Particularly for the measure related to residents' incomes, this lack of clarity resulted in multiple speculations regarding why this measure was re quired. Professionals thought it was selected because it was easy and universal to collect, it "tells a good story," it shows progress towards self sufficiency, or it would encourage residents to earn more and pay more in rent, thus reducing HUD's burden f or paying for the remaining rent on a unit through a housing voucher.

PAGE 138

130 Based on a review of HUD's strategic plans, annual performance plans, and performance and accountability reports over the past 15 to 20 years, these performance measures have been carri ed through year over year, even as department priorities or program models have changed. For instance, in 2001 2002 HUD was just beginning to promote permanent housing as an option for funded programs. Transitional housing was still a more common and more heavily promoted strategy. For the 2002 Performance and Accountability Report, HUD aimed to serve at least 20,000 people in permanent housing, but aimed to serve at least 1 15,000 in transitional housing (HUD, 2002). Transitional housing relies on the assum ption that after an extended stay (up to two years) in a housing program with supportive services, individuals would be able to sustain housing on their own. This emphasis on providing limited support aligned with HUD's philosophy at the time that "[t]he u ltimate goal of homeless assistance is to help homeless families and individuals achieve permanent housing and self sufficiency (HUD, 2002, p. 2 69). In contrast, HUD now recognizes that not all will be able to become self sufficient given the disabilit ies or other barriers some individuals experiencing homelessness may face. This shift is reflected through the following statement in HUD' s 2012 Annual Performance Report : For those individuals who are able, increasing self sufficiency requires access to l ife skills training, wealth creation and asset building opportunities, job training, and career services. For those who need long term support, HUD housing will provide access to income support and other benefits that can enhance an individual's quality of life (HUD, 2012, p. 13). HUD may now recognize the limitations of assuming that self sufficiency is appropriate or within reach for everyone experiencing homelessness, but the performance measures originally tied to this ideal (and correspondingly, th e transitional housing programs that

PAGE 139

131 supported this philosophy) have been carried through and used for permanent supportive housing, designed to provide long term housing assistance and support. As some professionals indicated, HUD now is interested in bot h maintaining and increasing residents' income, a reflection that the performance measures may be catching up to better reflect the intention and goals the program models emphasized for several years now, specifically permanent supportive housing and other housing first models. Impacts for Theory and Practice T his research extends the findings from the limited research on the intersection of citizen participation and performance regimes to include the experiences of marginalized citizens. Those who have experienced homelessness and are often living with severe disabilities, including severe mental health or s ubstance use disorders are stigmatized. This stigma may also bring judgment around an individual's capacity and ability to participate meaningfully in public processes. Rather than questioning marginalized citizens' ability to participate, this research demonstrates the potentially beneficial nature of engaging marginalized citizens in public processes, including those related to establishing performa nce regimes for programs designed t o serve that group of citizens. Although the research was conducted with people with the lived experience of homelessness, the findings and recommendations stemming from this research have application when working with ot her marginalized populations including, for example, people living in poverty and low income communities, immigrant or migrant populations, individuals identifying as LGBTQ, or communities of color. Engaging marginalized citizens in the process to establi sh performance regimes is beneficial as i t helps provide better context for those data collected. Funders, program

PAGE 140

132 providers, and others without the lived experience may provide some input to interpret data, but including the perspectives of those with the lived experience can reveal something very different. For example, one resident interviewed spoke of not having his income increase since moving in and not being interested in gaining any income at this point. Based on HUD's expectations for residents to increase their incomes (or maintain if one already has adequate income), this would be inte rpreted as a negative outcome. However, t his individual explained that he had a history of alcoholism. He was currently sober, on probation, and pleased he had a per manent place to live. Violating the law would mean a return to jail and a loss of his housing, and the key to avoiding breaking the law was to also avoid drinking. He explained his connection between a lack of income, his sobriety, and his ability to stay housed as follows: Well the biggest thing about alcohol is that it takes money to get it I don't have job, I'm not making any money. I don't drink. Because I can't buy it. So as soon as I get a job, start making money, I start buying alcohol. And then I destroy everything I built Now I've got this great place to live and everything. And, I can stay here, be broke and be happy and be off the streets. With the context this resident is able to provide, the "negative" outcome of no income and no desire to earn an income is reframed as a desire to remain housed. These are the nuances to an understanding of data that only those with the lived experience, who are stigmatized and criticized for their circumstances and choices, can provide. Engaging marginalized citizens in establishing performance regimes, particularly when it is they who benefit from the program in question, also better enables staff to determine where there are inconsistencies in participants' understanding of how their data is used and what t hose data may reveal Privacy was a theme that emerged frequently in conversation. While privacy could arguably be a concern of any individual who is asked to

PAGE 141

133 provide their personal information to others, often individuals who have experienced homelessness are expected to reveal not only their identifying information, but their personal histories repeatedly. With every intake to a new program or service, they may have to re tell their story of how and why they became homeless. Because they are asking for as sistance from others and because a large proportion of their life is lived in public (given their lack of a private residence), these individuals' sense of privacy is often disrupted or disregarded. In a permanent supportive housing community, residents are supposed to be living as if in any other apartment. Given funding requirements, concessions to neighbors who fought against the building, or other reasons, program staff may still be required to more closely monitor residents' whereabouts or collect m ore information on residents as compared to tenant s in market rate rental units. If these measures are all needed and there are no options for changing practices to provide a greater sense of privacy for residents, some of these frustrations or concerns ma y be alleviated if residents understood exactly how and why any data was collected from them or why certain practices were in place. For example, in regards to resident satisfaction, particularly as it concerns residents' sense of safety, some residents ma y feel comforted knowing the building is secure, staff is on site 24 hours a day, and that there are multiple cameras in place. For others without safety concerns, these practices may feel instead like they are being monitored and their privacy is being vi olated. For those individuals, if they were regularly reminded of the positive impact those practices had on those who were concerned for their safety (for instance because of past trauma or abuse), might they reframe the purpose of these practices, react less negatively, and in turn, report less frustration or greater satisfaction with their housing?

PAGE 142

134 Program staff may also benefit from hearing residents' interpretation of different data indicators. Depending on how a resident may define the purpose of an outcome measure, their response to self reported data may change. Understanding residents' knowledge and interpretation of performance data could assist program staff to anticipate any over or under estimates provided in self reported data. This knowledge can assist staff in interpreting data, particularly if any self reported data do not align with data collected from other sources. Finally, engaging marginalized citizens in performance processes enables them to provide input on what matters most to them Residents interviewed spoke of desiring a sense of control. Even if certain indicators are not reported to funders or external stakeholders, there are items that residents may find important, and that by measuring and working to improve through performan ce management efforts, could improve the experiences and lives of those citizens. Marginalized citizens are those who lack political efficacy. Individuals who have experienced homelessness have lacked this sense of efficacy and control for major portions o f their lives. Even if not everything may be under their control in a permanent supportive housing unit, involving them in this work, taking their opinions and input seriously, and actively working to utilize their input for management practices, for chang es in policy, for changes in the design or layout of the building, or for future advocacy efforts can instill a sense of control and a sense of contribution to those who have repeatedly felt their input matters less than others For any marginalized citiz en, this process can contribute to efforts to elevate those citizens' sense of efficacy and ability to participate in public processes. Widespread participation in public processes, including participation by those who are typically marginalized, is critic al for upholding democracy and continuing to build a more equitable state.

PAGE 143

135 Despite its value there are considerations when involving marginalized citizens in the performance process. Previous research has noted there is a cost (in staff time and other res ources) to engaging citizens in public processes. Particularly if citizens leave more frustrated or feeling less heard, program staff may determine there is a greater cost than benefit to this work. To mollify some of these concerns, staff must keep a clea r and ongoing line of communication with citizens. In some cases, staff and residents (or other participating citizens) may collaboratively establish expectations for participating in the performance process. Even if some expectations must be outlined prim arily by staff, these should be made clear from the beginning to avoid any citizen feeling their voice was not heard or their time was wasted. In particular, it would benefit all parties to explicitly express there may be conflicting preferences from among citizens, and a compromise may not always be feasible or appropriate. For every data point collected or every performance management decision made, there should be a rationale that can be shared with citizens, particularly if they are the ones most impact ed by those practices. Providing this rationale can demonstrate to citizens that every effort can be justified and have a purpose; staff are not trying to be nosey or to be controlling by asking for certain pieces of information or implementing certain pra ctices. Documenting these compromises could also help administrators recognize when or if others' preferences are consistently prioritized over those of the marginalized citizens participating. This practice could provide a cue for those administrators to examine how, if others' preferences are consistently adopted over marginalized citizens' preferences, the process may need to be adjusted to ensure marginalized citizens' perspectives are heard and are considered equitably.

PAGE 144

136 Aside from these issues, there may be instances in which being fully transparent around how or why data is collected could negatively impact another aspect of the program. In the case of people with the lived experience of homelessness, the performance measure related to residents' chr onicity and vulnerability is particularly problematic. Individuals experiencing homelessness have traditionally had to prove their worth and a certain level of functioning before being offered housing assistance. If this is still the understood practice am ong people experiencing homelessness, they m ay not be fully transparent about what they are experiencing. The stigma related to severe mental health issues, severe disabilities, or addictions may impact their willingness to share more than needed with prog ram staff, and as a result, any assessment of their vulnerability or level of need may be lower than in actuality. Professionals typically use t he indicator related to one's vulnerability or chronic status to identify those who are the most in need of hous ing. Those who are identified as being the most ill, the most disabled, or having the most barriers are no longer those who are screened out from a housing project. Instead, they are prioritized for housing, and the success of a permanent supportive housin g project relies on the project demonstrating they are serving those most in need in a community. If potential residents knew the actual intention of this data point, the project may be facing the opposite problem that is faced when people downplay any dis abilities or illnesses. Applicants may exaggerate their disabilities or severity of illness to demonstrate a higher vulnerability and move higher in the list of people prioritized for housing. This one example illustrates the persistent issue of engaging c itizens in establishing performance regimes. Although beneficial for several reasons, other considerations will always limit the ability for citizens (including marginalized citizens) to participate in the

PAGE 145

137 same vein as professional staff. While it may be a goal of public programs to engage citizens as thoroughly and authentically as possible, considerations for data quality may draw a line with where their participation should be limited. For public administrators or other professionals working with citizen s to establish performance measures, collect performance data, or use these data, they must understand the interpretations and biases citizens bring with them to anticipate where the process and data quality may be compromised with greater citizen particip ation. To maintain the integrity of the performance data collected, public administrators may choose to limit citizens' participation or withhold some pieces of information, revealing the persistent power differential between professionals and citizens in volved in the process. Citizen participation is a means to increase equity in the public sphere by providing citizens a voice in how their government operates. As citizen engagement gains a stronger hold in the public and non profit spheres, and phrases li ke "nothing about us without us" become mantras of community based efforts, recognizing there may still be limits to this engagement is imperative. The ideal of citizen participation cannot be implemented when efforts to enhance equity impact the effective ness or accuracy of the public process. In the case of permanent supp ortive housing, residents want a sense of con trol and professionals advocate for residents to have a voice in their community. Residents' status es as marginalized citizens whose experie nces and perspectives have been impacted by the stigma they have faced, continue to relegate them to a place of less control than staff of the project. As this cannot be easily avoided, this difference in power should be acknowledged and addressed as part of the public process. The reasons residents have been marginalized may not be the same as before someone who once was marginalized because they were

PAGE 146

138 homeless may now be renting their own apartment but the lack of control still stemming from a place of limited political efficacy should be validated, rather than glossed over or dismissed. Only by explicitly addressing these differences in power and how they may impact citizens' ability to be more fully engaged in certain public processes can public admin istrators maintain citizens' interest in participating in public efforts and identify other opportunities to utilize citizens' ideas, perspectives, and efforts. Limitations Several limitations to this research and its findings exist. This research relied on interviews with individuals residing in permanent supportive housing units at Redtail Ponds in Fort Collins, Colorado. Residents in Redtail Ponds were referred to their housing by either the community mental health center, a local emergency shelter, or the Department of Veteran Affairs (for those units reserved for veterans). Given this process, individuals referred for tenancy were those who were already connected with other service providers and generally participating in those services without issue s For some experiencing homelessness, particularly those who have been homeless for extended periods of time, they may be resistant to work with service providers due to issues with mental illness, previous poor experiences with service providers, a desire to not rely on others for assistance, or other reasons. Other individuals may not be able to engage with certain providers because they have been banned (even if temporarily) from accessing certain services because of violent or threatening behavior. Indi viduals with these tendencies were not likely referred to Redtail Ponds, and therefore not a part of this research study. In addition to the selection bias occurring from the overall tenant selection process for Redtail Ponds, r espondents self selected in to the research project Information on the

PAGE 147

139 opportunity to participate in this research study was provided to all, with 60% of residents opting to participate. It is feasible that those who volunteered to participate were more homogenous or different than the total population of Redtail Ponds. For instance, those who work during typical work hours (during which the interviews took place) may not have been able to participate. In addition, those who feel they have adequate income may not have been incentiviz ed as much as those with lower income levels to receive the gift card for participation. Finally, those with certain disabilities, including some mental health issues, may not have been able to participate as others, despite efforts by the researcher to re duce barriers to participation for all potential subjects. Given the sample size of the residents interviewed as well as the limitation of only interviewing in one housing project in one community, limited generalizations can be drawn. Participants were n ot randomly selected given the tenant selection process for Redtail Ponds and the process to recruit study participants and the findings from this research are not necessarily representative of the perspectives of all Redtail Ponds residents. In addition, this research was conducted approximately one year after the building had opened. Many of the residents had been homeless in Fort Collins while the building was being proposed and developed, and were exposed to more community feedback and input about home lessness, permanent supportive housing, and their future homes than typical. Expanding this research to s upportive housing programs that have been operating for years and are no longer active topics of conversation within the community may yield different perspectives, particularly related to outcomes related to community satisfaction. In addition, some permanent supportive housing exists as "scattered site" meaning individuals are provided rental assistance and supportive services (similar to a "single sit e" project like Redtail Ponds), but

PAGE 148

140 are living in a traditional market rate or affordable housing building. The expected outcomes for this type of supportive housing would remain the same, but the influence of having an entire apartment building serving as this type of housing may impact both professionals' and residents' perspectives on what matters most in measuring the performance of permanent supportive housing. Some of the th emes that emerged for residents, including "safety" and "control" may be refle ctive of realities faced by many marginalized communities, or they may be more specific to those who are marginalized becau se of their homeless status. Examining these research questions in a different context and with different marginalized populations ca n strengthen the theoretical contributions of this line of research. While there are limitations with the sample size and sampling strategy, this research does have relevance beyond the population of people with the lived experience of homelessness. Othe r groups of citizens may lack political efficacy for different reasons, but the recommendations for practice can be applicable when working with a range of citizen groups. Findings from this research can impact the work of non profits and government entiti es when serving other marginalized communities including individuals with disabilities, immigrant or migrant populations, communities of c olor, or low income households, for example. Similarly to the issues with generalizability for residents, the intervie ws with professionals working in homelessness or affordable housing provided a helpful touchpoint to compare perspectives with those marginalized citizens interviewed. Interview subjects were invited to participate by the researcher based on her knowledge of the field of homelessness and organizations and individuals in prominent positions. Specific individuals were invited to participate given their roles at different levels of government and in different

PAGE 149

141 functions. Given this process, sampling for the pro fessionals' interviews was not random, and only limited g eneralizations from these interviews can be drawn. Another limitation centers on the analysis strategy for the research. Q ualitative data is often coded by multiple individuals, with coding schemes checked for intercoder relia bility. As this research study wa s conducted by a single individual, these precautions were not taken. The researcher attempted to address inconsistencies in coding by reviewing every entry again once all interviews had been quo ted, ensuring any themes that emerged later in the coding process could then be identified in earlier interviews. In addition, the researcher conducted queries for word frequencies to confirm no major themes were missed. Future Research There is limited research on engaging citizens, particularly marginalized citizens, in establishing performance regimes. To continue to develop this stream of research, additional studies could examine a multitude of population characteristics or performance measures in ad dition to conducting research with more individuals who have experienced homelessness. These paths would strengthen current theoretical findings and indicate whether some of the themes that have emerged among the residents interviewed such as "safety" and "control," hold for other marginalized populations or are unique to the experience of homelessness. Another useful line of research would more closely examine the power dynamics at play between public administrators and citizens, inclu ding marginalized citizens, which may influence what performance measures are established and whose preferences carry the most weight. In this research, professionals and residents similarly expressed a desire for residents to play a role in shaping performance measures and program components of the permanent supportive housing in which they resided. The language used to express this desire differed

PAGE 150

142 between interview groups. Professionals spoke of giving residents a voice and a say in how their housing looked, felt, and oper ated. Residents spoke of wanting power and control in their housing. For professionals, they want residents to be involved, but their language reinforces the fact that those in the professional roles do have power and control over many aspects of the housi ng in which people are living Residents who have been beholden to others' schedules and rules for the duration of their homelessness experience still are not able to always feel they have a sense of control even though they are now no longer homeless and living in housing that is intended to feel like a "typical" apartment. These are the types of power differences that should be acknowledged and addressed during the course of engaging citizens. The power dynamics between professionals and citizens, especi ally marginalized citizens, are not often addressed. Future research could test different approaches for acknowledging and working through the power differences felt between professionals and the citizens they are engaging in public processes. Findings on how these different approaches impact the outcomes of engaging citizens could yield interesting theoretical insights as well as practical tools for administrators. Research regarding citizen participation in public processes acknowledges there are costs to engaging citizens. Many professionals interviewed in this research project also commented on those things program administrators should consider when engaging residents in certain aspects of program development, implementation, and evaluation. Knowing t hat other aspects of a public process, such as efficiency, are often diminished by more thoroughly engaging citizens, the field would benefit from understanding these relationships and to what extent broader or deeper citizen engageme nt actually impacts ot her value s of the public process. With this knowledge, future research could be used to help identify the

PAGE 151

143 thresholds at which citizen engagement is the most useful and can increase equity without seriously depleting the efficiency or effectiveness of the p ublic process and its resulting performance regimes. Future research should also more closely examine the process through which funders or those overseeing public programs design performance measures and to what degree considerations for citizen participa tion are taken. While those professionals interviewed generally felt it was useful to engage residents in establishing performance measures, this preference is not currently enforced in any of HUD's requirements. Only general expectations for participation on oversight committees or boards for the Continuum of Care process is mentioned. Funders have influence on what is emphasized in any program's design or evaluation. Those items that funders establish are the most important will be at the center of any pr ogram's implementation. Future research could better tease out the influence funders, other stakeholders, or other agency or program aspects (such as an organization's mission or values) have on the willingness to engage citizens in certain aspects of prog ram design, implementation, and evaluation. This line of research can lead to a practical understanding of what could be changed to more quickly or effectively encourage public programs to engage citizens in their decision making processes continuing to b uild the field's knowledge of how efforts to build equity are encouraged.

PAGE 152

144 REFERENCES Adler, G. (2012). Community Action and Maximum Feasible Participation!: An Opportunity Lost but Not Forgotten for Expanding Democracy at Home. Notre Dame Journal of Law, Ethics & Public Policy 8 (2), 547 571. Ansell, C., & Gash, A. (2008). Collaborative Governance in Theory and Practice. Public Administration Research and Theory 18 543 571. Barberis, P. (1998). The New Public Management And A New Accountability. Public Administration 76 (3), 451 470. doi:10.1111/1467 9299.00111 Barnes, M., Newman, J., Knops, A., & Sullivan, H. (2003). Constituting "The Public" in Public Participation. Public Administration 81 (2), 379 399. Barnes, M., Newman, J., & Sullivan, H. (2006). Discursive Arenas: Deliberation and the Constitution of Identity in Public Participation at a Local Level. Social Movement Studies 5 (3), 193 207. doi:10.1080/14742830600991487 Behn R. D. (2003). Why Measure Performance? Different Purposes Require Different Measures. Public Administration Review 63 (5), 586 606. Belcher, J. R., & DeForge, B. R. (2012). Social Stigma and Homelessness: The Limits of Social Change. Journal of Human B ehavior in the Social Environment 22 (8), 929 946. doi:10.1080/10911359.2012.707941 Bryer, T. A. (2013). Public Participation in Regulatory Decision Making. Public Performance & Management Review 37 (2), 263 279. doi:10.2753/PMR1530 9576370204 Bryson, J M., Quick, K. S., Slotterback, C. S., & Crosby, B. C. (2012). Designing Public Participation Processes. Public Administration Review 73 (1), 23 34. doi:10.111/j.1540 6210.2012.02678.x.Designing City of Chicago. (2014). City of Chicago Data Portal. Ret rieved from https://data.cityofchicago.org/ Cohn Berman, B. J. (2005). Listening To The Public: Adding the Voices of the People to Government Performance Measurement and Reporting New York, NY: Fund for the City of New York. Cohn Berman, B. J. (2008). Involving the Public in Measuring and Reporting Local Government Performance. National Civic Review 97 (1), 3 10. doi:10.1002/ncr Cohn Berman, B. J. (2012). When Governments Listen: Moving Toward Publicly Engaged Governing. New York, NY: Fund for the City of New York.

PAGE 153

145 Collins, P., & Barker, C. (2009). Psychological help seeking in homeless adolescents. International Journal of Social Psychiatry, 55 (4), 372 3 84. doi:10.1177/0020764008094430 Colorado Coalition for the Homeless (2006). Denver Housing First Collaborative: Cost benefit analysis & program outcomes report. Retrieved from http://www.coloradocoalition.org/!userfiles /Housing/Executive_Summary_DHFC_st udy .pdf Coulibaly, M., Green, R. D., & James, D. (1998). Segregation in federally subsidized low income housing in the United States Westport, CT: Praeger Publishers. Dang, M. T., & Miller, E. (2013). Characteristics of natural mentoring relationships from the perspectives of homeless youth. Journal of Child and Adolescent Psychiatric Nursing, 26 (4), 246 253. doi:10.1111/jcap.12038 Day, D. (1997). Citizen participation in the planning process: An es sentially contested concept? Journal of Planning Literature, 11 (3): 421 34. deLeon, P. (1997). Democracy and the Policy Sciences. Albany, NY: State University of New York Press. Dubnick M. (2005). Accountability and the Promise of Performance: In Search of the Mechanisms. Public Performance & Management Review 28 (3), 376 417. Easton, D., & Dennis, J. (1967). The child's acquisition of regime norms: Political efficacy. The American Political Science Review, 61(1), 25 38. Elkins, Z., Ginsburg, T., & Blount, J. (2008). The citizen as founder: Public participation in constitutional approval. Temple Law Review, 81(2), 361. Emerson, K., Nabatchi, T., & Balogh, S. (2011). An Integrativ e Framework for Collaborative Governance. Journal of Public Administration Research and Theory 22 (1), 1 29. doi:10.1093/jopart/mur011 Epstein, P., Wray, L., & Harding, C. (2006). Citizens as Partners In Performance Management. Public Management 88 (10), 18 22. Frederickson, H. G. (1971). Toward a new public administration. In F. Marini (Ed.), Toward a New Public Administration: The Minnowbrook Perspective (pp. 309 331). Scranton, PA: Chandler. Frederickson, H. G. (1976). The Lineage of New Public Administration. Administration & Society 8 (2), 149 174. doi:10.1177/009539977600800202

PAGE 154

146 Frederickson, H. G. (1996). Comparing the Reinventing Government Movement with the New Public Administration. Public Administration Review 56 (3), 263 270. F ung, A. (2006). Varieties of participation in complex governance. Public Administration Review, 66 (6): 66 75. Goffman, E. (1963). Stigma: Note son the management of spoiled identities. New York, NY: Simon & Schuster. GPRA (1993). Government Performanc e Results Act. Retrieved from https://www.whitehouse.gov/omb/mgmt gpra/gplaw2m Grosso, A. L., & Van Ryzin, G. G. (2011). How citizens view government performance reporting. Public Performance & Management Review, 35 (2), 235 250. Groton, D. (2013). Are housing first programs effective? A research note. Journal of Sociology and Social Welfare, 40(1), 51 63. Gupte, M. (2010). Accommodating marginalized groups in the planning process. In N. Harrison (Ed.), National, Regional and Gl obal Institutions, Infrastructures and Governance, vol. 1. EOLSS Publishers. Guy, M. E., & Mccandless, S. A. (2012). Social Equity: Its Legacy, Its Promise. Public Administration Review 72 (s1), s5 s13. doi:10.111/j.1540 6210.2012.02635.x.Social Halachmi, A., & Holzer, M. (2010). Citizen Participation and Performance Measurement: Operationalizing Democracy Through Better Accountability. Public Administration Quarterly 34 (3), 378 399. Hatry H. P. (2010). Looking into the Crystal Ball: Performance Management over the Next Decade. Public Administration Review 70 (s1), s208 s211. doi:10.1111/j.1540 6210.2010.02274.x Heikkila, T., & Roussin Isett, K. (2007). Citizen Involvement and Performan ce Management in Special Purpose Governments. Public Administration Review 67 (2), 238 248. Hirschman, A. O. (1970). Exit, voice, and loyalty: Responses to decline in firms, organizations, and states. Cambridge, MA: Harvard University Press. Hood, C. (1 991). A Public Management for All Seasons? Public Administration 69 3 19. Housing Catalyst (2013a). Redtail Ponds Good Neighbor Workshop [presentation]. Housing Catalyst. Presented on August 12, 2013. Housing Catalyst (2013b). Redtail Ponds Good Neighbor Statement of Operations. Housing Catalyst.

PAGE 155

147 Housing Catalyst (2015). Good Neighbors, Amazing Communities. Housing Catalyst. Retrieved from https://housingcatalyst.com/about us/good neighbor commitment/ HUD (1992). The final report of the National Commission on Severely Distressed in Public Housing. Washington, D.C. HUD (2002). Fiscal year 2002: Performance and accountability report. Washington, D.C. HUD (2005). Annual performance plan: Fiscal year 2006. Washington, D.C. HUD (2 012). FY 2012 annual performance report & FY 2014 annual performance plan. Washington, D.C. HUD Exchange (2012). Homeless Emergency Assistance and Rapid Transition to Housing. Federal Register Vol. 77, No. 147; 24 CFR Part 578. Retrieved from https://www.hudexchange.info/resources/documents/CoCProgramInterimRule.pdf Irvin, R. A., & Stansbury, J. (2004). in Participation Decision Making!: Worth the Effort!?, 64 (1), 55 65. Jo, Y. N. (2013). Psycho social dimensions of poverty: When poverty bec omes shameful. Critical Social Policy 33 (3), 514 531. doi:10.1177/0261018313479008 Johnstone, Q. (1958). Federal Urban Renewal Program. Faculty Scholarship Series. Paper 1896. http://digitalcommons.law.yale.edu/fss_papers/1896 Kennedy, S., Grewal, M., Roberts, E. M., Steinauer, J., & Dehlendorf, C. (2014). A qualitative study of pregnancy intention and the use of contraception among homeless women with children. Journal of He alth Care for the Poor and Underserved, 25 (2), 757 770. doi:10.1353/hpu.2014.0079 Kim, S., O'Leary, R., Van Slyke, D. M., Frederickson, H. G., & Lambright, W. H. (2010). The legacy of Minnowbrook. In R. O'Leary, D. M. Van Slyke, & S. Kim (Eds.), The Future of Public Administration Around the World: The Minnowbrook Perspective (pp. 1 16), Washington, D.C.: Georgetown University Press. King, C. S., Feltey, K. M., & Susel, B. O. (1998). The Question of Participation: Toward Authentic Public Participat ion in Public Administration. Public Administration Review 58 (4), 317. doi:10.2307/977561 Lavertu, S. & Moynihan, D. P. (2012). Agency political ideology and reform implementation: Performance management in the Bush administration. Journal of Public Administration Research and Theory, 23 (3), 521 549.

PAGE 156

148 Leech, N. L., & Onwuegbuzie, A. J. (2008). Qualitative data analysis: A compendium of techniques and a framework for selection for school psychology research and beyond. School Psychology Quarterly 23 (4), 587 604. doi:10.1037/1045 3830.23.4.587 Leroux, K. (2009). Paternalistic or Participatory Governance? Examining Opportunities for Client Participation in Nonprofit Social Service Organizations. Public Administration Review 69 (3), 504 517. Lim, H. (2006). Representative bureaucracy: Rethinking substantive effects and active representation. Public Administration Review, 66(2), 193 204. doi:10.1111/j.1540 6210.2006.00572.x Matthes, M. (2008). The Des Moines Experience with Citizen Informed Perfo rmance Measurement and Reporting. National Civic Review 97 (1), 13 20. doi:10.1002/ncr McKinney Vento. (2009). McKinney Vento Homeless Assistance Act. www.onecpd.info/resources/documents/homelessassistanceactamendedbyhearth.pdf Meyer, M. W., & Gupta, V. (1994). The performance paradox. Research in Organizational Behavior. 16 309 369. Mosher, F. C. (1982). Democracy and the public service. New York: Oxford University Press. Moynihan, D. P. (1970). Maximum Feasible Misunderstanding. New York: The Free Press. Moynihan, J. P. (2008). The Dynamics of Performance Management. Washington, D. C.: Georgetown University Press. Moynihan, J. P. (2009). Through a glass, darkly: Understanding the effects of performance regimes. Public Performance & Management Review 32 (4), 592 603. Moynihan, D. P., Fernandez, S., Kim, S., Leroux, K. M., Piotrowski, S. J., Wright, B. E., & Yang, K. (2011). Performance Regimes Amidst Governance Complexity. Journal of Public Administration Research and Theory 21 (Supplement 1), i141 i155. doi:10.1093/jopart/muq059 NAEH (2006). What is housing first? National Alliance to End Homelessness. Retrieved from http://b.3cdn.net/naeh/b974efab62feb2b36c_pzm6bn4ct.pdf National Center for Civic Innovation. (2014). Government Trailblazer Program. Retrieved at http://www.civicinnovation.o rg/trailblazer_history.html National Coalition for the Homeless (2006, June). McKinney Vento Act (Fact Sheet No. 18). Washington, D.C. Retrieved from http://www.nationalhom eless.org/publications/facts/McKinney.pdf

PAGE 157

149 Neale, J., & Stevenson, C. (2014). Homeless drug users and information technology: A qualitative study with potential implications for recovery from drug dependence. Substance use & Misuse, 49 (11), 1465 1472. doi:10.3109/10826084.2014.912231 Newcomer, K. E. (Forthcoming). From outputs to outcomes: Conceptualizing government performance. In M. E. Guy & M. M. Rubin (Eds.), Public Administration Evolving: From Foundations to the Future. New York, NY: Routledge. Newman, J., Barnes, M., Sullivan, H., & Knops, A. (2004). Public Participation and Collaborative Governance. Journal of Social Policy 33 (02), 203 223. doi:10.1017/S0047279403007499 Onwuegbuzie, A. J., & Leech, N. L. (2007). A Call for Qualitative Pow er Analyses. Quality & Quantity 41 (1), 105 121. doi:10.1007/s11135 005 1098 1 Osborne, D. & Gaebler, T. (1992). Reinventing Government. Reading, MA: Addison Wesley. Ostrom, E. (1990). Governing the Commons. New York: Cambridge University Press. O'Toole, L. J. (1977). Lineage, Continuity, Frederickson, and the "New Public Administration." Administration & Society 9 (2), 233 252. doi:10.1177/009539977700900206 Phelan, J., Link, B. G., Moore, R. E., & Stueve, A. (1997). The Stigma of Homelessness: The Impact of the Label "Homeless" on Attitudes Toward Poor Persons. Social Psychology Quarterly 60 (4), 323 337. Purdy, S., & Kwak, N. H. (2007). Introduction: New Perspectives on Public Housing Histories in the Americas. Journal of Urban History 33 (3 ), 357 374. doi:10.1177/0096144206297128 Putnam, R. D. (2000). Bowling Alone: The Collapse and Revival of American Community. New York: Simon & Schuster. Rosenbloom, D. H., & Gong, T. (2013). Coproducing "Clean" Collaborative Governance. Public Performan ce & Management Review 36 (4), 544 571. doi:10.2753/PMR1530 9576360403 Rubin, L. (1969). Maximum Feasible Participation!: The Origins, Implications, and Present Status. Annals of the American Academy of Political and Social Science 385 14 29. Schneider, A., & Ingram, H. (1993). Social Construction of Target Populations: Implications for Politics and Policy. The American Political Science Review 87 (2), 334 347.

PAGE 158

150 Shester, K. L. (2011). American public housing's origins and effects. (Doctoral dis sertation). Retrieved from etd.library.vanderbilt.edu. Singleton, R. A. & Straits, B. C. (2010). Approaches to social research (5th ed.). New York: Oxford University Press. Spray Kinney, A. (2008). Current Approaches to Citizen Involvement in Performan ce Measurement and Questions They Raise. National Civic Review 97 (1), 46 54. doi:10.1002/ncr Stivers, C. (1990). The Public Agency as Polis: Active Citizenship in the Administrative State. Administration & Society 22 (1), 86 105. doi:10.1177/0095399790 02200105 Stoloff, J. A. (2004, August). A Brief History of Public Housing Paper presented at the annual meeting of the American Sociological Association, San Francisco, CA. Retrieved from http://reengageinc.org/research/brief_history_public_housing.pdf Teddlie, C., & Yu, F. (2007). Mixed Methods Sampling: A Typology With Examples. Journal of Mixed Methods Research 1 (1), 77 100. doi:10.1177/2345678906292430 Thomas, J. C. (2012). Citizen, customer, partner: Engaging the public in public management. Armonk, NY: M.E. Sharpe. Thompson, F. J., & Riccucci, N. M. (1998). Reinventing Government. Annual Review of Political Science 1 (1), 231 257. doi:10.1146/annurev.polisci.1.1.231 USICH (2010). Opening doors: Federal strategic plan to prevent and end ho melessness. United States Interagency Council on Homelessness. Washington, D.C. Van Dooren, W. (2011). Better performance management: Some single and double loop strategies. Public Performance & Management Review, 34 (3), 420 433. Van Thiel, S., & Leeuw, F. L. (2002). The Performance Paradox in the Public Sector. Public Performance & Management Review 25 (3), 267 281. Webler, T., & Tuler, S. (2000). Fairness and Competence in Citizen Participation: Theoretical Reflections from a Case Study. Admin istration & Society 32 (5), 566 595. doi:10.1177/00953990022019588 Woolum, J. (2011). Citizen Involvement in Performance Measurement and Reporting. Public Performance & Management Review 35 (1), 79 102. doi:10.2753/PMR1530 9576350104 Yang, K., & Pandey, S. K. (2011). Further Dissecting the Black Box of Citizen Participation: When Does Citizen Involvement Lead to Good Outcomes? Public Administration Review 71 (6), 880 892.

PAGE 159

151 Yetano, A., Royo, S., & Acerete, B. (2010). What is driving the increasing presence of citizen participation initiatives? Environment and Planning C: Government and Policy 28 (5), 783 802. doi:10.1068/c09110

PAGE 160

152 A PPENDIX A Comparisons of Resident and Professional Responses Chart 1. Comparison of Me an Scores for Professionals and Residents 0 1 2 3 4 5 Retention in Housing Chronicity/Vulnerability Resident Satisfaction Reduced Jail Bed Days Increased Incomes of the Residents Reduced Shelter Bed Days Reduced Emergency Department Visits Community Satisfaction Voluntary Participation in Services Professional Mean Score Resident Mean Score

PAGE 161

153 Chart 2. Proportion of Respo ndents Ranking Measures in Top Three 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% % Residents Ranking in Top 3 % Professionals Ranking in Top 3

PAGE 162

154 Chart 3. Proportion of Respondents Ranking Measure a Five 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% % Residents Ranking a 5 % Professionals Ranking a 5

PAGE 163

155 A PPENDIX B Interview Tool for Residents First, I'm going to start by asking you a few questions about yourself. Before moving in here, how long had you been homeless? How old are you now? Are you a veteran or have you ever served in the armed forces? Now, I'd like to ask you some questions about Redtail Ponds and your experience living here. When did you move into Redtail Ponds? Why did you opt to move in here instead of remaining homeless or waiting for another housing option to open up? How would you describe Redtail Ponds to someone who has never heard of it? What are things you think make Redtail Ponds successful? What are the things you think make Redtail Ponds less successful? You just talked about how you determine if Redtail Ponds is suc cessful or not. I'm now going to read through some other things that other apartment complexes like Redtail Ponds may use to tell if they're successful or not. I'd like you to tell me how important you think each one is and why. I'll ask you to rank eac h one on a scale from 1 to 5, with 1 being "not at all important" and 5 being "absolutely important/critical." Here's the scale written down for you, if it's helpful to have it to look at. 1 = Not at all important 2 = A little important 3 = Fairly import ant 4 = Very important 5 = Absolutely important/critical *After interview subject provides ranking, researcher asks why they think that performance measure carries the importance assigned (e.g., Why is that "very important" to you?), and why it was not higher or lower (e.g., And why wasn't that quite at that 5 level, "absolutely important or critical", for you?), if applicable.

PAGE 164

156 1. How many residents stay here instead of moving out or being evicted and becoming homeless again 2. How much incomes of the residents increase after moving in (from a job or benefits) 3. How many new residents were homeless for a very long time or had multiple illnesses or disabilities 4. How many fewer nights residents would spend in jail (as compared to when they were home less) 5. How many residents voluntarily choose to participate in services 6. How satisfied or happy residents are living here 7. How satisfied or happy the surrounding neighborhood or community is with the building and housing project 8. How many fewer nights residents would use a shelter (as compared to when they were homeless) 9. How many fewer times residents would go to the emergency department (as compared to when they were homeless) Are there things that you've seen change in your life as a resu lt of living here? Of all of the outcomes we've talked about (including any just added), which three do you think are the most important for permanent supportive housing projects to collect? Why these three? Some of organizations that help fund programs like Redtail Ponds tend to look mostly at two outcomes: whether people have stayed in housing (housing retention) and whether people's income increased. Do you think these provide enough information to determine whether a housing program like Redtail Ponds is successful? Why or why not? If staff here wanted help deciding what they should be measuring and how they should measure it, would you, as a resident here, be interested in working with them and helping them to make those decisions? Do you hav e any final thoughts you'd like to share with me about your experience living here or how you think we can show whether or not programs like Redtail Ponds are successful?

PAGE 165

157 A PPENDIX C Interview Tool for Professionals First, I'd like to get some information related to your experience and professional background. How long have you been in your current position? How long have you been working in the affordable housing or homelessness field? In the course of your career, have you ever worked in a direct servi ce position, working with clients who are homeless? Now, I'd like to talk to you about permanent supportive housing and the outcomes and performance measures we see attached to these types of housing projects How do you (and your colleagues in your org anization) use outcomes or performance information related to permanent supportive housing, if at all? Now, I'm going to read off some different outcomes related to permanent supportive housing. I want you to tell me how important you think each outcome is for a PSH project to collect, on a scale from 1 to 5, with 1 being "not at all important" and 5 being "absolutely important/critical." Here's the scale written down for you, if it's helpful to have it to reference. We'll have the chance in a minute for you to share more of your thoughts on each of these outcomes. 1 = Not at all important 2 = A little important 3 = Fairly important 4 = Very important 5 = Absolutely important/critical 1. Retention in housing 2. Reduced jail bed days 3. General community satisfaction with project 4. Reduced shelter bed days 5. Resident satisfaction 6. Reduced emergency department visits 7. Voluntary participation in case management or other services

PAGE 166

158 8. Increased incomes of the residents 9. Chro nicity and/or vulnerability of new residents Now, I'd like to go back to each one to better understand why you think that outcome carries the importance you've stated. *Researcher re reads each outcome and asks "Why do you think this outcome is (________ __) important?", filling in the blank with the interview subject's previous response. 1. Retention in housing 2. Reduced jail bed days 3. General community satisfaction with project 4. Reduced shelter bed days 5. Resident satisfaction 6. Reduced emergency department visits 7. Voluntary participation in case management or other services 8. Increased incomes of the residents 9. Chronicity and/or vulnerability of new residents Aside from these nine outcomes we've been talking about, are there oth er outcomes that you think would be very important or critical to collect? Of all of the outcomes we've talked about, including those you just added, which three do you think are the most critical for permanent supportive housing projects to collect? Why these three? I'm also asking these questions of residents of PSH. Which three do you think a resident of PSH would say are the most important? HUD's required performance measures for PSH relate to housing retention and income growth. To what extent do t hese outcomes adequately determine whether a housing project is successful and performing well?

PAGE 167

159 Do you have any knowledge or sense of why HUD has selected these performance measures as the primary ones for PSH projects to collect? T his research is exami ning citizens' roles in shaping performance measures and outcomes in the public sector. Do you see a role for residents of PSH to participate in developing performance protocols (in other words, informing what is measured and how)? If yes, why? If no, why not? Do you have any final thoughts on permanent supportive housing, the performance measures associated with PSH, or the idea of involving residents in forming those performance measures?

PAGE 168

160 APPENDIX D Codebook Only those themes that emerged in at least three interviews are included. Code Description CantWork Indication that many clients can't work; have too many disabilities or barriers CommEngage Indication of community engagement/support or lack of engagement broadly Connections Mention of outcome or need to reconnect with family, friends, community, etc. Control Mention of having or lacking control over environment/home/surroundings, etc.; importance of having a sense of control Core Performance measure/outcome is at the core of PSH; what it's all about; the end game; etc. CostBenefit Economic/cost benefit to a certain outcome/the program Easy Outcome would be easy to track Entitled Indication of residents acting entitled, that they deserve something for nothing, or that they do not hav e a right to complain FunderLed Outcomes are designated by funder FundingTies Outcome is tied to funding for program or individual FuzzyOutcome Outcome isn't clear or can be manipulated GranteeLed Program aspects (include performance measures) driven by grantee/program Growth Resident shows progress, growth Health Indication that health of residents is important/would be valuable outcome to see/show; or improved health is shown through outcome Individualized Goals are individualized; tailored for each resident NAToAll Outcome doesn't apply to all residents NeedsMet Indication of importance of having residents' needs met

PAGE 169

161 NIMBY Expression of "not in my backyard" attitude from community or surrounding neighborhood ProgDesign Metric is marker of basic program design or implementation, not an outcome or performance measure ResChoice Indication that residents should/do have a choice in their apartment, participation in services, how their life looks, etc. ResImpact Impact to residents is important (more important than impact elsewhere) Safety Indication of safety being important Scarcity Resources are scarce SelfSufficiency Mention of self sufficiency Stability Mention of client becoming more stable or independent; "self sufficiency" not used o r mentioned as people aren't there or can't get to full self sufficiency Stigma Indication/recognition of stigma attached to homelessness, mental health, etc. Sustainability Sustainability of project Tension Hard to get along with others; indication of tension among residents Time Relevance or potential to see change with outcome changes over time Trigger Someone/something can trigger PTSD or past trauma Universal Outcome applies to all or most Voice Residents have a voice, have something to say about their living situation Well Being Description of outcome related to overall well being WelfareDep Client(s) will always be dependent on social services, welfare, etc.