Generating and Assessing Community Stabilization Data: The Role of the Practitioner
The other articles in this compilation focus principally on ways of using existing data to help shape and adapt strategies to stabilize neighborhoods that have been hit hard by foreclosures and other forms of economic disinvestment. Unfortunately, the availability and quality of neighborhood-level data vary widely across the country. Urban areas such as Chicago and Cleveland benefit from a rich array of information that can document changes on a sub-neighborhood level. In contrast, more rural areas may have data available only on a town or even a county level.
Even in areas where good public data are available, the information is frequently of limited usefulness in assessing the effectiveness of the entities engaged in the stabilization and revitalization of the subject communities. To assess their programmatic effectiveness, community development organizations frequently must determine the information they need and then collect it themselves. If done well, such data collection helps the practitioners refine their programs and activities to maximize their impacts. It can also build upon and enrich any existing neighborhood economic and social database.
This article focuses on ways in which organizations operating in less data-rich environments can track the effectiveness of their programs and their ability to stabilize and revitalize troubled neighborhoods in their markets. We use a case study of the HOPE family of organizations (the Hope Enterprise Corporation, Hope Credit Union, and Mississippi Economic Policy Center), which is working to promote economic development and community revitalization in the long-distressed Mid-South region of the country. Our case study examines how HOPE has relied on a mix of quantitative and qualitative data to track and understand local changes in employment, resident wealth, and investor perceptions--three critical components of sustainable community economic development. We highlight the critical role of thoughtful data collection on the part of a local or regional practitioner, both in assessing organizational effectiveness and in contributing to a broader understanding of community economic development and neighborhood improvement among policymakers.
HOPE Entities Target Revitalization in Mid-South
The Mid-South region of the country has long struggled to overcome the challenges of entrenched poverty. Consisting of Arkansas, west Tennessee, Louisiana, and Mississippi, the region has consistently lagged the rest of the nation on virtually all of the standard measures of social and economic well-being. Louisiana and Mississippi, for example, are almost invariably among the bottom five states with regard to poverty rates, median family income, and educational attainment. Conditions are particularly troubling for many of the region's African Americans, for whom good jobs and opportunities to build wealth remain elusive. One of the challenges in combating these problems is that much of the area is rural, with relatively few major areas with large employers.
One of the factors that hampered the region's development has been the inability of many lower-income individuals and small businesses to access affordable capital--money they need for home purchase and rehabilitation, equipment and inventory purchase, and working capital. As a way of addressing that need, the Foundation for the Mid South established the Enterprise Corporation of the Delta (ECD) in 1994. Now called the Hope Enterprise Corporation (www.hope-ec.org ), ECD started out as a small business loan fund serving companies in 58 counties and parishes throughout the Mississippi delta. A year later, ECD helped found the Hope Community Credit Union as a way of helping predominantly low-income individuals and families in the region access basic financial services, build savings, and access basic credit from an insured depository institution. Today, the two entities (together known as HOPE) collectively constitute a $200 million community development financial institution (CDFI), credit union, and policy center with 26,000 members and 20 locations in the four-state region. HOPE connects underserved entrepreneurs to start-up and expansion capital, families with limited savings to responsible mortgages, and individuals to affordable financial services in communities where only high-cost alternatives exist.
The Mid-South region has never had a particularly strong community development infrastructure. While certain municipalities collect a reasonable amount of potentially relevant data, there is a lack of consistency in the collection; a general dearth of timely, community-specific information; and no good information clearinghouse that makes the available information especially user-friendly. As a result, HOPE has consistently had to conduct its own primary data collection in order to assess the extent to which its activities have made a difference in the market.
Indicators of Change That HOPE Tracks
In measuring the effectiveness of its programs and the change that is taking place in its targeted communities, HOPE tracks a variety of indicators associated with its small business, mortgage, and consumer lending programs. Nearly all of the tracked loan-level information is stored in the loan operations databases for each respective area of lending. Additional impact data collected from surveys are input into a separate impact database. The data are linked through a unique loan-identification number.
Table 1 provides an overview of the community impact data HOPE collects and the methods it uses to do so. The second column of the table provides both a rationale for why certain programmatic activity was needed (i.e., the problem that HOPE sought to address) as well as the anticipated benefit that would accrue from successful implementation of the programmatic approach. The three major areas of change tracked by HOPE and the processes it uses to collect relevant data are discussed in more detail below.
|Activity||Purpose/community benefits||Indicators||Collection methods/data sources|
|Small business lending||
With the Mid-South's high poverty rates and low household and family incomes, HOPE recognized the importance of promoting business and employment growth in the region. Indeed, the economic health and sustainability of any community depends in large part on the availability of good and sustainable jobs for its residents. HOPE therefore requires each business that borrows money from one of its organizations (a "business borrower") to report annually on its current number of full-time and part-time employees. To the extent that the borrower is a developer or property manager, it is required to obtain such information from any commercial or nonprofit tenants in the development.
The simple availability of jobs in an area does not necessarily lead to an improvement in conditions for local residents; job quality also matters. HOPE's surveys therefore ask each business borrower to provide information about the full-time workers' wages. HOPE then compares the reported wages to the local Self-Sufficiency Standard, defined as the amount a household must earn to cover all basic expenses without any public or private assistance. (Described in more detail below, the standard is calculated using publicly available data.) HOPE's surveys also ask for information about the number of full-time workers receiving various benefits, from paid vacation and sick days to employer-paid health insurance and matching 401(k) or 403(b) contributions.
Even though there may be good jobs in an area, it is difficult to determine the beneficiaries of those jobs. It would be helpful to know how many of the jobs were going to local residents and thus helping to enhance the surrounding community's economic well-being. Obtaining data on individual employees' addresses is both costly and impractical, however; among other things, employers and employees tend to balk at such requests out of privacy concerns. As a result, HOPE is currently developing ways to determine how much of a given assisted business's workforce resides locally or commutes long distances.
Poverty alleviation and sustainable economic development ultimately depend on increasing the wealth of local residents and their communities. Tracking the increases requires an understanding of resident assets and the changes in those assets over time. As a credit union, HOPE has access to a range of financial information on its members. It can run reports that show aggregate changes in members' account balances, one indication of household wealth. When considering someone for a loan, the credit union pulls a credit report that includes the applicant's current credit score, another measure of an individual's economic health. Because many of its members take out multiple consumer loans, HOPE can track changes in credit scores over time. HOPE also surveys its credit union members about their current and prior relationships, if any, with other insured depository institutions--relationships that are critical for people to build and maintain financial assets. The survey questions on prior banking relationships allow HOPE to make inferences about how much members were spending on financial services before joining the credit union.
HOPE augments the information from its database with data gleaned from surveys. Each year, for example, HOPE surveys a subset of its residential borrowers. Beginning this fall, the survey will ask homeowners for information about their home's current value and improvements they have made to the property since purchasing it. The survey will also request information on the household's current level of non-house-related assets and debt. These data will be supplemented with repayment history and amortization schedules, helping HOPE estimate the extent of wealth accumulation among its borrowers.
HOPE also operates a foreclosure prevention program. During an initial counseling meeting, HOPE officials obtain information on the affected individual's financial condition and his or her desired outcome for the counseling engagement. HOPE defines success in the foreclosure mitigation program as facilitating the attainment of a client's desired outcome (avoiding a foreclosure, for example). By tracking participants over time, HOPE can measure the long-term effects of its counseling program on wealth accumulation.
The stabilization and revitalization of a community does not happen overnight. Before committing to invest (or remain invested) in an area, people have to be convinced of the investment's likely positive return. Changing potential investors' perception of a troubled community is therefore a critical first step toward stabilizing the area.
Many of the measures of new investment in an area (housing starts, loans originated, building permits issued, and so forth) are actually lagging indicators. They reflect decisions of individuals and institutions that may have been years in the making. One of the keys in identifying and tracking local change is to determine when and why investors' perceptions of the area change.
To help understand these changes and frame the broader implications of its work, HOPE supplements its quantitative data collection with qualitative approaches. As part of its borrower surveys, HOPE asks about perceptions of the surrounding community and changes therein, including public safety, the quality and affordability of local housing, the extent of interpersonal trust, and the quality of local educational institutions. HOPE augments the survey data with more in-depth interviews and case studies on specific transactions, focusing on the particular project and community factors that contribute to success.
In pulling the qualitative information together, HOPE's evaluation team works hand in hand with its loan production staff. HOPE has found that loan officers often can obtain better information from the borrowers because of the existing relationships they have; in the absence of such relationships, borrowers tend to be more guarded in speaking with evaluators.
How HOPE Uses Data
The information that HOPE collects has multiple uses. It helps the organization quantify the need and demand for its programs and services, as well as assess the success of its efforts. HOPE's research team aggregates the data quarterly and annually for review by the organization's senior management, board of directors, and investors. HOPE has used the information to make changes in its program design, advocate for new resources, and educate individuals and policymakers about conditions throughout HOPE's market area. The following subsections provide examples of the various uses and benefits of the data.
Assessing Program Effectiveness
When HOPE was founded, it focused its commercial lending on the manufacturing sector. At the time, manufacturing jobs in the South offered relatively higher wages than those in other sectors, and they frequently provided health insurance. After lending in the sector for a few years, HOPE analyzed internal loan data as well as regional business migration data. Interviews with production staff and actual business owners revealed two important trends. First, much of the work it was doing to support existing manufacturers was not enhancing access to capital for women- and minority-owned businesses; most of the money was going to companies owned by white males. Second, the effects of globalization were causing an increasing number of manufacturers to leave the region, reducing the number of potential deals and exacerbating weak market conditions for manufacturers in HOPE's portfolio. As a result, HOPE diversified its commercial lending to include other sectors and incorporated lending targets for women- and minority-owned businesses into its annual work plan. HOPE now works to target more than one-third of its loans each year to women- and minority-owned businesses.
Demonstrating Need and Attracting Additional Resources
Following Hurricane Katrina, the state of Mississippi designed a number of programs to support the rebuilding of homes on its Gulf Coast. One of the initial recovery programs (Phase I) provided up to $150,000 to single-family homeowners who suffered storm-surge damage but were not located in a flood zone and had homeowners insurance. However, as part of a subsequent recovery program (Phase II), the state initially indicated that it might only provide up to $50,000 to homeowners in the same situation as those in Phase I but who lacked insurance. Many of those prospective Phase II homeowners had low incomes. HOPE demonstrated that the proposed grant awards for Phase II would particularly disadvantage moderate- and middle-income families in the rebuilding process by creating gaps between recovery costs and awards. HOPE's analysis, in conjunction with the advocacy efforts of many local nonprofits, ultimately resulted in an increase in the maximum amount of Phase II awards to $100,000 per homeowner.
HOPE's data, gleaned from its experience working in low-income communities, also can influence public policy and resource allocation decisions. For example, HOPE's data contributed to the development of a state New Markets Tax Credit (NMTC) in Mississippi.1 HOPE has received multiple allocations of federal NMTCs and has used them to fund small business loan-fund pools and real estate projects. HOPE provided background on how NMTCs work and how they could be structured at the state level to the Ways and Means Committee of the Mississippi House of Representatives. HOPE was able to document the demand for credit and investment capital in the state's low-income markets, the number and quality of jobs that its borrower businesses were generating, and the companies' strong performance on their loans. Combined with specific examples of small business development in underserved communities, the data successfully made the case for a state-based NMTC program.
One of the HOPE organizations, the Mississippi Economic Policy Center (MEPC), focuses particularly on the advancement of public policies that support working families. Through public outreach and education efforts, it works to ensure that policy development and implementation takes into account the needs of low- and moderate-income Mississippians. MEPC was founded following Hurricane Katrina to track that recovery funds were equitably distributed and has since expanded its scope of work to address issues affecting state fiscal policy, workforce supports, and asset development.
One of MEPC's more noteworthy projects is the Self-Sufficiency Standard of Mississippi. Developed by Diana Pearce, then at Wider Opportunities for Women and now at the University of Washington in Seattle, the standard calculates how much it costs a family to live without any public or private assistance. The standard uses publicly available data to measure basic expenses such as housing, child care, food, transportation, and health care, and it accounts for variations in costs by county, family size, and family type. To make the standard more accessible, MEPC created a web-based interactive Self-Sufficiency Calculator.
While many variations of the calculator exist in states around the country, MEPC has made its tool particularly useful by linking county-level expense data to county-level wage and occupation information, which is regularly updated by the Mississippi Department of Employment Security. The integration of the data enables users to determine the wage that allows for basic self-sufficiency in their county and for their family type, and then research whether particular occupations pay wages above self-sufficiency levels. The final output also informs the user of the education levels needed to compete for jobs paying self-sufficiency wages in their county.
MEPC's interactive calculator has many practical applications. For example, it enables practitioners and policymakers to assess whether businesses in different parts of the state are creating jobs that allow workers to support themselves and their families. The Self-Sufficiency Calculator also helps individuals assess their career and lifestyle decisions.
The standard has been widely adopted. MEPC partnered with the Mississippi Council on Economic Education to develop a curriculum for its "Master Teachers in Economics" course. The teachers completing the course are taught how to integrate the information into their instruction. MEPC has been successful in integrating the material into other training programs as well. For example, the Mississippi Counseling Association has approved a training session for counselors on how to use the calculator with their students; the counselors earn continuing education credits for completing the session. In addition, MEPC has trained 500 high school freshmen in Lincoln County to use the standard to understand the importance of staying in school (there are no jobs in Mississippi for high school dropouts that pay self-sufficiency wages) and setting career goals that will lead to self-sufficiency in adulthood.
Lessons for Practitioners in the Field
Much of HOPE's data collection approach can be adapted and implemented by other organizations engaged in promoting neighborhood stabilization and revitalization. Surveys, benchmarking to public data, and contextualizing results all exhibit potential for replication. To that end, we offer three pieces of advice for those looking to develop and sustaining good measurement systems.
- Evaluation provides a return on investment. Evaluation is expensive. Surveys, analysis, and interviews require economic and human resources and can take people away from more direct programmatic activities. More complex evaluation can require costly analytical software. At the same time, sound evaluation systems and the results they generate demonstrate organizational and community capacity to potential philanthropic and institutional investors. Quality information thus strengthens the case for investment.
- Collecting and understanding data requires the involvement of both researchers and practitioners. Particularly in cases where organizations' resources are limited, program staff members play a key role not only in collecting data, but also in providing a context in which to assess it. The relationships that loan officers and other frontline staff have with business owners and homeowners can be critical in getting their participation in surveys and case studies, as well as in obtaining more candid responses.
- While much program-specific data may need to be collected, it is important to benchmark that data to public sources when possible. As already mentioned, the Self-Sufficiency Standard offered a benchmark against which to measure the job quality outcomes of HOPE's work in communities. It also served as a way of contextualizing HOPE's work on a broader public relations and public policy level. The Self-Sufficiency Calculator has potential for being replicated. There are now multiple measures of self-sufficiency available across the country, and state labor or employment security departments all have access to labor market information. Nonprofits in Maine, Rhode Island, and Texas have contacted MEPC staff members to assess the opportunity of creating similar tools for their communities.
About the Authors
Sean Zielenbach is the president of SZ Consulting, LLC, a community development consulting firm that focuses on issues of strategic planning, impact measurement, program and product development, and program evaluation. He has conducted numerous studies on issues pertaining to affordable and mixed-income housing and neighborhood revitalization. He has a doctorate in political science from Northwestern University and has published a book and numerous articles on affordable housing and community development.
Ed Sivak is the senior vice president of policy and evaluation with the Hope Enterprise Corporation and the founding director of the Mississippi Economic Policy Center. Sivak manages the strategic planning, research, training, and advocacy agenda for MEPC and HOPE's evaluation activities. He holds a master of public policy from the Georgetown Public Policy Institute and is a former member of the Federal Reserve Board's Consumer Advisory Council, where he chaired the Community Affairs and Housing Committee.
1. The federal NMTC program, managed by the U.S. Treasury Department, is designed to catalyze private investment in low-income areas. Investors receive federal income tax credits for equity investments they make in competitively selected community development entities (CDE), which then use the capital to make loans to or investments in eligible businesses located in census tracts with low median incomes and/or high poverty rates. Return to text