Skip to: [Printable Version (PDF)] [Bibliography] [Footnotes]

Finance and Economics Discussion Series: 2010-58 Screen Reader version ^♣

Does Credit Scoring Produce a Disparate Impact?

Robert B. Avery, Kenneth P. Brevoort, and Glenn B. Canner*

Board of Governors of the Federal Reserve System

October 12, 2010

Keywords: Credit scoring, discrimination, disparate impact

Abstract:

The widespread use of credit scoring in the underwriting and pricing of mortgage and consumer credit has raised concerns that the use of these scores may unfairly disadvantage minority populations. A specific concern has been that the independent variables that comprise these models may have a disparate impact on these demographic groups. By "disparate impact" we mean that a variable's predictive power might arise not from its ability to predict future performance within any demographic group, but rather from acting as a surrogate for group membership. Using a unique source of data that combines a nationally representative sample of credit bureau records with demographic information from the Social Security Administration and a demographic information company, we examine the extent to which credit history scores may have such a disparate impact. Our examination yields no evidence of disparate impact by race (or ethnicity) or gender. However, we do find evidence of limited disparate impact by age, in which the use of variables related to an individual's credit history appear to lower the credit scores of older individuals and increase them for the young.

I. Introduction

As the use of credit scoring has expanded over the past 20 years, concerns have been raised about whether its use may unfairly affect minorities and other populations.¹ Some of these concerns have focused on the specific predictive factors, or "credit characteristics," used in the models that generate credit scores and the question of whether the use of individual credit characteristics may have a disparate impact. These concerns about the fairness of credit scoring have lingered without being resolved.

Despite the public policy interest in addressing these questions, research on this topic has been largely nonexistent for two reasons.² First, credit scoring models are generally proprietary and, as a result, there is little or no information available about the specific credit characteristics that comprise these models. Second, there has been no data available that connects the demographic characteristics of individuals (including race or ethnicity, gender, or national origin) to their credit scores and credit history. The absence of data is partly a result of Federal laws that prohibit the collection of such information as part of non-mortgage credit applications.³

This paper takes advantage of a unique source of data to address the questions that have been raised about whether credit scoring has a disparate impact on minorities and other demographic groups. The data we rely on are based on a nationally representative sample of over 300,000 anonymous credit records that are observed at two points in time, June 2003 and December 2004. This dataset is similar to the data used in constructing and evaluating credit scoring models. These credit records are supplemented by demographic information on each individual from the Social Security Administration and a demographic information company.⁴ The resulting dataset is the first to combine this information for a nationally representative sample of individuals.

Using these data, we examine the individual predictive factors included in credit scoring models and assess whether including each of these factors in a credit scoring model results in a disparate impact by race or ethnicity, age, or gender. Credit characteristics are included in credit scoring models because they predict future credit performance; however, since these models cannot legally incorporate race or certain other demographic information, the predictiveness of an individual credit characteristic might arise because that characteristic is serving as a proxy for an excluded demographic characteristic. Using race as an example, a credit characteristic might serve as a proxy when (1) race is correlated with performance, and (2) the credit characteristic is correlated with race.⁵ A credit characteristic that derives its predictiveness solely by functioning as a proxy for demographics would not predict performance in a model that was estimated in a "demographically neutral environment," where demographics are controlled for or where the estimation sample is limited to a single demographic group. Credit characteristics that operate, in whole or in part, as proxies for a demographic characteristic have a "disparate impact" on individuals in that demographic group.

An analysis of the extent to which the credit characteristics that comprise a commercially available credit scoring model result in disparate impact would pose substantial data burdens. First, it would require detailed knowledge of the model being analyzed. This would include, among other things, a listing of the credit characteristics that comprise the model, the functional form in which each characteristic enters the model, and the weights assigned to each included characteristic. Second, such an analysis would require the actual sample that was used to develop the model, with the addition of the race, ethnicity, and gender of each individual in the sample, so that the model could be reestimated in demographically neutral environments. We are unaware of any commercially available credit scoring model for which this type of data is available.

Instead, we rely here on the model-building methodology developed as part of the Federal Reserve Board's Report to the Congress on Credit Scoring and its Effect on the Availability and Affordability of Credit (Board of Governors, 2007). This methodology emulates the process used by industry model builders to develop credit scoring models based exclusively on the information included in the credit records of individuals. The methodology is completely algorithmic which allows the process to be replicated using restricted or supplemented samples, such as those limited to a single demographic group. This allows us to perform two analyses on a baseline model that we develop using the entire sample: reestimation and redevelopment. Reestimation uses the same selection of credit characteristics that were selected for the baseline model to assess how the coefficients on each credit characteristic change when the model is reestimated in demographically neutral environments. Redevelopment replicates the entire model building process, including credit characteristic selection, in demographically neutral environments to evaluate how the selection of credit characteristics is affected. These two analyses allow us to examine the potential for disparate impact to emerge either from the coefficients estimated on each credit characteristic or from the choice of credit characteristics to include in the model.

The results of our analyses provide little or no evidence of disparate impact by race or ethnicity or by gender. Both reestimation and redevelopment of the baseline model in race-neutral and gender-neutral environments result in model coefficients, and consequently credit scores, that are very close to those produced by the baseline model. Additionally, we are unable to identify any credit characteristics whose omission from the model appears to be the result of correlations with these demographic groups.

However, we do find evidence of disparate impact by age. When the baseline model is reestimated or redeveloped in each of three age neutral environments, the scores of younger individuals decline and those of older individuals increase. We are able to trace these score changes to a single credit characteristic representing the average age of the credit accounts on file. The inclusion of this credit characteristic in our scoring model also appears to have an adverse effect on the credit scores of foreign-born individuals, and of recent immigrants in particular.

The remainder of the paper is organized as follows. The next section provides background information on credit record data in general and on the dataset used in this paper. Section III then conducts univariate analyses of the credit characteristics used in constructing our credit scoring model. Section IV discusses the model-building process that we follow and presents the baseline model. Sections V and VI then present the results of our model reestimation and redevelopment. Finally, Section VII presents our conclusions and suggests appropriate policy responses.

II. Background

Concerns about possible discrimination in the credit underwriting process are longstanding. Largely reflecting the availability of data, much of the research in this area has focused on the fairness of access to mortgage credit. The literature in this vein is quite large and varied in nature (Goering and Wienk, 1996; Ross and Yinger, 2002). Much of the research has attempted to replicate in some fashion the information available to underwriters and focus on whether similarly situated minorities have the same outcomes (whether in terms of denials or loan pricing) as nonminorities (Munnell, et al., 1996; Stengel and Glennon, 1999; Black, Boehm, and DeGennaro, 2003; Courchane, 2007). Another approach has been to evaluate the fairness of outcomes by examining loan performance (Berkovec, et al., 1994, 1996, 1998). Building upon the research into the economics of discrimination of Becker (1971), this approach is premised on the notion that biased lenders will require higher expected profits from loans to minority applicants.

A central issue in virtually all of the research in this area is the need to compare lending outcomes of loan applicants in similar financial and related circumstances. One of the most difficult aspects of such endeavours is accounting for possible differences in credit history, sometimes summarized by a credit score. However, little research has focused on the fairness of credit scores themselves.

II.1 Credit Record Data

The data that underlie most generic credit history scoring models come from the files of credit reporting agencies. Each of the three national credit reporting agencies (Equifax, Experian, and TransUnion) maintain records on as many as 1.5 billion credit accounts held by approximately 225 million individuals (Avery, Calem, and Canner, 2003). These credit records contain four types of information.

The first type is "tradeline" information which includes the details provided by creditors (and some other entities such as utility companies) on current and past loans, leases, and non-credit-related bills. This information includes the type of account (closed- or open-ended loan), the purpose of the account (for example, automobile loan, mortgage, student loan), the historical payment performance on the account, and details about other account derogatories (such as whether the account has been charged off or is in collection, is associated with a judgement, bankruptcy, foreclosure, or repossession).

The second type of information comes from monetary-related public records and includes records of bankruptcy filings, liens, judgements, and some foreclosures and lawsuits. The data distinguish (albeit imperfectly) between tax liens and other liens, though (unlike credit account data) the public record data do not provide a classification code for the type of creditor or plaintiff. Although public records include some details about the action, such as the date filed, the information available is much narrower in scope than that available on credit accounts.

Information on non-credit-related bills in collection that are reported by collection agencies constitutes the third type of information. These collection actions most commonly involve unpaid bills for medical or utility services. Collection agency records include only limited details about the action, including the date acquired by the collection agency, the original collection balance, and an indicator of whether the collection has been paid in full. There is no code indicating the type of original creditor or the date the account was opened or first became delinquent.

Finally, the fourth type of information reflects requests for information from an individual's credit record. Each time an individual or company requests information from an individual's credit record, an inquiry record is created. Only inquiries by creditors following an application ("hard" inquiries) are included in credit scoring models; inquiries for account management or solicitation purposes are not considered. The data on inquiries are maintained for two years and record only the type of firm making the inquiry, the date on which it was made, and the purpose of the inquiry.

In addition to these four types of information, credit records also include some personal identifying information, including each person's name, Social Security number, and a list of current and previous addresses. This information was not included in the data supplied to the Federal Reserve. Credit records do not include such personal information as race, ethnicity, or marital status. Age is sometimes included in credit records. The information reported in these files, generally, reflects monthly information received from creditors and others, with the records updated within one to seven days of receiving new information.

II.2 Data

The dataset compiled for this study is based on a nationally representative random sample of 301,536 individuals drawn as of June 30, 2003 from the credit bureau records of TransUnion. The records of these same individuals were also drawn as of December 30, 2004. Some individuals (15,743) in the initial 2003 sample no longer had active credit records as of December 30, 2004, leaving a total of 285,793 individuals with active credit files in both time periods.⁶

For each of these individuals, the Federal Reserve received the four types of information outlined in the previous section. In addition, TransUnion also provides 312 precalculated "credit characteristics," which contain summary information on each individual's credit record (such as the number of accounts on file or the average age of the accounts of file), for use in model construction.⁷ These are the credit characteristics that we evaluate in this study and comprise the group we select from in constructing our credit scoring models.

The sample of data also includes two different commercially available credit scores. The first is the TransRisk Account Management Score ("TransRisk score"), which is produced by TransUnion and predicts the likelihood that an individual will become seriously delinquent on at least one existing account during the next 24 months. The second is the VantageScore, produced by VantageScore Solutions, LLC, which predicts the likelihood that the individual will become seriously delinquent on a randomly selected new or existing account over the ensuing 24 months. Both scores were calculated for each of the two sample dates.

Credit scores could not be produced for all individuals in the sample. Individuals who have too few active credit accounts are generally considered "unscoreable," though the exact definition of what constitutes an unscoreable credit record varies across credit scoring models. About 17 percent of individuals in the sample (51,536) were not assigned a TransRisk score and 43,630 sample individuals did not receive a VantageScore. The sample used for most of the analysis here consists of the 232,467 individuals who had both scores.

These credit bureau records were supplemented by additional information on demographic characteristics from the Social Security Administration (SSA) and from a demographic information company that provides such information to creditors and other entities for use in marketing and solicitation activities.⁸ The SSA gathers demographic information when individuals apply for a Social Security card, including state or country of birth, race or ethnic description, gender, and date of birth.⁹ Only the race or ethnic description is provided on a voluntary basis. The data from the demographic information company included, to the extent available, details on each individual's race, education, sex, marital status, language preference, occupation, income range and date of birth. To resolve inconsistencies across different data sources for race, ethnicity, sex, and age, the decision was made to rely on the information provided in the official government records maintained by the SSA, unless we had strong reason to believe that the information was incorrect, in which case we deemed it "missing."¹⁰

Overall, almost 80 percent of the 301,536 individuals in the sample could be matched to SSA records. This includes 90 percent of individuals with both credit scores as of June 30, 2003, the sample most relevant for this analysis. Age and gender were available for virtually all of the individuals matched to the SSA records. Race or ethnicity was available for almost 97 percent of the individuals matched to the SSA records.

III. Credit Characteristics

For a credit characteristic to be included in a credit scoring model and to operate as a proxy for race or other demographic characteristics, it must be correlated with both performance and some demographic characteristic that is itself correlated with performance. In this section, we explore, in a univariate setting, the potential of each of the 312 credit characteristics to operate as a proxy, by examining the correlation of each credit characteristic with both performance and demographics.

Correlation coefficients for many credit characteristics could not be calculated because the characteristic took on "non-applicable" values. Often, these non-applicable values provide a significant portion of the characteristic's predictive power. For example, credit characteristic AT36, "total number of months since most recent account delinquency," takes on non-applicable values for individuals who have never been delinquent and this identification of the population that has never been delinquent has substantial predictive power for future delinquency. Instead, our approach is to estimate regressions of the form

$\begin{displaymath}Y_{i} = \alpha + I_{i}^{C}\beta + X_{i}^{C}\gamma + \epsilon_{i}\end{displaymath}$

The dependent variable, , reflects either performance or demographics. For performance, is an indicator variable that equals 1 if the individual had bad performance and zero otherwise. For demographics, the dependent variable can be continuous (in the case of age) or an indicator variable reflecting membership in a particular demographic group (for example, gender). Two right-hand-side variables are used to reflect the values of the credit characteristic. The first, $I_{i}^{C}$

, is a variable that indicates whether the value for credit characteristic is "not applicable." This variable is omitted from estimations that involve credit characteristics whose values are always calculable. The second, $X_{i}^{C}$ , is a continuous variable that equals the value of characteristic , or zero if the value is not applicable. The square root of the R-squared statistic from these regressions is used as the measure of correlation between each characteristic and performance or demographics.

In addition to the continuous dependent variable used for age, indicator variables are used to reflect demographics including race or ethnicity, gender, marital status, and whether the person was foreign born or a recent immigrant. Our definition of a recent immigrant is a person who is 30 or more years old and who applied for a Social Security card in the 10 years before the 2003 sample was drawn (which is a crude measure of when the person emigrated to the United States). We focus on this population because these individuals may have credit records that make them appear younger than their age and, consequently, they may be affected by credit characteristics that proxy for age. In estimations involving these demographic indicator variables, the regressions were run using only observations of individuals from that demographic group and from an appropriate base or comparison group. For example, the regression for black individuals was estimated using observations representing black individuals or non-Hispanic whites, which is the base group used for race.

The correlation measures for different demographic categories are shown in figure 1. Each panel shows, for each of the 312 credit characteristics, the correlation with performance on the y-axis and demographics on the x-axis. Points that are located farthest from both axes are those that are highly correlated with both performance and demographics and have the greatest potential to serve as proxies.

For most demographic characteristics, most notably those related to race or ethnicity, each credit characteristic's correlation with the demographic characteristic is lower than the correlation with performance. While several credit characteristics are highly correlated with performance, few have correlations with demographics that exceed 0.05 and even fewer have correlations approaching 0.1. The exceptions, however, are notable. Some credit characteristics reflecting past payment history are correlated with the black indicator variables, the only racial or ethnic characteristic that is correlated with credit characteristics at levels approaching 0.1. A second notable exception is the credit characteristics that are more highly correlated with being female (as observed in the estimations involving all females, single females, and married females) than with performance. While these credit characteristics are drawn from different groups, in each case they are characteristics that are associated with retail store tradelines.

The most significant exception is age. Not only are several credit characteristics more correlated with age than with credit performance, but in some cases the correlations with age are as high as 0.4, which is higher than any observed correlations with performance. All of the credit characteristics that have correlations with age exceeding 0.2 reflect the length of credit history. There are also, however, characteristics from the groups "new credit" and "amounts owed" that are correlated with age at levels in excess of 0.1.

These univariate results suggest that there are some credit characteristics that have correlations with both performance and demographics that are non-negligible and, consequently, have the potential to serve as proxies for demographic characteristics. These results also identify some demographic groups (blacks, females, and unspecified age groups) that are the most likely to be subject to disparate impact from the inclusion of credit characteristics that serve as proxies. By themselves, however, these results do not necessarily demonstrate the existence of such impact. Credit characteristics that are highly correlated with demographics may not be sufficiently correlated with performance to justify their inclusion in the model, or their correlation with demographics may be substantially reduced in a multivariate setting. To examine the extent to which credit characteristics that are likely to be included in credit scoring models result in disparate impact in the type of multivariate setting provided by these models, we develop a credit scoring model referred to here as the "baseline model."

IV. The Baseline Model

IV.1 Model Building Methodology

In this section, we present the credit scoring model used in this paper. The model-building methodology uses an algorithm that mirrors, to the extent possible, the development process used by industry model builders (Board of Governors, 2007). An algorithmic approach has the advantage that the rules governing the process are spelled out in precise detail and can be exactly replicated using different samples. However, this approach has the disadvantage of being devoid of any aspects of credit scoring "art" that could not be reduced to simple algorithmic procedures. While the algorithm does not resemble the process used by any individual model builder exactly, we believe, based on conversations with industry modellers and a review of the available literature, that it is a fair representation of industry practice as a whole regarding the model construction.

The first step in the development process is to select the outcome to be predicted. Our model predicts an individual's worst performance on an account during the 18-month performance period between our two samples (June 2003 and December 2004). We evaluate performance on new and existing accounts, meaning those accounts that were either opened during the first six months of the performance period (July to December 2003) or were open at the beginning of the performance period.

An individual's worst performance is classified based on the performance on her accounts. If she was 90 or more days past due during the performance period on a new or existing account, then she exhibited "bad" performance. If she was never past due during the performance period (beyond an isolated 30-day delinquency) and had at least one account with on-time payments, then she exhibited "good" performance. Otherwise, her performance was "indeterminate" (generally, these were individuals whose worst performance 60 days past due). Following industry practice, those individuals with indeterminate performance are not used in the estimation sample, though they are used in the rest of the analysis.

The next step in the model-building process is to decide which credit characteristics will be considered for possible inclusion in the model. The credit characteristics used in model development fall into five broad areas: payment history, amounts owed, length of credit history, types of credit in use, and acquisition of new credit. All five of these areas are represented in the 312 credit characteristics that TransUnion supplies for model-building purposes. We use these credit characteristics as the pool from which we select characteristics for our model.

Although a generic credit history score can be estimated using a single equation, estimation samples are generally divided into distinct subpopulations or "scorecards." Since we are using a smaller sample size than is commonly used by industry model builders, we restrict ourselves to three scorecards. While not empirically derived, these scorecards were selected to represent the major population segmentations used in scorecard development. The first scorecard, the "thin" scorecard, contains those individuals with two or fewer tradelines. Individuals with more than two tradelines are placed on the "dirty" scorecard if they have had one or more 90-day delinquency, a public record, or a collection account of more than $50. Otherwise, individuals with more than two tradelines are placed on the "clean" scorecard.¹¹ The process of creating attributes, selecting credit characteristics, and estimating models is then conducted separately for each of the three scorecards.

Following industry practice, credit characteristics enter a model as a series of dummy variables, called "attributes." An attribute reflects a specific range of values, with the attribute assigned a value of 1 if the value of characteristic falls within the specified range and zero otherwise. The attributes partition the space of possible values, so that a single attribute is assigned a value of 1 and the others equal zero.

Attributes are created for each of the 312 credit characteristics, with a separate set of attributes created for each scorecard. The first step in attribute creation is to determine whether the credit characteristic can include non-applicable values, which arise when the value of a credit characteristic cannot be calculated. For example, the credit characteristic "total number of months since the most recent account delinquency" cannot be calculated for individuals who have never had a delinquency. For those characteristics where a non-applicable value is possible, an attribute is created to reflect non-applicable values. For credit characteristics where non-applicable values are not possible, such as the "total number of mortgage accounts" (which takes on a value of zero for individuals who have never had a mortgage) attributes for non-applicable data are irrelevant and are not included.

Once the attributes corresponding to non-applicable values are created (if necessary), the range of remaining values for each credit characteristic is partitioned into a series of one or more attributes. This process begins by creating a single attribute that covers all of the remaining values of the credit characteristic. Then, each possible subdivision of this attribute into two candidate attributes, each covering a compact set of sequential values, is evaluated.¹² The subdivision that results in the smallest sum of squared residuals is selected. If the difference in mean performance between the two candidate attributes is statistically significant at the 5 percent level then the two candidate attributes replace the single attribute.

The process then examines each of the attributes of a credit characteristic in basically the same manner. Each attribute is subdivided into the best two candidate attributes. At this stage, to be considered a set of candidate attributes, a subdivision has to result in two attributes that would maintain the monotonicity of mean performance levels across all of the attributes of a credit characteristic. Subdivisions that do not maintain this monotonicity are not considered candidate attributes. The two candidate attributes that best predict performance then replace the attribute under examination if the difference in mean performance between the two candidate attributes is statistically significant at the 5 percent level. This process is repeated until no additional statistically significant and monotonicity-preserving subdivisions are possible. The number of attributes created for each credit characteristic varies from one (for those characteristics with no non-applicable values and no statistically significant subdivisions) to 21.

The next step in the process is to select the credit characteristics that appear on each of the three scorecards. When a credit characteristic is included in a model, all of its attributes are included, with the exception of the attribute representing the lowest values for that characteristic, which is the omitted category. Following standard model-building practice, we estimate a logit model subject to the constraint that the coefficients across the attributes of each credit characteristic must be monotonic (with the exception of the coefficient on the attribute for non-applicable values).¹³

Credit characteristics are added to the model in a forward-stepwise manner, in which the credit characteristic that produces the largest increase in the divergence statistic is chosen. Characteristics are added until the marginal increase in the divergence statistic that results when the characteristic is added to the model falls below 0.75 percent. This threshold was chosen to ensure that each scorecard contained approximately 10 to 15 credit characteristics, the number typically found on industry scorecards.

Once the stepwise process is complete, each characteristic is again evaluated to ensure that its marginal contribution to the divergence statistic continues to exceed the threshold. This is done by removing each of the credit characteristics that comprise a scorecard, calculating the divergence statistic based on a model that includes only the remaining characteristics, and calculating the increase in the divergence statistic that results when the characteristic is included. Any credit characteristic whose marginal contribution to the divergence statistic is below the threshold is removed from the model. If a characteristic is removed, then the algorithm again evaluates all of the remaining characteristics for inclusion.

The process of removing and adding credit characteristics continues until (a) each of the credit characteristics included in the model contributes to the divergence statistic a percentage increase on the margin that exceeds the threshold; and (b) none of the excluded credit characteristics would improve the divergence statistic by a percentage that exceeds the threshold if included in the model. Once these two conditions are met, the credit characteristics that comprise the model for the scorecard being constructed are set. This process is repeated for all three scorecards.

The final step in the model-building process involves normalizing the score to a rank-order scale. Fitted values are calculated for each individual in our full sample (including those individuals who had indeterminate performance and were not included in the estimation sample). Based on these fitted values, individuals are ranked and receive a score between 0 and 100 that reflects the percentile of the distribution into which the individual falls. As a result, five percent of individuals have a score of 5 or less and 50 percent have a score of 50 or less. Normalizing all of the credit scores in the sample to the same rank-order scale allows for a straightforward comparison of the different models being examined.

IV.2 Specification and Comparison with Commercial Scores

A full description of the baseline model is provided in panels (A) through (C) of table 1. This table provides a complete list of attributes and weights for each credit characteristic on the three scorecards. A baseline score is calculated for each individual by calculating the fitted value for each individual (using the equation $1/(1 + e^{-x})$ , where is the sum of the weights for each credit characteristic) and then normalizing this fitted value using the function depicted in figure 2. This normalized score is what we refer to as the baseline score.

A primary concern about evaluating the baseline model is that it may not closely resemble models used by industry. To evaluate how closely the baseline scores compare to scores from commercially available scores, we compare score distributions for different demographic groups generated by the baseline model with the distributions for the TransRisk score and the VantageScore. Both commercial scores are normalized to the same rank-order scale described earlier (so the distributions of each score for the entire population is approximately identical) to facilitate these comparisons.

As seen in table 2, distributions of each of the three scores are very similar for each demographic group. Mean and median baseline credit scores are generally within 2 points of the commercial scores. Unfortunately, comparisons of the distributions of two different credit scores for the same population of individuals are not amenable to standard statistical tests so we are unable to report statistical significance levels.¹⁴ Nevertheless, the similarities between the baseline scores and the TransRisk and VantageScores suggest that the baseline model is capturing most of the difference observed in credit scores across demographic groups.

IV.3 Credit Characteristics and Score Differences Across Demographic Groups

Baseline scores, as well as the two commercially available scores for the sample population, indicate that there are substantial differences across demographic groups. In this section, we examine how the score differences are affected by the inclusion of specific credit characteristics in the baseline model. For each characteristic in the baseline model, we estimate a revised model that excludes that characteristic. We then compare scores from these revised models (which are normalized to a rank-order scale) with the baseline scores. When a credit characteristic appears on more than one scorecard, a separate revised model is calculated for each scorecard (so that the credit characteristic is removed from one scorecard but left on the others). This process helps to identify which characteristics have large impacts on score differences of different demographic groups as a result of their inclusion in the model.

Table 3 provides a definition of the four-character names of each of the credit characteristics that appear in the baseline model or are used elsewhere in this study and table 4 shows the mean and median score changes, by demographic group, that result from the omission of each credit characteristic from the baseline model. Score changes are for the individuals in each demographic group whose records place them on the scorecard from which the characteristic was dropped. Very few of the credit characteristics in the baseline model, on the margin, have a substantial effect (either positive or negative) on the credit scores of any demographic group. This is particularly true for score differences across race or ethnicity, gender, and marital status, where score differences generally change by 1 point or less and almost none are changed by more than two points.

A number of credit characteristics, when excluded from the baseline model, alter the relative credit scores of different age groups by more than two points. These include four characteristics on the thin scorecard (, AT34, AT28, and RE20) and characteristic on both the clean and dirty scorecards. The finding that some credit characteristics have substantial effects on scores by age, but not by race or gender, is consistent with our earlier univariate finding of higher correlations of credit characteristics with age than with other demographic characteristics. The credit characteristics whose exclusion altered score differences across age groups also appear to have had relatively large effects on the scores of the foreign born and, in particular, recent immigrants.

V. Reestimating the Baseline Model in Demographically Neutral Environments

As discussed earlier, the coefficients on attributes of a credit characteristic that is functioning as a proxy for membership in a particular demographic group will change when estimated in a demographically neutral environment. In the extreme case where a credit characteristic is operating solely as a proxy for membership in a demographic group, the coefficients on the attributes of that characteristic will be close to zero when the model is estimated in an environment that is neutral with respect that demographic group. In other cases, where the credit characteristic operates as a demographic proxy but also has predictive power within each demographic group, the coefficients estimated for attributes of that characteristic may either increase or decrease in a demographically neutral environment, depending upon the relationship among the credit characteristic, demographics, and performance. In this section, we reestimate the baseline model in the eight different demographically neutral environments listed in table 5 to examine whether the credit characteristics included in the baseline model are causing disparate impact for members of a variety of demographic groups.

When reestimating the baseline model in these environments, we use the same credit characteristics and attributes as in the baseline model and continue to impose monotonicity across attribute coefficients. Fitted values for each individual in the sample are then calculated as though everyone was part of the same demographic group (i.e., everyone is the same age, gender, or race or ethnicity) and normalized to a rank-order scale using the full sample population of 232,467 individuals.¹⁵

We use the reestimated models to explore for the existence of disparate impact using a two-part process. First, we compare the baseline credit scores to the scores generated by the models reestimated in the demographically neutral environments. If a credit characteristic is operating as a proxy for membership in a demographic group, the credit scores of individuals who are benefited (harmed) by the proxy should fall (rise) in an environment that is neutral with regards to that demographic group. For example, if a credit characteristic in the baseline model is proxying for race in a manner that adversely affects blacks, we would expect the scores of blacks to increase when the model is reestimated in a race-neutral environment. In the second part of the process, for those demographic groups whose scores change significantly when the baseline model is reestimated in a demographically neutral environment, we trace any score differences to the credit characteristics that generate them. This is done by comparing the coefficients on each attribute in the baseline model with the coefficients from the reestimated models. This process allows us to identify any credit characteristic whose inclusion in the baseline model results in a disparate impact on any of the demographic groups examined.

The score changes are shown in table 6 for each demographically neutral environment for select demographic groups associated with that environment. Changes in the mean scores associated with reestimations in each race-neutral and gender-neutral environment were uniformly very small, in each case being under 0.2, the smallest increment allowed under the normalization. Changes in the median scores were 0.2 or zero for each demographic group listed, except for American Indians where the sample size is very small. This suggests that credit characteristics in the baseline model are unlikely to be operating as proxies for race, ethnicity, or gender.

In contrast, in each of the three age-neutral environments, reestimation results in lower mean scores for younger individuals and higher mean scores for older individuals than were produced by the baseline model. This pattern is consistent with what one would expect to observe if a credit characteristic was operating in whole or in part as a proxy for age. Additionally, the change in mean scores for foreign-born individuals, and for recent immigrants in particular, are uniformly lower when the baseline model is reestimated in age-neutral environments. This finding is also consistent with the presence of an age-proxy, as these individuals have been in the county for a shorter period of time than native-born individuals and are likely to have credit profiles (as reflected in the bureau data) that are similar to younger individuals in that they tend to have shorter credit histories.

To examine which credit characteristic is the cause of these differences, mean score changes resulting from the reestimation of the baseline model in age neutral environments were decomposed by scorecard. This decomposition indicates that the change in relative credit scores by age and immigration status can be traced to changes on the clean scorecard. While the attribute coefficients for most characteristics on this scorecard are of relatively similar magnitudes in the baseline and age-neutral models, ("average age of accounts on credit report") stands out as the source of the relative score differences by age group and immigration status. Table 7 provides the coefficients on each attribute of from the clean scorecard for the baseline model and for each of the demographically neutral models, along with the distribution of individuals on the clean scorecard across the different attributes. The differential in the coefficients on the lower and higher attributes of is greater in the age neutral models than the differential in coefficients in the baseline model. For example, the difference in the value of the coefficients on the modal attribute for the 30 and under population and the coefficient on the modal attribute for the 62 and older population is approximately 0.97 for each of the three age neutral models. This is substantially higher than the 0.72 difference between these coefficients in the baseline model. It is this widening difference in the value of the coefficients on the attributes of that results in the widening credit score difference across age groups when the models are reestimated in each of the three age neutral environments.

This result suggests that the inclusion of this credit characteristic on the clean scorecard has a disparate impact by age. Our results show that when the baseline model is reestimated in an age neutral environment the predictiveness of increases, so that score differences between individuals with high and low values of this credit characteristic widen. This implies that the baseline credit scores of older individuals are too low and the credit scores of younger individuals are too high as a result of this credit characteristic proxying for age.

The method in which disparate impact arises from this credit characteristic is counterintuitive. Given the positive correlations among age, , and performance, one would expect the relationship between and performance to become stronger as a result of this characteristic proxying for age. As a result, the coefficients on in the baseline model would be larger than those in the age-neutral models and the scores of the old (young) would be lower (higher) in the age-neutral models. Instead, we observe the opposite result: Credit scores of the old are higher in the age-neutral models and the relationship between and performance is dampened as a result of proxying for age. The reason for this counterintuitive result is that is more predictive of future performance between individuals of the same age than it is for individuals of different ages. As a result, in models estimated in environments that are not age-neutral, which would include most credit scoring models, we expect the relationship between length of credit history and performance to be weakened because of credit history variables proxying for age. This suggests that the use of this credit characteristic, , has a disparate impact by age that negatively affects older individuals and positively affects younger individuals.

VI. Redeveloping Models in Demographically Neutral Environments

Reestimating the baseline model in demographically neutral environments is useful in examining the potential of the credit characteristics selected for the baseline model to have a disparate impact on different populations. However, that approach holds constant the credit characteristics (and attributes) that comprised the model and does not evaluate the potential disparate impact that could emerge because of the selection of characteristics included in the model. In particular, it is possible that some of the credit characteristics that were not selected for the credit scoring model were omitted because the strength of the relationship between the characteristic and performance was dampened because the characteristic was proxying for demographics. If the model development process had been conducted in a demographically neutral environment, such characteristics would have been selected and the scores of different demographic groups may have been altered.

Because our method of creating attributes and selecting credit characteristics is algorithmic, we can re-run the model development process in the eight demographically neutral environments. We can then use these redeveloped models in a two-part analysis that is similar to that conducted for the reestimated models in the previous section. In the first part, we compare the credit characteristics that are selected for inclusion in each of the redeveloped models with the characteristics that were selected for the baseline model. Any credit characteristics that were omitted from the baseline model because the model was proxying for demographics, should appear in the models that are redeveloped in a demographically neutral environment. The second part of the analysis then examines how the model redevelopment affects the credit scores of different demographic groups.

Table 8 presents the credit characteristics that comprise each scorecard of the models that were redeveloped in the eight demographically neutral environments. The characteristics selected for each model are somewhat different than those selected for the baseline model. The extent to which the selection of baseline characteristics is similar to the characteristics in the redeveloped models appears to differ somewhat by scorecard, with more similarity on the thin and dirty scorecards than on the clean scorecard.

The differences in the characteristics that have been selected reveal few credit characteristics that appear to have been systematically excluded as a result of the characteristic proxying for demographics. Credit characteristics whose predictiveness is muted as a result of correlations with demographics would have enhanced predictive power in all of the environments that are neutral with respect to that demographic characteristic. As a result, these characteristics would be more likely to be included in each of the models that are redeveloped in those environments. There are very few credit characteristics where this appears to be the case.

The models that have been redeveloped in race-neutral environments fail to identify any credit characteristics that are being excluded as a result of correlations with race or ethnicity. There are two credit characteristics that are added to the models redeveloped in race neutral environments: AT34 ("Percentage of total remaining balance to total maximum credit for all open accounts reported in the past 12 months") on the clean scorecard and ("Number of accounts that have payments that are currently or previously 30 or more days past due within the past 18 months") on the dirty scorecard. However, these two credit characteristics are sufficiently similar to credit characteristics that are included in the baseline model, but excluded in the race-neutral models, to suggest that the difference results from random variation from using different samples or additional demographic control variables. In particular, AT34 appears a close replacement for RE34 ("Percentage of total remaining balance to total maximum credit for all open revolving accounts reported in the past 12 months") and and a close replacement for ("Number of accounts that have payments that are currently or previously 30 or more days past due in the past 12 months.").

A very similar result can be found in the selection of credit characteristics for models redeveloped in the age-neutral environments. Only one credit characteristic was excluded from a scorecard of the baseline model and subsequently appeared on that scorecard in each of the models redeveloped in age-neutral environments. That credit characteristic is RE34 on the thin scorecard, which, as with the race-neutral result, appears to be a close substitute for credit characteristic AT34. Otherwise, there appears to be little evidence of a credit characteristics being excluded from the baseline model as a result of correlations with age.

The models that were redeveloped in gender-neutral environments reveal one credit characteristic, G096 ("Total number of inquiries for credit"), that is not included in the dirty scorecard of the baseline model, but that appears on that scorecard in each of the redeveloped gender-neutral models. Unlike the credit characteristics in the redeveloped race-neutral models, this credit characteristic does not appear to be substituting for a very similar credit characteristic in the baseline model. Consequently, this credit characteristic may result in some disparate impact.

To evaluate how these different models affect the scores of individuals in different demographic groups, we evaluate how mean and median scores are changed, relatively to the baseline model, in each of the demographically neutral models. These score changes are provided in table 9. As that table shows, there is very little evidence of the type of consistent, substantial score changes in any of the race- or gender-neutral models that would be indicative of disparate impact. To the extent that these models were constructed using somewhat different credit characteristics, there is no evidence that these differences had any meaningful impact on the credit scores of any race, ethnicity, or gender group.

Again, however, there are consistent changes in the scores across age groups for models estimated in each of the age neutral environments. The score changes are similar to those found in the previous section when the baseline model was reestimated in age neutral environments. Since the credit characteristics that appear to have given rise to those differences (specifically remain in the models estimated in these environments, and since there appears to be little evidence of credit characteristics that were inappropriately excluded from the model as a result of their correlation with age, we surmise that these score differences reflect disparate impact arising from the credit characteristics identified in the previous section. Overall, there appears to be little evidence that the differences in credit characteristic selection had much, if any, disparate impact by age.

VII. Conclusions

This paper explores the potential for specific credit characteristics included in generic credit history scoring models to have disparate impacts on certain demographic groups, most notably minorities. A credit characteristic can have a disparate impact (either positive or negative) on members of a given demographic group if the predictiveness of that credit characteristic derives, in whole or in part, from its functioning as a proxy for membership in that demographic group.

Our results provide little or no evidence that the credit characteristics used in credit history scoring models operate as proxies for race or ethnicity. The distributions of credit scores for different racial or ethnic groups or across genders are essentially unaffected by the reestimation or redevelopment of the baseline credit scoring model in any of the race- or gender-neutral environments. This suggests that credit scores do not have a disparate impact across race, ethnicity, or gender. We do, however, find some evidence that credit characteristics associated with the length of an individual's credit history (specifically, credit characteristic , "Average age of accounts on credit report") may have a disparate impact by age. In particular, we find that the predictiveness of this credit characteristic increases when the credit scoring model is estimated in an age neutral environment. This suggests that the predictiveness of this credit characteristic is dampened as a result of its proxying for age and that, consequently, the credit scores of older (younger) individuals are lower (higher) than they should be.

This finding raises questions about what the appropriate public policy response should be. There are two primary courses of action to correct this disparate impact. The first is to require that credit score modellers estimate their credit scoring models in demographically neutral (specifically age neutral environments). While this would effectively eliminate the disparate impact associated with this and any other credit characteristics, the type of demographic data used in this study is not generally available to credit score model builders and consequently, while this might be our preferred remedy, this is not a feasible response. The second course of action is to prohibit the use of this credit characteristic in credit scoring models. The downside of this approach is that has substantial predictive power, particularly in demographically neutral environments, and showed up as a highly predictive variable in each of the redeveloped scoring models. Banning the use of this credit characteristic in credit scoring models, therefore, would pose a large cost in reduced model predictiveness.

Instead of these courses of action, we believe that the size of the disparate impact detected in this study is sufficiently small as to make inaction the preferred option. The disparate impact found lowered the scores of the elderly (who generally have very high credit scores) and raised the scores of the young (who generally have very low credit scores) only slightly. Consequently, the economic size of the harm caused by this disparate impact is unlikely to make either of the two potential remedies attractive.

Our results also indicate that credit characteristics associated with length of credit history may have an inappropriately adverse effect on foreign born individuals, in particular on recent immigrants. Because they were not born in the United States, these individuals have relatively short credit histories as reflected in their U.S. credit reports (presumably they may have had credit experience in their native countries, though this is not reflected in U.S. credit bureau files); such reports are consistent with those of younger individuals. Our result suggests that the credit scores of the foreign born population benefits to the extent that the coefficients in the baseline model are dampened as a result of disparate impact. Nevertheless, the fact that this population has shorter credit histories reflected in U.S. credit bureau records appears to result in lower scores for these individuals. This contributes to the tendency of this population to perform better on credit obligations, on average, than other native-born individuals with identical credit scores (Board of Governors, 2007). While this result is not related to the disparate impact we find by age, it does reflect that this specific characteristic is unfairly disadvantaging this population.

Unlike the disparate impact by age, there may be public policy options that reduce or eliminate this effect. For example, public policy might encourage or facilitate the gathering of information on the credit histories of recent immigrants from their native countries. This information can supplement the information provided in U.S. credit bureau records and may more accurately and completely reflect the credit histories of these individuals. Additionally, ongoing industry efforts to collect additional information on the use of non-traditional sources of credit (such as payday lending and pawn shops) or utility payments may broaden the information included in credit records and may serve to lengthen the period over which the foreign born have a credit record. Public policy efforts in these areas may reduce the disadvantage incurred by the foreign born, particularly recent immigrants, as a result of the use of credit characteristics related to length of credit history.

The conclusions in this paper are drawn from an analysis that has important limitations. Perhaps the most important is that the analysis is based upon a credit history scoring model that was developed specifically for this study and not upon a commercially available score. While the methodology attempts to emulate the process used by industry model builders, there is no standardized procedure in the industry so our methodology is approximate. Additionally, the sample size used in this study to estimate the model was substantially smaller than the sizes generally used to estimate commercial credit scoring models. As a result, our model was forced to utilize a smaller number of scorecards than would have been ideal and consequently may have missed possible disparate impact faced by small subsets of the population. Despite these limitations, we believe that the results of our analysis are generally applicable to most credit history scoring models that rely on credit bureau data.

References

Avery, Robert B., Raphael W. Bostic, Paul S. Calem and Glenn B. Canner, 1996, "Credit Risk, Credit Scoring, and the Performance of Home Mortgages," Federal Reserve Bulletin, vol. 82, July, pp. 621-48.

Avery, Robert B., Paul S. Calem, and Glenn B. Canner, 2003, "An Overview of Consumer Data and Credit Reporting," Federal Reserve Bulletin, 89, February, pp. 47-73.

Berkovec, James A., Glenn B. Canner, Stuart A. Gabriel, and Timothy H. Hannan, 1994, "Race, Redlining, and Residential Mortgage Loan Performance," Journal of Real Estate Finance and Economics, 9(3), pp. 263-94.

__________, 1996, "Mortgage Discrimination and FHA Loan Performance," Cityscape, 2(1), pp. 9-24.

__________, 1998, "Discrimination, Competition, and Loan Performance in FHA Mortgage Lending," Review of Economics and Statistics, 80(2), pp. 241-50.

Black, Harold A., Thomas P. Boehm, and Ramon P. DeGennaro, 2003, "Is There Discrimination in Mortgage Pricing? The Case of Overages." Journal of Banking and Finance, 27(6), pp. 1139-65.

Chandler, Gary, 1985, "Credit Scoring: A Feasibility Study," Credit Union Exec, 25, pp. 8-12.

Collins, M. Cary, Keith D. Harvey, and Peter J. Nigro, 2002, "The Influence of Bureau Scores, Customized Scores and Judgemental Review on the Bank Underwriting Decision Making Process," Journal of Real Estate Research, 24(2), pp. 129-52.

Courchane, Marsha J., 2007, "The Pricing of Home Mortgage Loans to Minority Borrowers: How Much of the APR Differential Can We Explain?" Journal of Real Estate Research, 29(4), pp. 399-439.

Elliehausen, Gregory E. and Thomas A. Durkin, 1989, "Theory and Evidence of the Impact of Equal Credit Opportunity: An Agnostic Review of the Literature," Journal of Financial Services Research, 2 (2), pp. 89-114.

Fortowsky, Elaine and Michael LaCour-Little, 2001, "Credit Scoring and Disparate Impact," Working paper.

Goering, John and Ron Wienk, eds., 1996, Mortgage Lending, Racial Discrimination, and Federal Policy (Washington, D.C.: Urban Institute Press).

Hand, David J. and Niall M. Adams, 2000, "Defining Attributes for Scorecard Construction in Credit Scoring," Journal of Applied Statistics, 27 (5), pp. 527-540.

Hunt, Robert M., 2005, "A Century of Consumer Credit Reporting in America," Working Paper, no. 05-13 Federal Reserve Bank of Philadelphia, June, pp. 1-54.

Lewis, Edward M., 1992, An Introduction to Credit Scoring, San Rafael, CA: Athena Press.

Martell, Javier, Paul Panichelli, Rich Strauch, and Sally Taylor-Shoff, 1991, "The Effectiveness of Scoring on Low-to-Moderate-Income and High-Minority Area Populations" (San Rafael, Calif: Fair Isaac).

Munnell, Alicia H., Lynn E. Browne, James McEnearney, and Geoffrey M. B. Tootell,, 1996, "Mortgage Lending in Boston: Interpreting HMDA Data." American Economic Review, 86(1), pp. 25-53.

Rosenberg, Eric and Alan Gleit, 1994, "Quantitative Methods in Credit Management: A Survey," Operations Research, 42 (4), pp. 589-613.

Ross, Stephen L. And John Yinger, 2002, The Color of Credit: Mortgage Discrimination, Research Methodology, and Fair-Lending Enforcement (New York: MIT Press).

Schreiner, Mark, 2002, "Scoring: The Next Breakthrough in Microcredit?" Consultative Group to Assist the Poorest.

Stengel, Mitchell, and Dennis Glennon, 1999, "Evaluating Statistical Models of Mortgage Lending Discrimination: A Bank-Specific Analysis." Real Estate Economics, 27(2), pp. 299-334.

Straka, John W., 2000, "A Shift in the Mortgage Landscape: The 1990s Move to Automated Credit Evaluations," Journal of Housing Research, 11(2), pp. 207-32.

Yago, Glen, Betsy Zeidman, and Bill Schmidt, 2002, "Creating Capital, Jobs and Wealth in Emerging Domestic Markets," The Ford Foundation.

**Figure 1:** Correlations with Credit Performance and Demographics 312 Credit Characteristics

Accessible Version

**Figure 1:** Correlations with Credit Performance and Demographics for the 312 Credit Characteristics (continued)

Accessible Version

**Figure 1:** Correlations with Credit Performance and Demographics for the 312 Credit Characteristics (continued)

Accessible Version

**Figure 2:** Mapping from Unnormalized to Normalized Score