Description of the Survey
The Survey of Household Economics and Decisionmaking (SHED) was fielded in November and December of 2017.49 This is the fifth year of the survey, conducted annually in the fall of each year since 2013.50
On average, the survey takes respondents 24 minutes (median time) to complete. The questions in the survey were written by staff of the Federal Reserve Board in consultation with other Federal Reserve System staff, outside academics, and professional survey experts.51 In selecting questions, a priority is to provide new information on the financial experiences and challenges among low- and moderate-income populations. These questions are intended to complement and augment the base of knowledge from other data sources, including the Board's own Survey of Consumer Finances (SCF). In addition, some questions from other surveys are included to allow direct comparisons across datasets.52 Most new survey questions were reviewed by survey design experts at NORC to improve comprehension and minimize potential confusion among respondents. The full survey questionnaire can be found in appendix A of the supplemental appendixes to this report (see www.federalreserve.gov/consumerscommunities/shed_publications.htm).
GfK, a private consumer research firm, administers the survey using its KnowledgePanel, a nationally representative probability-based sample. GfK selects respondents into KnowledgePanel based on address-based sampling (ABS).53 SHED respondents are then selected from this panel based on the criteria described below.
Survey Sample
The SHED sample is designed to be representative of adults ages 18 and older living in the United States. It includes a subset of respondents from the 2016 SHED ("re-interviewed respondents"), adults who did not participate in the previous year ("fresh respondents"), and an oversample of individuals with a household income less than $40,000 per year ("lower-income oversample").
The respondents in the 2017 SHED had to agree to several separate decisions to participate. First, they agreed to participate in GfK's KnowledgePanel and complete an initial demographic profile survey. Second, they agreed to complete the 2017 SHED. Only 12 percent of individuals contacted to join KnowledgePanel agreed to join (recruitment rate), and 63.6 percent of these recruited participants completed the initial profile survey necessary to become a panel member (profile rate). Then, of the 22,355 panel members contacted to take the 2017 SHED, 12,246 participated, yielding a final-stage completion rate of 54.8 percent (table 38).54 Taken together, the cumulative response rate is 4.2 percent.
Table 38. Survey response statistics
| Sample type | Number sampled | Completed responses | Completion rate (percent) |
|---|---|---|---|
| 2016 re-interviews | 2,913 | 2,305 | 79.1 |
| Fresh cases | 14,617 | 7,552 | 51.7 |
| Lower-income oversample | 4,825 | 2,389 | 49.5 |
| Overall | 22,355 | 12,246 | 54.8 |
GfK uses email reminders and small monetary incentives to encourage participation in the SHED. GfK sent two reminders to non-responders on the third and eleventh days in the field. GfK also maintains a modest incentive program with raffles and lotteries to encourage KnowledgePanel members to participate in surveys. Respondents receive a $5 incentive for completing the SHED, in addition to the standard incentives offered by GfK.
Although the sample is designed to be nationally representative, some hard-to-reach populations will likely be excluded. Homeless populations are likely missed by address-based sampling, and non-English speakers may not participate in the survey conducted in English.55 To better understand the effect of the language restriction, a portion of this year's survey was translated to Spanish and asked to a small sample of Spanish speakers. Box 4 discusses the results across the English and Spanish samples.
Box 4. Spanish-Language Sample
In the main SHED, the interview questions are all asked in English. People who are less fluent in English may, therefore, be less represented in the survey. To understand financial well-being among non-English speakers, in 2017 a subset of SHED questions were asked in Spanish to 260 additional respondents from GfK's Spanish-language panel ("Spanish-language sample").1
Table A indicates some differences in the demographic characteristics of the two groups of Hispanic respondents. Hispanics interviewed in Spanish, on average, have lower incomes, less education, and are older than those interviewed in English. As discussed in the main report, ethnicity, income, education, and age are all used in weighting the survey data to be representative of the U.S. population. Therefore, by design, the combined sample of Hispanic respondents and the main SHED sample provide very similar estimates of the share of Hispanic adults with each of these demographic characteristics.
Table A. Demographic characteristics of Hispanic respondents to SHED (by survey language)
Percent
| Characteristic | Experimental survey of Hispanics (English and Spanish interviews) | Main survey sample of Hispanics (English interviews only) | ||
|---|---|---|---|---|
| English-interview Hispanics | Spanish-interview Hispanics | Combined Hispanics | ||
| Family income | ||||
| Less than $40,000 | 48 | 56 | 51 | 51 |
| $40,000-$100,000 | 33 | 36 | 34 | 33 |
| Over $100,000 | 18 | 7 | 14 | 16 |
| Education | ||||
| High school degree or less | 45 | 69 | 53 | 50 |
| Some college or associate degree | 36 | 19 | 30 | 33 |
| Bachelor's degree or more | 19 | 12 | 17 | 17 |
| Age | ||||
| 18-29 | 33 | 11 | 25 | 26 |
| 30+ | 67 | 89 | 75 | 74 |
Note: The English-language Hispanics weighted to match the Hispanic population represents the sample and weights used throughout this report.
The Hispanic respondents in the Spanish-language sample and the SHED sample also differ in their responses to survey questions (table B). Hispanics in the Spanish-language sample report lower levels of economic well-being and more financial distress than Hispanics who took the main survey in English. For example, 61 percent of Hispanics in the Spanish-language sample say they are at least doing okay financially, compared to 69 percent of those interviewed in English. Similarly, 47 percent of Spanish-language Hispanics would be able to pay all of their current month's bills if faced with an emergency $400 expense, versus 55 percent of Hispanics interviewed in English. However, despite these differences, after weighting based on observable characteristics, the Hispanic sample who took the survey in English appears to reflect the Hispanic population as a whole across these measures.
Some differences between the English-language and combined samples remain when using survey weights. The share engaging in online gig work is somewhat lower for the combined sample of Hispanic respondents than is seen among just the SHED sample of Hispanic respondents who took the survey in English, whereas the share having problems with landlords and the share lacking a bank account are somewhat higher. As a result, readers of this report should keep in mind the potential for additional differences between the largely English-speaking population completing this survey and those with other language preferences that are less likely to be represented.
Table B. Selected survey measures among Hispanic respondents to SHED (by survey language)
Percent
| Characteristic | Experimental survey of Hispanics (English and Spanish interviews) | Main survey sample of Hispanics (English interviews only) | ||
|---|---|---|---|---|
| English-interview Hispanics | Spanish-interview Hispanics | Combined Hispanics | ||
| At least doing okay financially | 69 | 61 | 66 | 66 |
| Dealing with emergencies | ||||
| Pay $400 emergency using cash or equivalent | 47 | 45 | 46 | 45 |
| Could pay current month's bills in-full after $400 expense | 55 | 47 | 52 | 52 |
| Gig economy employment | ||||
| Online gig activities | 18 | 12 | 16 | 19 |
| Offline gig activities | 25 | 27 | 26 | 25 |
| Any gig work | 36 | 35 | 36 | 36 |
| Rental experience | ||||
| Any challenge getting landlord to fix problems in rental unit (among renters) |
24 | 33 | 28 | 25 |
| Bank status | ||||
| Unbanked | 11 | 23 | 15 | 11 |
| Underbanked | 25 | 26 | 25 | 26 |
Note: The English-language Hispanics weighted to match the Hispanic population represents the sample and weights used throughout this report.
1. A separate set of survey weights is used to combine this Spanish-language sample of Hispanics with the English-language SHED sample of Hispanics in the main survey report. With the weights, the combined responses are representative of the
U.S. population. Return to text
Survey Mode
While the sample is drawn using probability-based sample methods, the SHED is administered to respondents entirely online. Probability-based online interviews are less costly than telephone or in-person interviewing, and can still be an effective way to interview a representative population.56 GfK's
online panel offers some additional benefits. Their panel also allows the same respondents to be re-interviewed in subsequent surveys with relative ease, as they can be easily contacted for several years. Furthermore, internet panel surveys have numerous existing data points on respondents from previously administered surveys, including detailed demographic and economic information. This allows for the inclusion of additional information on respondents without increasing respondent burden. The respondent burdens are further reduced by automatically skipping irrelevant questions based on responses to previous answers.
The "digital divide" could bias participation in online surveys, so recruited panel members who do not have a computer or internet access are provided with a laptop and access to the internet to complete the surveys. Consequently, the raw distribution of KnowledgePanel mirrors that of U.S. adults fairly closely. Occasional disparities may occur in certain subgroups due to differential attrition rates among recruited panel members. Nonetheless, individuals who complete an online survey may have greater comfort or familiarity with the internet and technology than the overall adult population. For the 2017 SHED sample, 96 percent report that they or someone else in their household uses the internet at home. This is higher than the estimated three-quarters of adults reporting use of the internet at home in the July 2015 Computer and Internet Use Supplement to the Current Population Survey. This difference exists among both urban and rural respondents to the surveys. SHED respondents are also more likely than Current Population Survey respondents to use the internet at other locations, such as at work, suggesting that differences in internet usage across the surveys are due to different interests or comfort levels rather than availability.
Weighting
The selection methodology for the general population sample from KnowledgePanel ensures that the resulting samples behave as an equal probability of selection method (EPSEM) samples. This methodology starts by weighting the entire KnowledgePanel to the benchmarks in the latest March supplement of the Current Population Survey along several dimensions. This way, the weighted distribution of the KnowledgePanel matches that of U.S. adults. The geo-demographic dimensions used for weighting the entire KnowledgePanel include gender, age, race, ethnicity, education, census region, household income, home ownership status, and metropolitan area status.
Using the above weights as the measure of size (MOS) for each panel member, in the next step a probability proportional to size (PPS) procedure is used to select study specific samples. Since this survey includes a lower-income oversample, the departures caused by this oversample from an EPSEM design are corrected by adjusting the corresponding design weights accordingly with the Current Population Survey benchmarks serving as reference points.
Once the sample has been selected and fielded, and all the study data are collected and made final, a post-stratification process is used to adjust for any survey non-response as well as any non-coverage or under- and over-sampling resulting from the study specific sample design. The following variables were used for the adjustment of weights for this study: age, gender, race, ethnicity, census region, residence in a metropolitan area, education, and household income. Demographic and geographic distributions for the noninstitutionalized, civilian population age 18 and over from the March Current Population Survey are used as benchmarks in this adjustment.
Although weights allow the sample population to match the U.S. population based on observable characteristics, similar to all survey methods, it remains possible that non-coverage or non-response results in differences between the sample population and the U.S. population that are not corrected using weights.
References
49. The exact field dates were November 3 through November 18 and December 15 through December 24. The additional field dates in December were targeted at low-income and hard-to-reach populations in order to increase their participation. Return to text
50. Data and reports of survey findings from all past years are available at www.federalreserve.gov/consumerscommunities/shed.htm. Return to text
51. The survey instrument was also available for public comment through the Federal Reserve Board's website. Return to text
52. For a comparison of results to select overlapping questions from the SHED and Census Bureau surveys, see Jeff Larrimore, Maximilian Schmeiser, and Sebastian Devlin-Foltz, "Should You Trust Things You Hear Online? Comparing SHED and Census Bureau Survey Results," Finance and Economics Discussion Series Notes (Washington: Board of Governors of the Federal Reserve System, October 15, 2015). Return to text
53. Prior to 2009 respondents were also recruited using random-digit dialing. Return to text
54. Of the 12,246 respondents who completed the survey, 59 were excluded from the analysis in this report due to either leaving responses to a large number of questions missing, completing the survey suspiciously quickly, or both. Hence, 12,187 respondents are included in the analysis in this report. Return to text
55. For example, while the survey does weight to match the race and ethnicity of the entire U.S. adult population, there is evidence that the Hispanic population in the survey is somewhat more likely to speak English at home than the overall Hispanic population in the United States. Sixty-three percent of Hispanics who responded to the SHED speak Spanish at home, versus 73 percent of the overall Hispanic population who do so based on the American Community Survey. See table B16006 at factfinder.census.gov. Return to text
56. David S. Yeager, Jon A. Krosnick, LinChiat Chang, Harold S. Javitz, Matthew S. Levendusky, Alberto Simpser, and Rui Wang, "Comparing the Accuracy of RDD Telephone Surveys and Internet Surveys Conducted with Probability and Non-Probability Samples," Public Opinion Quarterly 75, no. 4 (2011): 709-47. Return to text