Description of the Survey
The Survey of Household Economics and Decisionmaking was fielded from November 13 through November 30, 2020. This was the eighth year of the survey, conducted annually in the fourth quarter of each year since 2013.57 Staff of the Federal Reserve Board wrote the survey questions in consultation with other Federal Reserve System staff, outside academics, and professional survey experts.
Ipsos, a private consumer research firm, administered the survey using its KnowledgePanel, a nationally representative probability-based online panel. Since 2009, Ipsos has selected respondents for KnowledgePanel based on address-based sampling (ABS). SHED respondents were then selected from this panel.
Survey Participation
Participation in the 2020 SHED depended on several separate decisions made by respondents. First, they agreed to participate in Ipsos' KnowledgePanel. According to Ipsos, 10.5 percent of individuals contacted to join KnowledgePanel agreed to join (study-specific recruitment rate). Next, they completed an initial demographic profile survey. Among those who agreed to join the panel, 61.8 percent completed the initial profile survey and became a panel member (study-specific profile rate). Finally, selected panel members agreed to complete the 2020 SHED.
Of the 18,077 panel members contacted to take the 2020 SHED, 11,713 participated and completed the survey, yielding a final-stage completion rate of 64.8 percent.58 Taking all the stages of recruitment together, the cumulative response rate was 4.2 percent. After removing a small number of respondents due to high refusal rates or completing the survey too quickly, the final sample used in the report included 11,648 respondents.59
Targeted Outreach and Incentives
To increase survey participation and completion among hard-to-reach demographic groups, Board staff and Ipsos utilized a targeted communication plan with monetary incentives. The target groups—young adults ages 18 to 29, adults with less than a high school degree, adults with household income under $40,000 who are under age 60, and those who are a race or ethnicity other than White, non-Hispanic—received additional email reminders and text messages during the field period, as well as additional monetary incentives.
All survey respondents not in a target group received a $5 incentive payment after survey completion. Respondents in the target groups received a $15 incentive. These targeted individuals also received follow-up emails during the field period to encourage completion. Additionally, the incentives offered to some targeted individuals increased to $25 during the field period to increase the incentive for completion.60
The final-stage completion rate in 2020 was slightly higher than 2019, due to an increase in the response rate among the non-targeted group. However, the final-stage completion for the target group was unchanged from 2019.
Survey Questionnaire
The 2020 survey took respondents 20 minutes (median time) to complete.
One priority for the 2020 survey was to understand how individuals, families, and communities—particularly those with low- to moderate-income—have been faring since the onset of the COVID-19 pandemic. The questions were intended to complement and augment the base of knowledge from other data sources, including the Board's Survey of Consumer Finances. In addition, some questions from other surveys were included to allow direct comparisons across datasets.61 The full survey questionnaire can be found in appendix A of the appendixes to this report.
Survey Mode
While the sample was drawn using probability-based sampling methods, the SHED was administered to respondents entirely online. Online interviews are less costly than telephone or in-person interviews and can be an effective way to interview a representative population.62 Ipsos' online panel offers some additional benefits. Their panel allows the same respondents to be re-interviewed in subsequent surveys with relative ease, as they can be easily contacted for several years.
Furthermore, internet panel surveys have numerous existing data points on respondents from previously administered surveys, including detailed demographic and economic information. This allows for the inclusion of additional information on respondents without increasing respondent burden. The respondent burdens are further reduced by automatically skipping irrelevant questions based on responses to previous answers.
The "digital divide" and other differences in internet usage could bias participation in online surveys, so recruited panel members who did not have a computer or internet access were provided with a laptop and access to the internet to complete the surveys. Even so, individuals who complete an online survey may have greater comfort or familiarity with the internet and technology than the overall adult population, which has the potential to introduce bias in the characteristics of who responds.
Sampling and Weighting
The SHED sample was designed to be representative of adults age 18 and older living in the United States.
The Ipsos methodology for selecting a general population sample from KnowledgePanel ensured that the resulting sample behaved as an equal probability of selection method (EPSEM) sample. This methodology started by weighting the entire KnowledgePanel to the benchmarks in the latest March supplement of the Current Population Survey along several geo-demographic dimensions. This way, the weighted distribution of the KnowledgePanel matched that of U.S. adults. The geo-demographic dimensions used for weighting the entire KnowledgePanel included gender, age, race, ethnicity, education, census region, household income, homeownership status, and metropolitan area status.
Using the above weights as the measure of size (MOS) for each panel member, in the next step a probability proportional to size (PPS) procedure was used to select study specific samples. This methodology was designed to produce a sample with weights close to one, thereby reducing the reliance on post-stratification weights for obtaining a representative sample.
After the survey collection was complete, statisticians at Ipsos adjusted weights in a post-stratification process that corrected for any survey non-response as well as any non-coverage or under- and over-sampling in the study design. The following variables were used for the adjustment of weights for this study: age, gender, race, ethnicity, census region, residence in a metropolitan area, education, and household income. Demographic and geographic distributions for the noninstitutionalized, civilian population age 18 and older from the March Current Population Survey were the benchmarks in this adjustment. Household income benchmarks were obtained from the 2019 American Community Survey (ACS).63
One feature of the SHED is that a subset of respondents also participated in prior waves of the survey. In 2020, about one-third of respondents had participated in the fall 2019 survey. Results using this repeated sample are shown in box 1 and utilize separate weights from the full sample. These weights use a similar procedure as described above to ensure estimates based on the repeated sample are representative of the U.S. population.
Although weights allow the sample population to match the U.S. population (excluding those in the military or in institutions, such as prisons or nursing homes) based on observable characteristics, similar to all survey methods, it remains possible that non-coverage, non-response, or occasional disparities among recruited panel members result in differences between the sample population and the U.S. population. For example, address-based sampling likely misses homeless populations, and non-English speakers may not participate in surveys conducted in English.64
Despite an effort to select the 2020 SHED sample such that the unweighted distribution of the sample more closely mirrored that of the U.S. adult population, the results indicate that weights remain necessary to accurately reflect the composition of the U.S. population. Consequently, all results presented in this report utilize the post-stratification weights produced by Ipsos for use with the survey.
References
57. Data and reports of survey findings from all past years are available at https://www.federalreserve.gov/consumerscommunities/shed.htm. Return to text
58. Three hundred ninety-four respondents were not included in the analysis because they started, but did not complete, the survey (known as break-offs). The study break-off rate for the SHED was 3.3 percent. Return to text
59. Of the 11,713 respondents who completed the survey, 65 were excluded from the analysis in this report due to either leaving responses to a large number of questions missing, completing the survey too quickly, or both. Return to text
60. All targeted adults received an email encouraging completion between three and six days into the field period. Three days before closing the survey, targeted adults received an email reminder mentioning a $25 incentive for completing the survey, and non-targeted adults received an email reminder mentioning a $5 incentive. Of the 5,804 respondents in a targeted group, 134 received the higher $25 incentive payment and the rest received the $15 incentive payment. Return to text
61. For a comparison of results to select overlapping questions from the SHED and Census Bureau surveys, see Jeff Larrimore, Maximilian Schmeiser, and Sebastian Devlin-Foltz, "Should You Trust Things You Hear Online? Comparing SHED and Census Bureau Survey Results," Finance and Economics Discussion Series Notes (Washington: Board of Governors of the Federal Reserve System, October 15, 2015). Return to text
62. David S. Yeager et al., "Comparing the Accuracy of RDD Telephone Surveys and Internet Surveys Conducted with Probability and Non-Probability Samples," Public Opinion Quarterly 75, no. 4 (2011): 709–47. Return to text
63. The ACS was used for the income benchmark because survey non-response in the 2020 March Supplement to the Current Population Survey (CPS ASEC) increased due to the COVID-19 pandemic, particularly for households with lower income. See Jonathan Rothbaum and Adam Bee, "Coronavirus Infects Surveys, Too: Survey Nonresponse Bias and the Coronavirus Pandemic," SEHSD Working Paper Number 2020-10 (Suitland, MD: U.S. Census Bureau, March 30, 2021), https://www.census.gov/content/dam/Census/library/working-papers/2020/demo/sehsd-wp2020-10.pdf. Return to text
64. For example, while the survey was weighted to match the race and ethnicity of the entire U.S. adult population, there is evidence that the Hispanic population in the survey were somewhat more likely to speak English at home than the overall Hispanic population in the United States. According to the SHED, 64 percent of Hispanic adults speak Spanish at home versus 74 percent according to the 2019 American Community Survey. See table B16006 at https://data.census.gov. See the Report on the Economic Well-Being of U.S. Households in 2017 for a comparison of results to select questions administered in Spanish and English. Return to text