skip to main navigation skip to secondary navigation skip to content
Board of Governors of the Federal Reserve System
skip secondary navigation

Other ResearchConsumer Research

Executive Summary

Forum on Consumer Research & Testing: Tools for Evidence-based Policymaking in Financial Services 



On November 9, 2010, the Consumer Community Affairs Division of the Board of Governors of the Federal Reserve hosted the Forum on Consumer Research Testing: Tools for Evidence-based Policymaking in Financial Services. With roughly fifty participants and thirteen research presentations, attendees represented a variety of disciplines including economics, psychology, marketing and decision sciences, sociology, and plain language, as well as different perspectives from experience in academia, consumer advocacy, industry, and government. The following summarizes the panel presentations as well as the discussions of the day.

In her opening remarks, Anna Alvarez Boyd, Senior Associate Director of the Division of Consumer and Community Affairs of the Federal Reserve Board, reviewed the Division's history of consumer testing in support of regulatory development. She identified objectives for the group by asking participants to consider how to make better use of research tools, close important gaps in the understanding of consumer finance, incorporate insight from other disciplines, research methods, and technologies, and strategically foster dialogue and collaboration that would promote evidence-based policymaking in financial services.

Panel I: Exploring the root disciplines for studying consumer behavior
Susan Woodward of Sand Hill Econometrics summarized findings from her research using three different datasets containing retail and wholesale mortgage loan originations, and used the results as the basis for a discussion of consumer behavior in the mortgage market and possible implications for policy. She addressed mortgage originator compensation and the role of yield-spread premiums (YSP)--a type of compensation paid to mortgage brokers by lenders in mortgage transactions. Theoretically in fully-efficient markets, there should be a dollar-for-dollar offset in upfront fees paid to the broker by the consumer and YSPs paid to brokers by the lender. However, her data showed offsets that ranged from a 50 cent reduction in upfront fees per dollar of YSP to a 35 cent increase in upfront fees per dollar of YSP. Holding other factors constant, less educated borrowers received worse deals, while borrowers with "no cost" loans--i.e. loans for which the consumer does not pay upfront fees because all the costs are reflected in the interest rate--obtained better deals. Woodward also provided evidence from an auction model that showed that borrowers who shop around for more than one quote save money. Furthermore, she argued consumers are confused by the rate-point tradeoff, and the mortgage markets may be more competitive when products are simpler, as in the case of no cost loans. She also suggested that in responding to the financial crisis, policymakers should take advantage of the opportunity to make changes in the mortgage finance system, including possible restrictions on fees, points, and charges.

John Payne of Duke University addressed consumer choices by bringing important insights from the field of psychology. Payne emphasized that the financial decision making process itself, task structure, and individual differences are all important mechanisms and opportunities for engineering better consumer choices. Psychology posits that individuals do not actually know their values prior to making decisions, but rather they construct their values during the decision-making process itself. For financial decisions especially, Payne argued that responses to the choice architecture of a task are shaped by selective attention which is driven by cognitive capabilities as well as "affective ease." He explained that affective ease is the idea that consumers require a point of reference to assess information, in order to remember it. Finally, task structure and individual differences will also cause disparities in consumer choice. In particular, Payne shed light on emotional differences among consumers that shape decisions. For example, even the sharpest consumers will have difficulty coping with the emotional challenges of retirement. He recommended giving consumers tools to deal with financial challenges as well as their emotional responses in the future.

Dan Bartels of Colombia Business School added to John Payne's explanation of consumer choice and harnessed marketing principals to encourage goal-based policy intervention. A lesson from marketing would find that facilitating savings goes beyond understanding consumer choice and markets the right products to the right consumers for their needs. Bartels explained that although planning is important for a long-term savings commitment, incentives for consumers are misaligned and the best products will hold consumers accountable to their savings goals. He noted that although perceived progress can facilitate goal-striving among consumers, information about progress can also hinder consumer choice when it distracts them from the main goals. For example, consumers could be motivated to pay off debts in full the closer they are to the goal. By the same token, consumers tend to pay off the smallest debts first even though larger debts may have higher interest rates. In designing consumer products Bartels advocated automated savings products, such as target-date funds, when consumers cannot self-regulate. He also explained that marketing these products should focus on increasing long-term financial incentives for consumers who are less connected with their future identity and thereby less timely in their goal setting agenda.

According to Margaret Sherraden, a sociologist from the University of Missouri-St. Louis, financial literacy can be linked to financial inclusion in society. Consumers who are opportunity-constrained need to be included in social institutions to effectively shape consumer behavior. Sherraden argued that because consumers are socially embedded, their financial decisions are affected not just by their education, but also the environment which surrounds them, including their access to financial services. Defining financial capability as a function of both an individual's financial education and their financial participation dramatically changes the policy prescription for improving consumer's financial capability and economic well-being. In studies of Individual Development Account program participants, Sherraden and her colleagues found that financial education encouraged saving up to a point: For up to ten hours of financial education, there was a positive effect on saving, but additional financial education beyond ten hours did not make a difference. After this point, the resources devoted to education may be better spent on improving access to financial services and products and directly helping underserved populations apply their knowledge.

Panel II: Exploring research methodologies for consumer testing and studying consumer behavior
Joe Garrett of Knowledge Networks discussed the role of online surveys in studying consumer behavior. Researchers can use internet panels to create a sample that is statistically representative of target populations. Although not all online survey panels are created with a random sampling frame, Garrett argued that online surveys offer data quality advantages by minimizing social desirability bias, giving consumers the chance to consult financial records, and facilitating message testing to see if questions were truly understood by respondents. Moreover, there are cost advantages to online surveys that provide a more efficient use of resources, minimizing the time between data collection and analysis. Garrett also argued that online surveys provide a better respondent experience by allowing consumers the choice of when they want to take the survey.

Turning to qualitative research methods, Susan Kleimann of Kleimann Communication Group presented a model that emphasized continual inquiry and feedback to develop strategic communications. In document design, such as for disclosures, Kleimann argued that making the task consumers need to accomplish the key element of focus helps researchers prioritize form content. Furthermore, informed decision-making requires that consumers go beyond comprehension of information to synthesis and evaluation, because the information consumers "know" and the information they use in decisions may be very different. She recommended designs for consumer testing that stimulate cognitive processes for informed decision-making. For example, think-aloud protocols reveal consumer thought processes without cueing the consumer as to what the researcher may think is important. In qualitative research, patterns can emerge as to how well the document is working after as few as five interviews with consumers. Kleimann argued that what is learned from this process can be significant. By targeting the appropriate audience for testing and fine-tuning the tools used to test documents, Kleimann believes that qualitative research can be a robust method for designing documents that consumers can use effectively in financial decision making.

In discussing experimental design for consumer decision making studies, Adair Morse of University of Chicago made the case that the research methodology and design used to inform policy must produce valid statistical results to be useful. For example, researchers may be able to measure the changes in the behavior of a group that receives a policy "treatment." However, if the treated group is not representative of the population, if the behavior change of the treated group is affected by factors other than the treatment, or if the treatment spills over onto the untreated population, the measured changes in behavior will not accurately reflect the real outcome of interest: the effect of the treatment on the general population. Morse advocated using field experiments to inform policy because randomization can account for a range of variables beyond the treatment. Internal validity problems arise in experiments when the treatment and control groups differ on unobservable attributes that bias the results. Similarly, external validity is lost when the study participants are not representative across the distribution of the population. To illustrate the challenges of implementing a sound experimental design, Morse discussed her research on payday loans. In the experiment, several different information "treatments" were delivered to payday loan customers to identify what disclosure changes could be made to increase consumer understanding and change behavior. The results showed that changing disclosure to show how payments will add up over time reduced payday borrowing by ten percent. She advocated for understanding the bias that motivates a particular consumer decision at a particular time in order to shape disclosure reform.

Hal Ersner-Hershfield of Northwestern University drew on the theory of the future self to design better interventions for consumers and increase savings for retirement. He expanded upon the theory discussed briefly by Bartels earlier in the conference and argued that under-saving results from a disconnect between consumers and their future selves, which increases with each degree of temporal difference. Ersner-Hershfield tested this theory first by asking 152 adults from the Bay area how closely they identified with their future self in ten years and what their savings and assets were. Although results were self-reported, he found evidence of a positive relationship between future connectivity and accrued assets over time, controlling for other characteristics such as education. Ersner-Hershfield also hypothesized that because the brain distinguishes between selfish thinking and thinking about others, different brain activity should be observed when a person thinks about themselves today ("selfish thinking") and their projected future self ("thinking about others"). In an experiment, he found that the brain activity of a person thinking of their future self is similar to that of a person thinking about an entirely different person, and this difference is intensified the larger the number of years between today and the subject's future self. Ersner-Hershfield proposed that if consumers could obtain a more vivid picture of their future self tomorrow, they would be more likely to change their behavior today. In a virtual reality experiment, computer programmers created avatars which allowed consumers to interact with their future selves and found that those consumers exposed to their future self were more likely to dedicate higher allocations to a hypothetical savings account. In a more simplified presentation, the face of the avatar was reconstructed and placed along the ends of a sliding scale; as saving levels changed along the scale, the facial expression for the avatar changed from happy to sad, depending on the amount of retirement savings. Ersner-Hershfield used this to demonstrate how advances in technology could provide better tools for shaping consumer behavior.

Lessons learned from Federal Reserve Board consumer testing
During lunch, Ellen Merry of the Federal Reserve Board explained how consumer testing has demonstrated the merits and limitations of disclosures in informing consumer decision-making. In particular, she referenced the Board's use of focus groups, cognitive interviews, usability testing, and quantitative validation surveys to inform disclosure development. While it is difficult to measure consumer decision-making per se, consumer testing has proved useful for making disclosures more comprehensible and usable. Using information from consumer testing on credit card disclosures, Merry illustrated that plain language works in concert with design to improve comprehension. Providing a context for the information in disclosures can also improve consumer understanding of disclosure content. Also, Merry noted that neutrality in disclosures can be difficult and that standardization is not always beneficial. For example, consumer testing of Truth in Lending Act mortgage disclosures showed that standardized tables of monthly payments were less effective in conveying the potential risk of higher future payments for adjustable rate mortgages than more customized tables that showed the "worst case scenario" for future payments. However, Merry noted that many other factors affect consumer decision-making and can override the effects of disclosures. For example, the setting in which disclosures are delivered, as well as who supplies the information and how it is presented, may influence consumer choice. She emphasized the need to understand the limitations of disclosures in improving financial capability when products are complex. Consumer protection may involve combining proper disclosure with education and regulation as well.

Panel III: Exploring key consumer issues
In her presentation, Elizabeth Costle of AARP explained how a "do-it-yourself" retirement approach today has complicated the savings decision and largely changed the financial game for elderly populations. Her policy recommendations to combat these rising problems drew on a three-way framework of the relationships between consumers, financial products, and intermediaries. Costle pointed out that even if consumers are financially capable, they are still hindered by obstacles in the marketplace such as unpredictable credit products, hidden fee structures for financial products, and complex investments. In using choice architecture to find suitable methods for consumers to make decisions, Costle argued that proper disclosures are not enough. She explained that no matter how perfect the product, regulations may be needed to govern how products are sold to consumers by intermediaries. Finally, Costle touched on issues such as fraud targeted at the elderly and an overall concern about how diminished capacity in old age affects senior citizens' ability to make financial decisions.

Michael Staten of University of Arizona presented financial trends on two interesting populations: youth and financially distressed consumers. Staten explained that in general, the youth population is more disconnected from financial institutions today than are other populations. His evidence also suggests that young consumers are more comfortable with technology and alternative forms of payment, such as prepaid debit cards. This challenges financial institutions to seek new methods for recruiting such customers, and Staten made the point that regulators need to think more about a wider array of information and disclosure opportunities to engage this population. He advocated revamping the financial education system at earlier ages and harnessing newer technology and interactive teaching methods to ensure students are engaged in education programs. However, a caveat to this is ensuring that teachers themselves understand course material. Staten explained that a lack of confidence among teachers concerning their own financial knowledge can limit their effectiveness. Staten also briefly discussed why most truly financially distressed consumers do not file for bankruptcy, but rather turn to non-profit counseling, debt settlement, or debt consolidation. He explained how the ability of non-profit counseling agencies to serve consumers is hindered by the single product they can offer: debt consolidation and full repayment in under 60 months. This product does not serve consumers who have too much debt or insufficient income to repay their debt in the allotted time. Consumers can thus be seduced by the pitches of debt settlement companies who claim to settle debts for pennies on the dollar. While not all these offers are scams, many are, and the practice of withholding payment on debts significantly damages a consumer's credit score. Moreover, as demonstrated in the session, sometimes not even the most financially capable consumers can differentiate between debt settlement companies and non-profit credit counselors. Staten recommended better product development, greater investments in messaging, and more clarity in the market to identify trusted counselors.

Jennifer Tescher of the Center for Financial Services Innovation addressed changes in the financial services marketplace, what this means for consumers, and how regulations and policy need to adjust to cope with these changes. In the near term, increased consumer protection may have unintended consequences for the unbanked and under-banked populations, potentially increasing the number of people without bank accounts in the U.S. Tescher argued that banks and credit unions will have to rethink their business models and pricing in light of new credit constraints and regulations. As banking becomes more expensive in the future, fees and other restrictions for consumers with insufficient account balances will make these financial services inaccessible for some. Tescher explained that older Americans and emerging adults will have redefined perspectives on financial services, given the context of the recent financial crisis and cultural paradigms. New technology has increased consumer expectations of immediate information access and this will change the way consumers are offered financial products and services, such as opportunities for account alerts. Products such as prepaid cards and mobile banking come with a broader array of service providers beyond "traditional banks." Tescher concluded with a brief discussion on how consumer expectations conflict with consumer needs, which complicates the task of designing a system that allows consumers to shop around for financial products. She advocated coupling education with how consumers plan to use the products and a better understanding of product substitutes.

Rene Pelegero brought experience from PayPal to the conversation regarding consumer payment choices and geared his presentation towards consumer protection. He explained that traditionally, the consumer payments framework included a value layer, such as banks and institutions that allow consumers to use their accounts to pay for products, a network layer such as online banking, and an access vehicle layer such as cash or a checkbook. However, Pelegero reasoned that a new delivery layer has been added to this framework by companies like PayPal and firms offering mobile payments, and this new layer is changing the payment environment. Increased competition in this consumer payment layer has spurred innovations such as mobile payments and the use of prepaid cards. He explained that new laws and regulations are needed in order to govern these new instruments and protect consumer interests. For example, the ambiguity over who holds the value of a pre-paid card could be a large problem if card issuers go out of business and the deposits are uninsured, leaving consumers with useless cards in line behind secured creditors to recover their money. He concluded that consumer and industry education are crucial in order to keep up with financial developments, and research is needed to ensure that the appropriate policies and regulations are made for these new markets and services.

In order to address Forum objectives set forth earlier in the day, small discussion groups were formed to synthesize panel presentations and discuss remedies for various issues of concern to consumers. While not surprising in a discussion among researchers, all groups said that better data is needed to inform consumer financial research. One group proposed a unique mortgage loan identifier, which would allow mortgage data from various institutions to be combined. The need to link consumer knowledge with understanding and then behavior change was commonly identified in discussions, as were tools for changing consumer financial decisions, whether through education or better consumer products. The complex relationship between financial products and regulation was debated widely and it was agreed that a proper balance should be achieved when engineering products and laws so that neither one negates the other in consumer protection. In particular, one group referenced the distinction between rule-based or principle-based regulations as a way of avoiding the "write a rule, avoid that rule, write another rule" process. The concern for a national research agenda was raised, and all groups seemed to support some type of unified consumer research network (national database, research consortium, or even an online experts network) in order to ease information flow and facilitate discussion, especially between policy makers and academics.

In his closing remarks, Allen Fishbein, Assistant Director of the Division of Consumer and Community Affairs, recognized the complexities of today's financial markets, consumer behavior, and the demands of applied, formative, and evaluative research. He emphasized the need for a cross-disciplinary approach to consumer behavior research in order to inform all aspects of future policy development: legislation, regulation, implementation, and evaluation. In particular, Fishbein encouraged researchers outside the policy arena to participate in the discussion of consumer protection and welcomed new insights from non-traditional areas of expertise.

1. The views expressed here are those of the presenters and do not necessarily represent the views of the Federal Reserve Board or the Federal Reserve System. Return to text

Return to topReturn to top

Skip stay connected section
Last update: March 30, 2012