Board of Governors of the Federal Reserve System

May 21, 2018

Is Operational Risk Regulation Forward-looking and Sensitive to Current Risks?

Marco Migueis¹

I. Introduction
Recent legislative initiatives have aimed to constrain the approaches followed by US banking agencies in setting operational risk capital.² Three principles appear to guide these legislative efforts: 1) requirements should be based in current businesses and exposures (rather than on losses from divested businesses); 2) requirements should be forward-looking (rather than backward-looking and fully reliant on past losses); and 3) requirements should account for operational risk mitigants. In this essay, I evaluate how current operational risk capital requirements and possible future reforms fare versus these principles.

Over the last fifteen years, operational failures and fraud have resulted in major losses to large US banks. The largest losses are dominated by multi-billion dollar legal settlements due to improper origination and securitization of loans in the run-up to the financial crisis, but the list of large operational losses is long and includes the rigging of LIBOR and auction rate securities markets, the Enron and WorldCom accounting scandals, the London Whale, and the Wells Fargo account fraud scandal. Current capital standards require the consideration of these losses.

Large, internationally active US banks are required to follow the Advanced Measurement Approach (AMA) to set operational risk capital requirements.³ This capital standard requires banks to model exposure to operational losses. Given the large losses banks have experienced, AMA models have generally resulted in large capital requirements. As of June 2017, the AMA contributes 29% of the risk-weighted assets of Advanced Approaches banks--a group of eleven of the largest and most complex US banks.⁴

The AMA follows a standard set by the Basel Committee in its Basel II reforms. On its own, the AMA is not prescriptive and gives banks wide latitude on how to calculate risk-weighted assets. But to ensure that capital requirements are appropriately conservative and comparable across banks, US regulators have attempted to narrow the range of practice, both through supervisory activities and formal guidance. In particular, banks have been expected to use their past operational losses in their exposure models, unless they can show that these losses are no longer relevant (Board of Governors of the Federal Reserve System 2014). This regulatory stance has been criticized by banks, with some bankers even questioning whether the operational risk capital requirement should exist at all (Dimon 2017).

Meanwhile, over the last few years the Basel Committee on Banking Supervision (BCBS) has undertaken a reform effort aiming to improve the simplicity and comparability of capital requirements (BCBS 2013, 2017a). Within this effort, the BCBS introduced a new standardized approach for operational risk. This standardized approach replaces both the AMA and the three existing standardized approaches for operational risk capital (these standardized approaches have not been adopted in the US) in the Basel standards. The new standardized approach uses an income statement metric, similar to gross income, plus past internal losses of banks to set the capital requirement through a regulatory formula. US banking agencies have indicated that they support the final package of Basel III reforms, which includes the standardized approach for operational risk (Board of Governors of the Federal Reserve System et al. 2017). But as in any federal rule change, the adoption of this new capital approach will first require a notice-and-comment rulemaking procedure, where the agencies will have to consider public feedback before finalizing the rule.

The remainder of this essay is organized as follows: section II gives a brief overview of the AMA and discusses how the principles of sensitivity to current risks, "forward-lookingness," and incorporation of risk mitigants relate to it; section III gives a brief overview of new standardized approach for operational risk capital and discusses how implementation of the three principles would affect its adoption in the US; section IV discusses how the three principles could be implemented within an alternative approach for operational risk capital; section V discusses whether the treatment of operational risk within CCAR can conform with the three principles; finally, section VI concludes.

II. The AMA
The AMA is the methodology that large, internationally active US banks are required to follow to set risk-weighted assets for operational loss exposure. The rule requires banks to calculate the 99.9^th percentile of their one-year operational loss exposure and sets this figure as the operational risk capital requirement. In this calculation, banks are required to consider past internal losses, past losses of other banks, scenario analysis, and business, environment, and control factors (BEICF). Outside of these data requirements the rule does not explicitly define how banks should calculate operational loss exposure.

In theory, the flexibility of the AMA framework allows for a variety of approaches, which could lead to lack of comparability across banks. However, through two interagency guidance documents, US banking regulators have narrowed the recommended range of practice for the AMA. In 2011 guidance (Board of Governors of the Federal Reserve System et al. 2011), regulators recommended that scenario analysis should not be used to lower the estimates obtained from a model based on loss data and that BEICF-based adjustments should have a limited magnitude. Thus, the 2011 guidance encourages banks to use historical loss data as the main pillar of the exposure model. 2014 interagency guidance further narrows the range of practice (Board of Governors of the Federal Reserve System 2014). This guidance asserts that cases where external losses are used to lower estimates relative to models relying on internal losses alone will be subject to additional scrutiny. When put together, the 2011 and 2014 guidance documents imply that banks' exposure estimates should generally be floored by model estimates based on past internal losses.

Skepticism regarding the use of external data, scenario analysis, and BEICF as the main data elements for AMA estimation results from challenges in their application to operational risk:

External losses--public information on operational losses is limited, as banks have incentives to not disclose this information. Some vendor datasets exist (such as SAS OpRisk Global Data), built from public sources such as news articles and financial disclosures, but these sources tend to not be comprehensive and to focus on the largest events. To solve these data limitations, banks have joined industry consortia, such as the Operational Risk eXchange (ORX). However, consortia loss data is typically anonymized due to its sensitivity.⁵ Such anonymity challenges the use of external loss data in bank models, as approaches for scaling the loss data and criteria for excluding loss events (e.g., should the largest losses in the industry history be included in the models of the smaller AMA banks?) are hard to justify.

Scenario analysis--scenario analysis in operational risk is a process through which bank risk management and business experts estimate exposure. Typically it involves experts from different areas of the bank participating in workshops where they identify key risks and estimate their likely frequency and severity (Board of Governors of the Federal Reserve System et al. 2011). Scenario analysis can be used to produce statistical estimates of exposure, which can be used to estimate AMA capital requirements (Shevchenko and Wütrich 2006). In multiple other countries scenario analysis is often the main AMA estimation methodology (BCBS 2009). To ensure the credibility of estimates, banks are expected to implement mechanisms to reduce bias and provide effective challenge to the scenario process. Nevertheless, scenario estimates are subjective and scenario workshop participants have incentive to underestimate exposure so as to not increase the bank's capital requirements. In addition, when estimating legal risks, banks are often concerned that revealing true estimates of exposure could, if made public, result in additional liabilities. Given the inherent subjectivity of scenario analysis and banks' incentives, absent a mechanism guaranteeing that banks have appropriate incentives to estimate exposure, scenario analysis likely cannot form the basis for a credible capital framework.

Business, environment, and control factors (BEICF)--BEICF are an umbrella term for a variety of other factors that can be useful to estimate operational loss exposure. Some, such as risk control self-assessments (RCSA) are subjective and, thus, suffer from similar weaknesses as scenario analysis in their application to capital calculations. Others, such as key risk indicators (KRI), are based on verifiable data and can conceivably be used to supplement past losses in estimating loss exposure. However, industry standard exposure models have not emerged for most risks so far.

The 2014 guidance also elaborates on the issue of loss exclusion. For purposes of estimating loss frequency, banks are generally allowed to exclude past losses from divested businesses. However, for purposes of estimating loss severity, the guidance sets a higher standard for the exclusion of past losses--particularly large legal losses--even from divested businesses. This different treatment can be justified as follows: loss frequency is typically dominated by large volumes of small losses, and when a bank discontinues a business (e.g., credit cards), assuming that the bank is no longer exposed to the large volume of small operational losses associated with this business (e.g., credit card fraud) is reasonable. However, allowing a similarly swift elimination of large legal losses associated with discontinued businesses from severity estimation would likely result in systematic underestimation for a couple reasons. First, even after divestiture, banks are often still exposed to residual legal liability. A second and more fundamental problem is statistical. Operational loss exposure models are often dominated by large and infrequent loss events. Typically, after experiencing large loss events, banks quickly take action to reduce exposure to the specific risk driver--for example, most banks reduced their exposure to subprime mortgages after the 2008 financial crisis. Sometimes, the market/product associated with the operational loss may collapse or be discontinued due to lack of confidence (Maxey 2012, Burne 2017). Thus, the failures that resulted in the large losses in the past are unlikely to cause large losses in the future. However, if a statistical model of exposure, based on historical losses, consistently sees large historical losses eliminated because the bank has divested from the business or because the losses are no longer "representative of current activities," the model is bound to permanently underestimate exposure. While large historical losses can be quickly eliminated from models, losses from new businesses take time to materialize. In addition to this problem, deciding whether a business has truly been exited is often subjective. For example, should a bank that no longer provides "sub-prime mortgages," but still is in the mortgage business, be able to remove operational losses related to "sub-prime mortgages" from capital calculations?

Given its flexibility, the AMA could be fully compatible with the principles of reliance on current exposures, forward-lookingness, and incorporation of risk mitigation. However, some of the supervisory recommendations described in the 2011 and 2014 would likely need to change to fully adopt these principles. The ensuing paragraphs examine how the AMA relates to these three principles.

1) Requirements should be based on current businesses and exposures--The intent of the AMA rule and subsequent interagency guidance documents is consistent with this principle. Nevertheless, the current implementation of the rule would likely need to change to fully achieve this principle. In particular, the reliance on past losses from divested businesses would likely need to be reduced. Regulators could argue that current supervisory practice--whereby past loss frequency of divested businesses can be quickly eliminated from models but elimination of large losses of divested businesses from severity estimation requires close regulatory scrutiny --complies with this principle, and that allowing easy elimination of large losses from divested businesses would lead to systematic underestimation. But while these regulatory concerns are sensible, the backward-looking nature of such approach is also clear.

2) Requirements should be forward-looking (and not solely based on past losses)--The AMA rule requires banks to consider external loss data, scenario analysis, and BEICF besides internal loss data. But the 2011 interagency guidance limited banks use of these elements to lower estimates and recommended that loss data form the basis of exposure estimation. This reliance on past internal losses can be justified by them being an objective, observed measure of a bank's riskiness. Adopting this principle would likely require supervisory practice to change and for forward-looking elements to assume a larger role on AMA estimates. However, given the subjectivity of scenario analysis and qualitative BEICF, adoption of this principle within the AMA framework would likely reduce the credibility of estimates and their comparability across banks.

3) Requirements should account for operational risk mitigants--Operational risk mitigants are allowed under the AMA rule, and the rule specifically discusses the use of insurance. However, the bar for recognition of insurance is relatively high as the rule requires that the methodology capturing the effects of insurance accounts for "mismatches in coverage between the policy and the hedged operational loss event."⁶ Such requirement is hard to implement within loss distribution approach models, the type of backward-looking loss model used by most banks to estimate exposure. Also, insurance policies often do not cover the largest legal exposures of banks, thus limiting banks' ability to meaningfully lower capital requirements through insurance. Some banks have included insurance in capital calculations through scenario analysis, but overall the use of insurance in the AMA has been limited. Banks can request the use of other mitigants besides insurance, but have not done so thus far. In summary, use of risk mitigants is possible under the current rule, but a broader use of mitigants would require a change in supervisory practices and possibly a rule change.

Overall, operational risk capital requirements could be set under the AMA, as defined in the US rule, while complying with the principles of reliance on current exposures, "forward-lookingness," and incorporation of risk mitigants. Some supervisory practices would need to change, and some small rule adjustments may be necessary (in particular, to add greater flexibility to the incorporation of risk mitigants). But full adoption of these principles within the AMA framework would likely result in gaming of the requirements, reduction of conservatism, and loss of comparability across banks.

III. The new standardized approach
Due to concerns regarding the complexity and lack of comparability of capital requirements (BCBS 2013), the BCBS worked over the last few years on revisions to the risk-weighted assets framework used to set capital requirements. Within this review, the operational risk capital framework from the Basel II accord was found particularly lacking and the BCBS replaced it with a new standardized approach, which would be a single regulatory formula to calculate operational risk capital (BCBS 2017a). The new standardized approach formula uses banks' financial statement information from the previous three years and operational losses from the previous ten years.

The principles of reliance on current exposures, "forward-lookingness," and incorporation of risk mitigants conflict with the goal of standardization embedded in BCBS's new standardized approach. In particular, ensuring that the operational risk capital framework is forward-looking would likely require US agencies to change the new standardized approach before adoption. Below I discuss in detail how the three principles relate to the new standardized approach:

1) Requirements should be based on current businesses and exposures--Similar to the AMA, this principle questions the use in the standardized approach of past losses from divested businesses or relating to failures for which the bank has improved controls. The new standardized approach allows banks to request supervisory approval to remove losses that are "no longer relevant to the banking organization's risk profile." The Basel text discusses what supervisors should take into account while making this decision, but no prescriptive restrictions are included. Thus, adopting the new standardized approach while following this principle should be feasible. However, unlike in the AMA, in the new standardized approach there is no distinction between loss frequency and severity; losses are either included or excluded from the calculation. And the problems discussed in the AMA section around how to estimate exposures for new and continuing businesses and of the definition of "exited business" would remain. Also, the BCBS did not have information to account for loss exclusions in its impact assessment;⁷ generalized exclusion of losses from discontinued businesses would likely meaningfully decrease capital requirements under the standard, which regulators could not directly estimate during calibration of the new standardized approach. So, if losses from divested businesses are excluded from capital calculation in an effort to comply with this principle, the comparability and conservatism of the framework would likely be reduced.

In addition to the use of losses from divested businesses, the use of a regulatory formula also presents challenges to the implementation of this principle. To show that the capital requirement is appropriately sensitive to current exposures, the US agencies would need to show how the formulas of the new standardized approach correlate with operational loss exposure. So far, the BCBS has not publicly provided evidence of the risk sensitivity of the new standardized approach. In proposing new rules, US regulatory agencies have to discuss the merits of their choices,⁸ and showing that a risk-based requirement is sensitive to the risk it pertains should likely be part of the rationale for such new rule. Given the lack of evidence provided by the BCBS on this regard so far, US agencies would need to fill this gap.

2) Requirements should be forward-looking (and not solely based on past losses)--Implementation of this principle poses the biggest challenge to the adoption of the new standardized approach. Can the new standardized approach formulas and exposure metrics be considered forward-looking? Regulators could argue that the financial statement component of the new standardized approach proxies for "current exposure"; but it represents an average from a three year window, which falls short of what most would likely consider "current exposure." Worse still, past losses are used for a window of ten years, which can hardly be defended as "forward-looking." These windows could be shortened by US regulators or the BCBS, but doing so would be unwise. Very short windows for income and past losses in the new standardized approach would result in meaningful capital instability--capital would shoot up after years where large losses were observed and decrease sharply in other years. This instability would likely make capital not risk sensitive or forward-looking. Still, despite the clear backwardness of the metrics used in the new standardized approach, perhaps the approach can be considered forward-looking if the exposure metrics and formulas used in it prove predictive of future exposure. Curti and Migueis (2016) find that past operational losses and income metrics are predictive of future operational losses. But the BCBS is yet to provide evidence that the specific exposure metrics and formulas of the new standardized approach are predictive of future exposure and thus "forward-looking" in this sense. Putting aside the statistical arguments that can be made for its "forward-lookingness," the new standardized approach does not represent what has commonly been understood as a forward-looking approach in the operational risk community over the last decade--which typically involves the use of scenario analysis and BEICF.

In my view, ensuring that the operational risk capital framework is forward-looking would likely require US agencies to, at a minimum, modify the new standardized approach before adoption. Would the introduction of a forward-looking element to the new standardized approach be sufficient? The answer is unclear because such a modified framework would include forward-looking and backward-looking elements. Is a framework "forward-looking" if it mandates the use of a backward-looking element for part of the calculation? Another relevant consideration is that, to ensure compliance with the final Basel III agreement, any such modification would require gold-plating the framework (i.e., the forward-looking element could not lead to a capital reduction versus the capital requirement obtained when the formula set out by the BCBS is used) and thus an increase in the conservatism of the approach.

3) Requirements should account for operational risk mitigants--The new standardized approach for operational risk allows netting of insurance recoveries in the calculation of past losses. This adjustment gives credit to insurance mitigation, but only on a backward-looking basis. Other risk mitigants (except for divestitures) are not incorporated into the standardized approach calculations. Unfortunately, I am not aware of methods that allow inclusion of operational risk mitigants on the new standardized approach on a forward-looking basis and simultaneously maintain the simplicity and comparability of the framework.

In summary, adopting the principles of reliance on current exposures, "forward-lookingness," and incorporation of risk mitigants would require modifications to the new standardized approach. These modifications could hamper the comparability and simplicity of the framework, and would likely result in a decrease of the originally intended conservatism. Ultimately, it is unclear whether the new standardized approach is compatible with the principle that the operational risk capital requirement should be forward-looking.

IV. A forward-looking capital framework
Both the AMA and the new standardized approach have significant shortcomings, which I discuss in Migueis (2018a). In summary, the AMA is prone to gaming and focuses on an unachievable statistical standard (the 99.9^th percentile of annual exposure) and the new standardized approach is backward-looking and thus unlikely to be risk sensitive in cases where a bank's risk profile is changing.

In Migueis (2018b), I propose an alternative framework for operational risk capital--the Forward-looking and Incentive-compatible Approach (FIA)--that aims to make the capital framework forward-looking. To achieve this goal, the FIA would allow banks to use their preferred models to model exposures in a forward-looking basis. But to ensure that this forward-looking component is not gamed and that the capital outcomes are comparable across banks, I propose that this component be made incentive-compatible. This means that banks are incentivized to provide accurate estimates. Incentive-compatibility is accomplished through a back-testing mechanism, whereby banks are required to hold additional capital if their estimates fell short of realized losses in the past. To ensure compliance with Basel III, as well as a degree of conservatism and comparability, the framework would use the new standardized approach as a floor.

In my view, the FIA is generally in line with the principles of reliance on current exposures, "forward-lookingness," and incorporation of risk mitigants. The approach would rely on forward-looking assessments, wherein banks could use whichever approaches they judge best to forecast exposure. Banks would only have to focus on current businesses and exposures and estimates could be adjusted to account for operational risk mitigants. However, a hard requirement that operational risk capital be forward-looking would present some challenges for the FIA. To ensure that banks do not systematically underestimate exposure, the FIA would require comparison of past loss estimates to realized losses. Looking back in this fashion is necessary to ensure that the forward-looking portion of the framework has proper incentives. Also, the FIA would need to retain the backward-looking new standardized approach as a floor to comply with Basel III. Use of such backward-looking information could be challenged if the principle that the requirements should be forward-looking was codified in law.

V. Operational risk in CCAR
The Federal Reserve runs models for an array of risks and products to estimate stressed losses for CCAR banks. The risks modeled include operational risk, which is included in the projected "Pre-Provisions Net Revenue" or "PPNR." Bank-by-bank projections for PPNR are publicly disclosed in the Federal Reserve's annual CCAR report, but the contribution of operational losses to these projections are not (Board of Governors of the Federal Reserve System 2017a). These operational loss projections are part of the buffer that the Federal Reserve requires banks to hold to face stress losses, and thus arguably constitute a capital requirement.

Federal Reserve's stress testing models are generally designed to apply industry wide.⁹ Ensuring that such framework is based on a firm's current exposures, is forward-looking, and allows for risk mitigants is likely to be challenging. The current Fed CCAR operational risk models rely on banks' reported historical losses (Board of Governors of the Federal Reserve System 2017a). The Fed started to collect data on banks' individual forward-looking scenarios in the FR-Y14A report in December 2016. However, the report does not require operational risk scenarios to be consistent across banks or provide guidance on how to ensure that they are comprehensive. Also, there has been no public indication of such scenario data being used in the Fed's internal model. In addition, the Fed currently does not collect data on risk mitigation, insurance contracts, or mapping of losses to specific businesses (outside of Basel business line definitions) in the FR-Y14 reports. In my view, the Fed would need to collect much more granular data to be able to adhere to the three principles within the CCAR operational risk modeling process. And even if this additional data is collected, incorporating such data into operational risk models would likely be challenging, as such data would be vast and hard to standardize across banks.

Even if the Fed approach were to follow the three principles, verifying compliance from the outside would be challenging. Historically, the Federal Reserve has not disclosed much detail regarding how stressed losses are modeled in CCAR due to concerns with gaming and with banks imitating the Fed's stress tests in their own stress tests (Tarullo 2014); the same has been true for the operational risk model.¹⁰ The most recent description of the model provided by the Fed is likely insufficient to allow independent verification that the model relies on current exposures, is forward-looking, and incorporates risk mitigants. In December 2017, the Federal Reserve issued a proposed rule aiming to enhance the transparency of the Fed's CCAR models (Board of Governors of the Federal Reserve System 2017b). However, the operational risk model is not specifically mentioned in this proposal, and it is unclear how much additional disclosure the Fed plans to provide regarding the model's inner workings. Still, the proposed disclosure for losses on loan portfolios shows that the transparency needed to allow independent verification of compliance with the three principles is likely beyond what policy makers have envisioned, as it is unclear whether anything short of full disclosure of the operational risk models would be sufficient.

VI. Conclusion
Ensuring that operational risk capital requirements reflect banks current exposures, are forward-looking, and give credit for risk mitigants are reasonable goals as they would increase the risk sensitivity of the framework. However, a strict interpretation of these principles and, in particular, their codification into law would cause some challenges within the current operational risk framework and new standardized framework developed by the BCBS. I believe regulators should strive to make the capital framework more forward-looking, and I have put forward a proposal to this effect (Migueis 2018b). However, backward-looking information (including past loss information, even of divested businesses) is often critical to understand future exposure, assess banks' model performance, and guarantee that underestimation does not persist. Thus, backward-looking information is necessary to control gaming and ensure capital comparability. A reformed operational risk capital framework should balance the objectives of reflecting current exposures, forward-lookingness, and incorporating risk mitigants with producing comparable capital requirements and limiting gaming opportunities.

References
Basel Committee on Banking Supervision (2009). "Observed range of practice in key elements of Advanced Measurement Approaches (AMA)."

Basel Committee on Banking Supervision (2013). "The regulatory framework: balancing risk sensitivity, simplicity and comparability."

Basel Committee on Banking Supervision (2017a). "Basel III: Finalising post-crisis reforms."

Basel Committee on Banking Supervision (2017b). "Basel III Monitoring Report."

Board of Governors of the Federal Reserve System (2014). "Supervisory Guidance for Data, Modeling, and Model Risk Management under the Operational Risk Advanced Measurement Approaches." Basel Coordination Committee Bulletin 14-1.

Board of Governors of the Federal Reserve System (2017a). "Dodd-Frank Act Stress Test 2017: Supervisory Stress Test Methodology and Results."

Board of Governors of the Federal Reserve System (2017b). "Enhanced Disclosure of the Models Used in the Federal Reserve's Supervisory Stress Test."

Board of Governors of the Federal Reserve System, Federal Deposit Insurance Corporation, Office of the Comptroller of the Currency (2017). "U.S. banking agencies support conclusion of reforms to international capital standards." https://www.federalreserve.gov/newsevents/pressreleases/bcreg20171207b.htm. Retrieved on April 9, 2018.

Board of Governors of the Federal Reserve System, Federal Deposit Insurance Corporation, Office of the Comptroller of the Currency, and Office of Thrift Supervision (2011). "Interagency Guidance on the Advanced Measurement Approaches for Operational Risk."

Burne, Katy (2017). "Banks Pick New Reference Rate to Replace U.S. Dollar Libor." The Wall Street Journal. https://www.wsj.com/articles/banks-vote-on-new-u-s-reference-rate-to-replace-libor-1498163666. Retrieved on April 9, 2018.

Curti, Filippo and Marco Migueis (2016). "Predicting Operational Loss Exposure Using Past Losses." Finance and Economics Discussion Series 2016-02. Board of Governors of the Federal Reserve System.

Dimon, Jamie (2017). Letter to shareholders. JP Morgan Chase. https://www.jpmorganchase.com/corporate/investor‐relations/document/ar2016‐ceolettershareholders.pdf. Retrieved on April 9, 2018.

Maxey, Daisy (2012). "For Many Auction-Rate Investors, the Freeze Goes On." The Wall Street Journal. https://www.wsj.com/articles/SB10001424052970203960804577243554110930094. Retrieved on April 9, 2018.

Migueis, Marco (2018a). "Evaluation of the AMA and the New Standardized Approach for Operational Risk Capital." Social Science Research Network.

Migueis, Marco (2018b). "Forward-looking and Incentive-compatible Operational Risk Capital Framework." Social Science Research Network.

Shevchenko, Pavel and Mario Wütrich (2006). "The structural modeling of operational risk bia Bayesian inference: combining loss data with expert opinions." Journal of Operational Risk Vol. 1 (3), 3-26.

Tarullo, Daniel (2014). "Stress Testing after Five Years." Speech at the Federal Reserve Third Annual Stress Testing Modeling Symposium, Boston, Massachusetts.