| According to the Bank of International Settlements, | | | | stress testing outcomes nor did they sufficiently take |
| the financial crisis has highlighted weaknesses in | | | | account of qualitative expert judgment to develop |
| stress testing practices employed prior to the start | | | | innovative ad-hoc stress scenarios. |
| of the turmoil in four broad areas: | | | | Therefore, banks generally underestimated the |
| 1. Use of stress testing and integration in risk | | | | strong interlinkages between, for example, the lack |
| governance; | | | | of market liquidity and funding liquidity pressures. |
| 2. Methodologies | | | | The reliance on historical relationships and ignoring |
| 3. Scenario selection; and | | | | reactions within the system implied that firms |
| 4. Testing of specific risks and products. | | | | underestimated the interaction between risks and the |
| 1. Use of stress testing and integration in risk | | | | firm-wide impact of severe stress scenarios. |
| governance | | | | Prior to the crisis, most banks did not perform tests |
| Board and senior management involvement is critical | | | | that took a comprehensive firmwide perspective |
| in ensuring the appropriate use of stress testing in | | | | across risks and different books. Even if they did, the |
| banks' risk governance and capital planning. | | | | stress tests were insufficient in identifying and |
| This includes setting testing objectives, defining | | | | aggregating risks. As a result, banks did not have a |
| scenarios, discussing the results of tests, assessing | | | | comprehensive view across credit, market and |
| potential actions and decision making. | | | | liquidity risks of their various businesses. |
| At banks that were highly exposed to the financial | | | | An appropriately conducted firm-wide test would |
| crisis and fared comparatively well, senior | | | | have beneficially drawn together experts from across |
| management as a whole took an active interest in | | | | the organisation. For example, the expertise of retail |
| the development and operation of testing, with the | | | | lenders, who in some cases were reducing exposure |
| results of tests serving as an input into strategic | | | | to US subprime mortgages, should have counteracted |
| decision making which benefited the bank. | | | | the overly optimistic outlook of traders in securities |
| Testing practices at most banks, however, did not | | | | backed by the same subprime loans. |
| foster internal debate nor challenge prior assumptions | | | | 3. Scenario selection |
| such as the cost, risk and speed with which new | | | | Most bank stress tests were not designed to |
| capital could be raised or that positions could be | | | | capture the extreme market events that were |
| hedged or sold. | | | | experienced. Most firms discovered that one or |
| The financial crisis has also revealed weaknesses in | | | | several aspects of their tests did not even broadly |
| organisational aspects of stress testing programmes. | | | | match actual developments. In particular, scenarios |
| Prior to the crisis, testing at some banks was | | | | tended to reflect mild shocks, assume shorter |
| performed mainly as an isolated exercise by the risk | | | | durations and underestimate the correlations |
| function with little interaction with business areas. This | | | | between different positions, risk types and markets |
| meant that, amongst other things, business areas | | | | due to system-wide interactions and feedback |
| often believed that the analysis was not credible. | | | | effects. |
| Moreover, at some banks, the testing programme | | | | Prior to the crisis, "severe" stress scenarios typically |
| was a mechanical exercise. While there is room for | | | | resulted in estimates of losses that were no more |
| routinely operated stress tests within a | | | | than a quarter's worth of earnings (and typically much |
| comprehensive stress testing programme (eg for | | | | less). History has shown that when stress events |
| background monitoring), they do not provide a | | | | occur, banks can easily lose more than one quarter |
| complete picture because mechanical approaches can | | | | of earnings. |
| neither fully take account of changing business | | | | A range of techniques have been used to develop |
| conditions nor incorporate qualitative judgments from | | | | scenarios. At the most basic level there are |
| across the different areas of a bank. | | | | sensitivity tests, which only shock one single |
| Furthermore, in many banks, tests were carried out | | | | parameter, holding constant all other factors. Given |
| by separate units focusing on particular business lines | | | | that these scenarios ignore multiple risk factors or |
| or risk types. This led to organisational barriers when | | | | feedback effects, their main benefit is that they can |
| aiming to integrate quantitative and qualitative testing | | | | provide a fast initial assessment of portfolio |
| results across a bank. | | | | sensitivity to a given risk factor and identify certain |
| Prior to the crisis, many banks did not have an | | | | risk concentrations. |
| overarching stress testing programme in place but | | | | More sophisticated approaches apply shocks to many |
| ran separate tests for particular risks or portfolios | | | | parameters simultaneously. |
| with limited firm-level integration. Risk-specific testing | | | | Approaches are typically either historically based or |
| was usually conducted within business lines. | | | | hypothetical. |
| While stress testing for market and interest rate risk | | | | Historical scenarios were frequently implemented |
| had been practiced for several years, testing for | | | | based on a significant market event experienced in |
| credit risk in the banking book has only emerged | | | | the past. Such stress tests were not able to capture |
| more recently. Other tests are still in their infancy. As | | | | risks in new products that have been at the centre |
| a result, there was insufficient ability to identify | | | | of the turmoil. |
| correlated tail exposures and risk concentrations | | | | Furthermore, the severity levels and duration of |
| across the bank. | | | | stress indicated by previous episodes proved to be |
| Stress testing frameworks were usually not flexible | | | | inadequate. The length of the stress period was |
| enough to respond quickly as the crisis evolved (eg | | | | viewed as unprecedented and so historically based |
| inability to aggregate exposures quickly, apply new | | | | tests underestimated the level of risk and interaction |
| scenarios or modify models). | | | | between risks. |
| Further investments in IT infrastructure may be | | | | Banks also implemented hypothetical stress tests, |
| necessary to enhance the availability and granularity | | | | aiming to capture events that had not yet been |
| of risk information that will enable timely analysis and | | | | experienced. Prior to the crisis, however, banks |
| assessment of the impact of new stress scenarios | | | | generally applied only moderate scenarios, either in |
| designed to address a rapidly changing environment. | | | | terms of severity or the degree of interaction across |
| For example, investing in liquidity risk management | | | | portfolios or risk types. |
| information systems that would enhance the ability | | | | At many banks, it was difficult for risk managers to |
| of a bank to automate end-of-day information, | | | | obtain senior management buy-in for more severe |
| obtain more granularity as to unencumbered assets, | | | | scenarios. Scenarios that were considered extreme |
| and forecast balance sheet needs of business units. | | | | or innovative were often regarded as implausible by |
| 2. Methodologies | | | | the board and senior management. |
| Tests cover a range of methodologies. Complexity | | | | 4. Specific risks |
| can vary, ranging from simple sensitivity tests to | | | | Particular risks that were not covered in sufficient |
| complex tests, which aim to assess the impact of a | | | | detail in most stress tests include: |
| severe macroeconomic stress event on measures | | | | 1. The behaviour of complex structured products |
| like earnings and economic capital. | | | | under stressed liquidity conditions; |
| Tests may be performed at varying degrees of | | | | 2. Basis risk in relation to hedging strategies; |
| aggregation, from the level of an individual instrument | | | | 3. Pipeline or securitisation risk; |
| up to the institutional level. These tests are | | | | 4. Contingent risks; and |
| performed for different risk types including market, | | | | 5. Funding liquidity risk |
| credit, operational and liquidity risk. | | | | Scenarios were not sufficiently severe when stress |
| Notwithstanding this wide range of methodologies, | | | | testing structured products and leveraged lending |
| the turmoil has highlighted several methodological | | | | prior to the crisis. This may, to some degree, be |
| weaknesses. | | | | attributed to reliance on historical data. In general, |
| At the most fundamental level, weaknesses in | | | | stress tests of structured products suffered from |
| infrastructure limited the ability of banks to identify | | | | the same problems as other risk management |
| and aggregate exposures across the bank. This | | | | models in this area in that they failed to recognise |
| weakness limits the effectiveness of risk | | | | that risk dynamics for structured instruments are |
| management tools - including stress testing. | | | | different from those of similarly-rated cash |
| Most risk management models, including tests, use | | | | instruments such as bonds. |
| historical statistical relationships to assess risk. They | | | | These differences were particularly pronounced |
| assume that risk is driven by a known and constant | | | | during the crisis, further degrading the performance |
| statistical process, ie they assume that historical | | | | of the stress tests. Furthermore, stress tests also |
| relationships constitute a good basis for forecasting | | | | assumed that markets in structured products would |
| the development of future risks. | | | | remain liquid or, if market liquidity would be impaired, |
| The turmoil has revealed serious flaws with relying | | | | that this would not be the case for a prolonged |
| solely on such an approach. | | | | period. |
| First, given a long period of stability, backward-looking | | | | This also meant that banks underestimated the |
| historical information indicated benign conditions so | | | | pipeline risk related to issuing new structured |
| that these models did not pick up the possibility of | | | | products. |
| severe shocks nor the build up of vulnerabilities within | | | | In many cases tests dealt only with directional risk |
| the system. Historical statistical relationships, such as | | | | and did not capture basis risk, thereby reducing the |
| correlations, proved to be unreliable once actual | | | | effectiveness of hedges. |
| events started to unfold. | | | | Another feature of the crisis was wrong-way risk, |
| Second, the financial crisis has again shown that, | | | | for example related to the credit protection |
| especially in stressed conditions, risk characteristics | | | | purchased from monoline insurers. |
| can change rapidly as reactions by market | | | | Another weakness of the models was that they did |
| participants within the system can induce feedback | | | | not adequately capture contingent risks that arose |
| effects and lead to system-wide interactions. These | | | | either from legally binding credit and liquidity lines or |
| effects can dramatically amplify initial shocks as | | | | from reputational concerns related, for example, to |
| recent events have illustrated. | | | | off-balance sheet vehicles. |
| Extreme reactions (by definition) occur rarely and | | | | Had tests adequately captured in such exposures |
| may carry little weight in models that rely on historical | | | | may have been avoided. |
| data. It also means that they are hard to model | | | | With regard to funding liquidity, tests did not capture |
| quantitatively. The management of most banks did | | | | the systemic nature of the crisis or the magnitude |
| not sufficiently question these limitations of more | | | | and duration of the disruption to interbank markets. |
| traditional risk management models used to derive | | | | |