Here is a brief excerpt from an article written by Bernhard Babel, Kevin Buehler, Adam Pivonka, Bryan Richardson, and Derek Waldron for the McKinsey Quarterly, published by McKinsey & Company. To read the complete article, check out other resources, learn more about the firm, obtain subscription information, and register to receive email alerts, please click here.
To learn more about the McKinsey Quarterly, please click here.
* * *
Regulators have not issued specific instructions on how to do this. In the United States, they have stipulated that banks are responsible for ensuring that risks associated with machine-learning models are appropriately managed, while stating that existing regulatory guidelines, such as the Federal Reserve’s “Guidance on Model Risk Management” (SR11-7), are broad enough to serve as a guide.2
Enhancing model-risk management to address the risks of machine-learning models will require policy decisions on what to include in a model inventory, as well as determining risk appetite, risk tiering, roles and responsibilities, and model life-cycle controls, not to mention the associated model-validation practices. The good news is that many banks will not need entirely new model-validation frameworks. Existing ones can be fitted for purpose with some well-targeted enhancements.
New risks, new policy choices, new practices
There is no shortage of news headlines revealing the unintended consequences of new machine-learning models. Algorithms that created a negative feedback loop were blamed for the “flash crash” of the British pound by 6 percent in 2016, for example, and it was reported that a self-driving car tragically failed to properly identify a pedestrian walking her bicycle across the street.
The cause of the risks that materialized in these machine-learning models is the same as the cause of the amplified risks that exist in all machine-learning models, whatever the industry and application: increased model complexity. Machine-learning models typically act on vastly larger data sets, including unstructured data such as natural language, images, and speech. The algorithms are typically far more complex than their statistical counterparts and often require design decisions to be made before the training process begins. And machine-learning models are built using new software packages and computing infrastructure that require more specialized skills.
The response to such complexity does not have to be overly complex, however. If properly understood, the risks associated with machine-learning models can be managed within banks’ existing model-validation frameworks, as the exhibit below illustrates.
Highlighted in the exhibit are the modifications made to the validation framework and practices employed by Risk Dynamics, McKinsey’s model-validation arm. This framework, which is fully consistent with SR11-7 regulations and has been used to validate thousands of traditional models in many different fields of banking, examines eight risk-management dimensions covering a total of 25 risk elements. By modifying 12 of the elements and adding only six new ones, institutions can ensure that the specific risks associated with machine learning are addressed.
* * *
Here is a direct link to the complete article.
Bernhard Babel is a partner in McKinsey’s Cologne office; Kevin Buehler is a senior partner in the New York office, where Adam Pivonka is an associate partner and Derek Waldron is a partner; Bryan Richardson is a senior expert in the Vancouver office.
The authors wish to thank Roger Burkhardt, Pankaj Kumar, Ryan Mills, Marc Taymans, Didier Vila, and Sung-jin Yoo for their contributions to this article.