An analysis of the value of external studies to risk managers, and how to improve them

Once again during the last part of the year, academic institutions, consultancy firms, think-tanks and insurance companies are publishing studies on top risks. But what is the value of these studies to risk managers? The impact on the media and social networks surely justifies the marketing value for the organisations funding the reports. However, the impact on street-level risk managers is to be discussed.

As one example, a large insurance firm recently issued a report (the consolidated perceptions of 2,700 risk experts from 54 countries) which showed pandemics and infectious deceases was the top emerging risk. It was followed by climate change, cybersecurity, geological instability, and social discontent. 

A professional instiute published another risk report at the same time. It showed that cybersecurity, compliance, digitalisation and finance are the top risks for 2021. The study covered responses from chief audit executives.

The highly generic risks listed by these studies cannot be extrapolated to specific scenarios for decision-making.

Even as alerts of threats, the applicability of a generic external risk list offers little insight. The very different time horizon of risks scenarios in the lists also prevents their applicability for business cases. The time horizon of a risk triggered by climate change is completely different from cybersecurity, for instance. As most of these lists are missing impacts and probabilities, they cannot be used for benchmarking either.

Many of these studies are generally based on structured surveys distributed to members and commercial contacts without following scientific methods. Interviews are a less popular alternative. As a result, findings are inconsistent between the different annual editions.  

The selection of questions and how they are answered is highly influenced by the availability bias of preparers and participants. In consequence, the evolution of reported top risks is matching the media coverage of recently highly-visible cases from compliance fines and cyber-security to pandemics.

Moreover, participants are likely to adjust their responses so they can be externally shared while complying with their confidentially agreements. The unclear definitions of scenarios, qualitative thresholds and vocabulary also creates cognitive distortions in this approach.  

A data-driven approach

Studies on trends based on historical datasets offer much better insight for risk managers, in particular when they are focused on a sector, regulation type or geographical area.

Reports based on this approach can help to identify trends for applicable risk sources relevant for a particular decision-making process. Consolidating publicly available data on risk incidents, such as regulatory breaches or cybersecurity events, can be used for benchmarking or references in assessments. As long as there are not changes in the environment or objectives, past information can provide insights about the sources, probabilities and impacts of risks.

For instance, there is a value in studies analysing trends of fines related to privacy. The statistical study of the different types of control failures which triggered fines in the past can help to assess compliance risks. The assessment of impacts for specific requirements can be calculated or updated by using external data on articles of laws and regulations, which were breached and finally supported the imposition of fines.

Several associations and risk data providers offer this type of information. As an example, the ORX, a non-profit association, secures and anonymises the exchange of high-quality operational risk loss data across banks around the world.

Organisations funding these type of studies should also focus on their areas of expertise when recommending lessons learned and practical tips to risk managers. Reports on control best practices and recommendations for common scenarios can greatly assist risk managers.

The risk management profession has come a long way from a data analysis and quantification perspective and should therefore avoid giving undue attention to transient perceptions.

External lists based on general opinions and polls (offering up titles of disaster films in the press release) are attractive to an uneducated audience, but they do not address the needs of the data-driven risks manager.  

Professor Hernan Huwyler is head of Vendor Due Diligence and Third Party Risk at Danske Bank