Paper: arXiv 2410.05297

Abstract

Cyber risk classifications are widely used in the modeling of cyber event distributions, yet their effectiveness in out of sample forecasting performance remains underexplored. In this paper, we analyse the most commonly used classifications and argue in favour of switching the attention from goodness-of-fit and in-sample predictive performance, to focusing on the out-of sample forecasting performance. We use a rolling window analysis, to compare cyber risk distribution forecasts via threshold weighted scoring functions. Our results indicate that business motivated cyber risk classifications appear to be too restrictive and not flexible enough to capture the heterogeneity of cyber risk events. We investigate how dynamic and impact-based cyber risk classifiers seem to be better suited in forecasting future cyber risk losses than the other considered classifications. These findings suggest that cyber risk types provide limited forecasting ability concerning cyber event severity distribution, and cyber insurance ratemakers should utilize cyber risk types only when modeling the cyber event frequency distribution. Our study offers valuable insights for decision-makers and policymakers alike, contributing to the advancement of scientific knowledge in the field of cyber risk management.

Complexity vs Empirical Score

  • Math Complexity: 7.0/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced statistical methods including dynamic extreme value theory, GAMLSS, and sophisticated scoring rules (CRPS, ES, rCRPS, rES), indicating high mathematical complexity. It also demonstrates strong empirical rigor through the use of a real-world dataset (Advisen), rolling-window out-of-sample forecasting, and simulation studies to validate findings.

Research Flowchart

  flowchart TD
  A["Research Goal: Assess Cyber Risk Classifications for Out-of-sample Forecasting"] --> B{"Data & Methodology"}
  
  B --> C["Input: Cyber Event Data<br>Risk Classifications<br>Threshold Weighted Scoring"]
  C --> D["Computational Process:<br>Rolling Window Analysis"]
  D --> E["Evaluate Out-of-sample<br>Distribution Forecasts"]
  
  E --> F["Key Findings & Outcomes"]
  
  F --> G["Dynamic & Impact-based<br>Classifiers perform best"]
  F --> H["Business-motivated<br>Classifiers too restrictive"]
  F --> I["Recommendation:<br>Use types for Frequency,<br>not Severity distribution"]

Cited References (98)

  1. The drivers of cyber risk (2020)
  2. Insurability of cyber risk: An empirical analysis (2015)
  3. Modelling and management of cyber risk (2015)
  4. Insurability of cyber risk: An empirical analysis (2015)
  5. Content analysis of cyber insurance policies: How do carriers price cyber risk? (2019)
  6. Cyber risk and the changing role of insurance (2017)
  7. Impact of insurance for operational risk: Is it worthwhile to insure or be insured for severe losses? (2011)
  8. Cyber risk for the financial sector: A framework for quantitative assessment (2018)
and 90 more
  1. Cyber-risk decision models: To insure IT or not? (2013)
  2. Cyber-risk management (2015)
  3. Cyber risk research in business and actuarial science (2020)
  4. What do we know about cyber risk and cyber risk insurance? (2016)
  5. Data breaches: Goodness of fit, pricing, and risk measurement (2017)
  6. Cyberattacks on critical infrastructure: An economic perspective (2019)
  7. Residual life time at great age (1974)
  8. An extreme value approach for modeling operational risk losses depending on covariates (2016)
  9. A taxonomy of operational cyber security risks (2010)
  10. A taxonomy of operational cyber security risks version 2 (2014)
  11. Operational loss scaling by exposure indicators: evidence from the ORX database (2008)
  12. A Clearer Vision for Assessing the Risk of Cyber Incidents (2020)
  13. Scaling models for the severity and frequency of external operational loss data (2010)
  14. What are the actual costs of cyber risk events? (2019)
  15. Data breaches: Goodness of fit, pricing, and risk measurement (2017)
  16. Hype and heavy tails: A closer look at data breaches (2016)
  17. Human behaviour as an aspect of cybersecurity assurance (2016)
  18. A scaling model for severity of operational losses using generalized additive models for location scale and shape (GAMLSS) (2013)
  19. Heavy-tailed distribution of cyber-risks (2010)
  20. Examining the costs and causes of cyber incidents (2016)
  21. Statistical inference using extreme order statistics (1975)
  22. Cost of Data Breach Report 2019 (2019)
  23. Likelihood ratio tests for model selection and non-nested hypotheses (1989)
  24. Cyber risk ordering with rank-based statistical models (2020)
  25. Dealing with the biased effects issue when handling huge datasets: the case of INVALSI data (2015)
  26. A simple sequentially rejective multiple test procedure (1979)
  27. Cyber risk ordering with rank-based statistical models (2020)
  28. Statistical machine learning analysis of cyber risk data: event case studies (2018)
  29. Understanding cyber-risk and cyber-insurance (2018)
  30. Generalized additive models for location, scale and shape (2005)
  31. Generalized additive models for location scale and shape ( GAMLSS ) in R (2008)
  32. Statistics of extremes by oracle estimation (2008)
  33. Nonparametric adaptive estimation of conditional probabilities of rare events and extreme quantiles (2015)
  34. A simple general approach to inference about the tail of a distribution (1975)
  35. Flexible regression and smoothing: using GAMLSS in R (2017)
  36. The modelling of operational risk: experience with the analysis of the data collected by the Basel Committee (2004)
  37. Capital and risk: New evidence on implications of large operational losses (2006)
  38. A tale of tails: an empirical analysis of loss distribution models for estimating operational risk capital (2006)
  39. Data clustering with actuarial applications (2020)
  40. An integrated system for estimating the risk premium of individual car models in motor insurance (1986)
  41. Model-based and nonparametric approaches to clustering for data compression in actuarial applications (2017)
  42. An empirical comparison of some experimental designs for the valuation of large variable annuity portfolios (2016)
  43. A data mining framework for valuing large portfolios of variable annuities (2017)
  44. Pattern recognition by an adaptive process of sample set construction (1962)
  45. Some methods for classification and analysis of multivariate observations (1967)
  46. A bootstrap goodness of fit test for the generalized Pareto distribution (2009)
  47. A simple method for testing goodness of fit in the presence of nuisance parameters (1980)
  48. The exact and asymptotic distributions of C ram 'e r-von M ises statistics (1996)
  49. A review of quadratic discriminant analysis for high-dimensional data (2018)
  50. Transforming big data into smart data: An insight on the use of the k-nearest neighbors algorithm to obtain quality data (2019)
  51. Application of K-nearest neighbors algorithm on breast cancer diagnosis problem. (2000)
  52. Extreme Data Breach Losses: An Alternative Approach to Estimating Probable Maximum Loss for Data Breach Risk (2021)
  53. Copula approaches for modeling cross-sectional dependence of data breach losses (2018)
  54. Problems with scoring methods and ordinal scales in risk assessment (2010)
  55. Attack chain detection (2015)
  56. The use of the rank transform in regression (1979)
  57. Infinite mean models and the LDA for operational risk (2006)
  58. The calculation of minimum regulatory capital using single-loss approximations (2010)
  59. Probabilistic forecasts, calibration and sharpness (2007)
  60. Comparing density forecasts using threshold-and quantile-weighted scoring rules (2011)
  61. Comparing density forecasts via weighted likelihood ratio tests (2007)
  62. Axiomatic characterization of the quadratic scoring rule (1998)
  63. Strictly proper scoring rules, prediction, and estimation (2007)
  64. Cyber loss model risk translates to premium mispricing and risk sensitivity (2023)
  65. Cyber risk frequency, severity and insurance viability (2022)
  66. Evaluating the discrimination ability of proper multi-variate scoring rules (2022)
  67. Energy statistics: A class of statistics based on distances (2013)
  68. New goodness-of-fit tests for Pareto distributions (2009)
  69. Comparing density forecasts via weighted likelihood ratio tests (2007)
  70. Standards for security categorization of federal information and information systems (2004)
  71. Australian Cyber Security Centre: Annual Cyber Threat Report. (2021)
  72. An investigation of cyber loss data and its links to operational risk (2019)
  73. Defining cyber risk (2021)
  74. The drivers of cyber risk (2022)
  75. Cyber risk management in the US banking and insurance industry: A textual and empirical analysis of determinants and value (2022)
  76. Reducing informational disadvantages to improve cyber risk management (2018)
  77. Cyber risk management: History and future research directions (2021)
  78. Cyber risk and the US financial system: A pre-mortem analysis (2022)
  79. The future of financial stability and cyber risk (2018)
  80. Cyber risk definition and classification for financial risk management (2023)
  81. Cyber runs: How a cyber attack could affect US financial institutions (2019)
  82. Some principles for regulating cyber risk (2019)
  83. Cyber risk and financial stability: It’sa small world after all (2020)
  84. Managing cyber risk, a science in the making (2023)
  85. Building up cyber resilience by better grasping cyber risk via a new algorithm for modelling heavy-tailed data (2023)
  86. Cyber claim analysis using Generalized Pareto regression trees with applications to insurance (2021)
  87. The nature of losses from cyber-related events: risk categories and business sectors (2023)
  88. European Cybersecurity Centres of Expertise Map - Definitions and Taxonomy (2018)
  89. A proposal for a European cybersecurity taxonomy (2019)
  90. CRO forum concept paper on a proposed categorisation methodology for cyber risk (2017)

0 of 98 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.