Paper: arXiv 2410.02846

Abstract

We introduce a novel machine learning model for credit risk by combining tree-boosting with a latent spatio-temporal Gaussian process model accounting for frailty correlation. This allows for modeling non-linearities and interactions among predictor variables in a flexible data-driven manner and for accounting for spatio-temporal variation that is not explained by observable predictor variables. We also show how estimation and prediction can be done in a computationally efficient manner. In an application to a large U.S. mortgage credit risk data set, we find that both predictive default probabilities for individual loans and predictive loan portfolio loss distributions obtained with our novel approach are more accurate compared to conventional independent linear hazard models and also linear spatio-temporal models. Using interpretability tools for machine learning models, we find that the likely reasons for this outperformance are strong interaction and non-linear effects in the predictor variables and the presence of spatio-temporal frailty effects.

Complexity vs Empirical Score

  • Math Complexity: 9.0/10
  • Empirical Rigor: 8.5/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper integrates advanced spatio-temporal Gaussian processes with tree-boosting, involving significant latent variable modeling and non-linear function estimation, indicating high mathematical complexity. It also demonstrates empirical rigor through application to a large U.S. mortgage dataset with comparative backtesting against multiple models, including portfolio-level metrics.

Research Flowchart

  flowchart TD
  A["Research Goal<br>Mortgage Credit Risk Modeling"] --> B{"Methodology"}
  
  B --> C["Data Inputs<br>Large U.S. Mortgage Dataset"]
  C --> D["Novel ML Model<br>Tree-Boosting + Latent Spatio-Temporal GP"]
  
  D --> E["Computational Process<br>Efficient Estimation & Prediction"]
  E --> F["Key Findings"]
  
  F --> G["Predictive Accuracy<br>Superior vs. Linear Models"]
  F --> H["Interpretable Drivers<br>Non-linearities & Spatio-Temporal Frailty"]

Cited References (87)

  1. Nonseparable dynamic nearest neighbor Gaussian process models for large spatio-temporal data with an application to particulate matter analysis (2016)
  2. Impacts of extreme weather events on mortgage risks and their evolution under climate change: A case study on Florida (2024)
  3. Joint models for longitudinal and discrete survival data in credit scoring (2023)
  4. Machine learning for corporate default risk: Multi-period prediction, frailty correlation, loan portfolios, and tail probabilities (2023)
  5. Grabit: Gradient tree-boosted Tobit models for default prediction (2019)
  6. Latent Gaussian model boosting (2023)
  7. Are hazard models superior to traditional bankruptcy prediction approaches? A comprehensive test (2014)
  8. Forecasting bankruptcy more accurately: A simple hazard model (2001)
and 79 more
  1. Financial ratios, discriminant analysis and the prediction of corporate bankruptcy (1968)
  2. Methodological issues related to the estimation of financial distress prediction models (1984)
  3. Spatial dependence in credit risk and its improvement in credit scoring (2016)
  4. Spatial dependence in microfinance credit default (2022)
  5. Commercial and residential mortgage defaults: Spatial dependence with frailty (2019)
  6. “Birds of a feather” fail together: exploring the nature of dependency in SME defaults (2019)
  7. Spatial contagion in mortgage defaults: A spatial dynamic survival model with time and space varying coefficients (2020)
  8. Machine learning models and bankruptcy prediction (2017)
  9. Financial contagion through space-time point processes (2021)
  10. Strictly proper scoring rules, prediction, and estimation (2007)
  11. Goodness of fit and related inference processes for quantile regression (1999)
  12. Evaluating volatility and interval forecasts (1999)
  13. Iterative methods for Vecchia-Laplace approximations for latent Gaussian process models (2025)
  14. Spatio-temporal models: Estimation & testing with an application to credit default rates (2012)
  15. Evaluating probabilistic classifiers: The triptych (2023)
  16. Real estate investors, the leverage cycle, and the housing market crisis (2011)
  17. A note on Gauss—Hermite quadrature (1994)
  18. Estimation and model identification for continuous spatial processes (1988)
  19. Hierarchical nearest-neighbor Gaussian process models for large geostatistical datasets (2016)
  20. Correlation-based sparse inverse Cholesky factorization for fast Gaussian-process inference (2023)
  21. Permutation and grouping methods for sharpening Gaussian process approximations (2018)
  22. A unified approach to interpreting model predictions (2017)
  23. From local explanations to global understanding with explainable AI for trees (2020)
  24. Measuring bank contagion in Europe using binary spatial regression models (2017)
  25. Spatial regression models to improve P2P credit risk management (2019)
  26. Joint modeling: an application in behavioural scoring (2019)
  27. Frailty correlated default (2009)
  28. Modeling frailty-correlated defaults using many macroeconomic covariates (2011)
  29. Gaussian processes for machine learning (2006)
  30. Joint model for longitudinal and spatio-temporal survival data — → 2311.04008
  31. Forecasting short-term defaults of firms in a commercial network via Bayesian spatial and spatio-temporal methods (2023)
  32. Gaussian process learning via Fisher scoring of Vecchia’s approximation (2021)
  33. A general framework for Vecchia approximations of Gaussian processes (2021)
  34. Greedy function approximation: a gradient boosting machine (2001)
  35. Additive logistic regression: a statistical view of boosting (2000)
  36. Boosting algorithms: Regularization, prediction and model fitting (2007)
  37. Gradient and Newton boosting for classification and regression (2021)
  38. Tree boosting with XGBoost-Why does XGBoost win" every" machine learning competition? (2016)
  39. Tabular data: Deep learning is not all you need (2022)
  40. Why do tree-based models still outperform deep learning on tabular data? (2022)
  41. Gaussian process boosting (2022)
  42. Classification and regression trees (1984)
  43. A boosted decision tree approach using Bayesian hyper-parameter optimization for credit scoring (2017)
  44. Ensemble boosted trees with synthetic features generation in application to bankruptcy prediction (2016)
  45. Forecasting with trees (2022)
  46. GPBoost: Combining tree-boosting with Gaussian process and mixed effects models (2021) — link
  47. SME default prediction: A systematic methodology-focused review (2024)
  48. SME default prediction: A systematic methods evaluation (2024)
  49. Benchmarking forecast approaches for mortgage credit risk for forward periods (2022)
  50. Thy neighbor’s mortgage: does living in a subprime neighborhood affect one’s probability of default? (2012)
  51. Modeling spatially interdependent mortgage decisions (2014)
  52. An analysis of mortgage termination risks: a shared frailty approach with MSA-level random effects (2011)
  53. Reducing estimation risk using a Bayesian posterior distribution approach: Application to stress testing mortgage loan default (2020)
  54. Credit scoring and its applications (2017)
  55. The good, the bad and the impaired: A credit risk model of the Irish mortgage market (2016)
  56. Mixture cure models in credit scoring: If and when borrowers default (2012)
  57. Mortgage prepayment and default decisions: A Poisson regression approach (1993)
  58. An empirical comparison of classification algorithms for mortgage default prediction: evidence from a distressed mortgage market (2016)
  59. Recent developments in consumer credit risk assessment (2007)
  60. Benchmarking state-of-the-art classification algorithms for credit scoring: An update of research (2015)
  61. Credit scoring with macroeconomic variables using survival analysis (2009)
  62. A note on the comparison of logit and discriminant models of consumer credit behavior (1980)
  63. Not if but when will borrowers default (1999)
  64. Forecasting and stress testing credit card default using dynamic models (2013)
  65. Support vector machines for credit scoring and discovery of significant features (2009)
  66. Modelling credit risk of portfolio of consumer loans (2010)
  67. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach (1988)
  68. The contagion effect of foreclosed properties (2009)
  69. Housing and Mortgage Markets with Climate Risk: Evidence from Wildfires in California (2024)
  70. Flood damage and mortgage credit risk: A case study of Hurricane Harvey (2020)
  71. Statistics for spatial data (2015)
  72. Statistics for spatio-temporal data (2011)
  73. COVID-19 pandemic in New York City — Wikipedia , The Free Encyclopedia (2025) — link
  74. Semiparametric spatio-temporal frailty modeling (2003)
  75. A Bayesian semiparametric temporally-stratified proportional hazards model with spatial frailties (2012)
  76. Modeling spatial survival data using semiparametric frailty models (2002)
  77. Do neighbours influence value-added-tax introduction? A spatial duration analysis (2017)
  78. An accuracy-runtime trade-off comparison of scalable Gaussian process approximations for spatial data (2025) — link
  79. Bayesian prediction of clipped Gaussian random fields (2000)

1 of 87 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.