Machine Learning Methods as a Benchmark for Classical Rating Approaches
Comparison of ML-based and classical statistical rating approaches using ROC curves and Gini coefficients in the credit risk context.
Authors: Prof. Dr. Dirk Schieborn, Prof. Dr. Volker Reichenberger
Journal: Zeitschrift für das gesamte Kreditwesen
Year of publication: 2018
Abstract
This article represents one of the early contributions to the systematic evaluation of machine learning methods in the context of bank ratings. The authors examine whether and to what extent machine learning approaches outperform classical statistical rating models – such as logistic regression or linear discriminant analysis – in terms of predictive accuracy.
Methodology
The central performance criterion is the ROC curve and the derived Gini coefficient, which are established as the standard measure of discriminatory power for rating systems in the regulatory context. Methods compared include:
- Classical approaches: Logistic regression, linear discriminant analysis
- ML approaches: Random Forest, Gradient Boosting, Support Vector Machines
Results
ML methods consistently achieve higher Gini coefficients than classical rating models across the credit portfolios examined. At the same time, the authors show that the superiority of ML methods increases with growing data complexity and non-linearity of dependencies – and discuss the regulatory implications for approval in the IRBA context.