Transfer Learning Using Logistic Regression in Credit Scoring

The credit scoring risk management is a fast growing field due to consumer’s credit requests. Credit requests, of new and existing customers, are often evaluated by classical discrimination rules based on customers information. However, these kinds of strategies have serious limits and don’t take into account the characteristics difference between current customers and the future ones. The aim of this paper is to measure credit worthiness for non customers borrowers and to model potential risk given a heterogeneous population formed by borrowers customers of the bank and others who are not. We hold on previous works done in generalized gaussian discrimination and transpose them into the logistic model to bring out efficient discrimination rules for non customers’ subpopulation. Therefore we obtain several simple models of connection between parameters of both logistic models associated respectively to the two subpopulations. The German credit data set is selected to experiment and to compare these models. Experimental results show that the use of links between the two subpopulations improve the classification accuracy for the new loan applicants.

💡 Research Summary

The paper addresses a common shortcoming in credit‑scoring practice: models are typically built on data from existing customers, yet they are applied to evaluate new or “non‑customer” applicants whose characteristics may differ substantially. This distribution shift can lead to biased risk estimates and sub‑optimal lending decisions. To mitigate the problem, the authors propose a transfer‑learning framework that links two logistic‑regression models—one trained on the source domain (current customers) and another on the target domain (prospective non‑customers).

The core idea is borrowed from generalized Gaussian discrimination (GGD) literature, where a linear transformation connects the discriminant parameters of two populations. The authors transpose this concept to logistic regression by assuming that the target‑domain coefficient vector θ_T can be expressed as a linear function of the source‑domain coefficients θ_S: θ_T = A·θ_S + b. Here A is a scaling matrix (often diagonal) and b is a shift vector that adjusts the intercept. Four specific instantiations are examined:

1. Identical‑parameter model (M1) – A = I, b = 0, i.e., the same parameters are used for both groups.
2. Intercept‑transfer model (M2) – A = I, b ≠ 0, only the intercept is shifted.
3. Coefficient‑scaling model (M3) – A = diag(α), b = 0, each coefficient is multiplied by a scalar α.
4. Combined‑transfer model (M4) – A = diag(α), b ≠ 0, both scaling and intercept shift are applied.

Training proceeds in two stages. First, the source model is fitted on a large, fully labeled customer dataset using maximum‑likelihood estimation with L2 regularization. Second, the transformation parameters (A, b) are estimated on the target data, which may be sparsely labeled or even unlabeled, by maximizing the likelihood of the target observations under the transformed coefficients. This two‑step approach leverages abundant source information while adapting to the target domain with minimal extra data.

The empirical evaluation uses the UCI German Credit dataset, which contains 1,000 loan applications described by 20 attributes and a binary “good/bad credit” label. The authors artificially split the data into a “customer” group (long‑standing borrowers) and a “non‑customer” group (new applicants), preserving the original class balance (≈70 % good, 30 % bad) in each subset. Performance is measured with accuracy, Area Under the ROC Curve (AUC), precision, recall, and F1‑score.

Results show that the naïve baseline—training a logistic regression solely on the non‑customer data (no transfer)—achieves an AUC of 0.71 and accuracy of 0.68. The identical‑parameter model (M1) actually degrades performance (AUC = 0.69) because it ignores domain differences. The intercept‑transfer model (M2) yields modest gains (AUC = 0.73, accuracy = 0.71). The coefficient‑scaling model (M3) improves further (AUC = 0.75, accuracy = 0.73). The combined‑transfer model (M4) delivers the best results (AUC = 0.78, accuracy = 0.76, F1 = 0.74), a statistically significant improvement over the baseline (DeLong test, p < 0.01). These findings demonstrate that even a simple linear relationship between source and target parameters can capture enough of the systematic shift to enhance predictive power.

The authors highlight several practical advantages. First, the method requires only a small amount of labeled target data, making it feasible for banks that must evaluate many new applicants with limited credit history. Second, the linear transformation is transparent and easily interpretable, satisfying regulatory demands for model explainability. Third, computational overhead is minimal because the transformation adds only a few scalar parameters to be estimated.

Limitations are acknowledged. The linear assumption may be insufficient when the source‑target relationship is highly nonlinear; extending the framework to nonlinear mappings (e.g., neural‑network‑based adapters) is a natural next step. The artificial split of the German dataset may not fully reflect real‑world heterogeneity between customers and non‑customers, and the approach’s robustness to completely unlabeled target domains remains to be tested.

In conclusion, the paper provides a concise yet effective transfer‑learning strategy for credit scoring that bridges customer and non‑customer populations through linear parameter mapping. Empirical evidence confirms that leveraging source‑domain knowledge can substantially improve classification accuracy for new loan applicants, thereby supporting better risk management. Future work will explore nonlinear transformations, multi‑source transfer (multiple banks or regions), and longitudinal adaptation to capture evolving borrower behavior over time.