Predicting Student Loan Defaults in South Korea Using Machine Learning: Insights obtained from SHAP Analysis on KOSAF Data

2.1 Direct Lean & Income-Contingent Loan

Since the importance of national fiscal soundness is increasing and education finances also occupy a large sector of national finances, it is worth conducting research on the factors of student loan default. Examining the student support systems of major countries, student loans are divided into two categories, i.e., Direct Loan (DL) and Income-Contingent Loan (ICL)[4].

DL are operated in the United States, Germany, Korea, and Japan, and are similar to loans handled by financial institutions, such as banks, with the repayment period after the loan being set. If the scheduled repayment is not repaid within the deadline, it is classified as overdue. In some cases, debt collection may proceed. ICLs were first introduced in Australia in 1989 and are being operated in the United States, New Zealand, Korea, and Japan. Repayment begins when the borrower earns a certain level of income. Unlike direct loans, these have the advantage of letting the students concentrate on their studies without having to pay off the loans while attending school, when no income is generated.

Even if the borrower becomes insolvent, the repayment of ICL is suspended until graduation, so the insolvency is revealed after a sizeable time. In Korea, ICL borrowers, who have repaid less than 5% of the principal amount five years after graduation, are separately managed as “long-term not repaying borrowers,” but there are also limitations limited to graduates. There can be a wide variety of economic changes for students attending or taking a leave of absence, and the risk of insolvency can be responded to early only when these changes are recognized in a timely manner. This study differs from others by identifying the factors for both DL and ICL default.

Predicting and preemptively preparing for poor student loans has developed into an academic field called risk management in financial institutions, including banks. In the event of loan insolvency, financial institutions impose delayed compensation and statistically analyze credit information on individuals to quantify the possibility of credit risk [5]. Thus, loans to low-credit customers are blocked in advance, or a higher interest rate is charged to prevent the expected risk.

2.2 Related Studies

Literatures related to the subject of the current study are divided primarily into four categories.

The first relates to factors contributing to student loan defaults. Many studies have shown that demographic factors, such as race, gender, age, and income, have a significant relationship with the level of student loan debt and the difficulty in repaying it[6-12]. Studies have also been conducted on the default rate of student loans by the type of educational institution. For-profit and other schools exhibit higher student loan default rates than traditional public and non-profit private schools. Han & Jeong[13] also developed a model for predicting the number of delinquencies in Korea by applying a logistic regression model according to the period people were delinquent in direct loans. This study identified vulnerable groups in delinquency, such as graduate students and humanities students. Additionally, the researcher mentioned that it is necessary to develop a model that includes additional variables, such as the financial information of delinquents, and to compare its performance with that of other models.

The second is related to ‘credit rating using non-financial information. Several studies have proven the effect of using non-financial information on personal credit ratings. Viani B. Djeundje et al. [14] analyzed whether alternative personal information, including gender and age, e-mail sending time, and psychological test results, affect default, using data from Lenddo, the world’s first alternative credit rating company. It was found that some variables significantly affect default; in particular, the predictive power (AUC) of the model increased by up to 17%, with the use of the above-mentioned variables, compared to that of individual data. Young-Jun Kwon et al. [15] conducted a study, using additional data on electricity rates, i.e., non-financial information, in their personal credit rating model. Customers who regularly paid electricity bills without delinquency, but did not engage in financial activities, were classified as unevaluable or high-risk groups in the existing credit rating model; therefore, credit loan approval was impossible.

The third constitutes a study of credit rating using machine-learning and deep learning. The development of big data analysis technology has helped researchers enhance the accuracy of default predictions using machine learning. Pedro et al. [16] applied XGBoost to 30 annual financial ratio datasets of 156 commercial banks from 2001 to 2015 to predict bank defaults in the United States. They revealed that lower values for retained earnings to average equity and pretax return on assets increase the risk of bankruptcy. Joo Wan Park et al.[17] used the decision tree, logistic regression, artificial neural network and random forest models, and support vector machine to build a credit rating model for small business owners. By comparing the correct classification rate and F-1 score of each model, using the evaluation data, they found that the logistic regression model had the best prediction performance. Kim & Moon [18] constructed various corporate default prediction models, using corporate data provided by the Taiwan Economic Journal, and compared their classification performance. Apart from individual classification models, the researcher also used ensemble models, such as XGBoost and LightBGM, and found that the classification performance of LightGBM was slightly better.

The fourth is related to the eXplainable AI (XAI). It is necessary to explain the prediction results in order to identify the main default factors. Therefore, researchers are actively developing explicable AI. Moscato et al. [19] constructed a personal credit rating model by applying logistic regression analysis and random forest to lending club data. Subsequently, they demonstrated the effect of the independent variables on the model, using LIME and SHAP. Chun et al.[3] described the factors that significantly influence credit ratings for individuals using Lending Club data and SHAP (Shapley Additive Explanation) values, based on the XGBoost model. In the logistic regression model, the feature importance of the default of each independent variable can be easily identified from the coefficients. However, it is difficult to verify the feature importance of other AI models without the SHAP values. Therefore, it is necessary to identify these factors using XAI to derive future policy implications.

As can be observed, student loan default and AI models have been investigated separately by several researchers, but few studies have combined the two topics.

Therefore, for accurate analysis, it is necessary to apply various machine learning techniques rather than statistical modeling to predict student loan defaults. Additionally, unlike DLs, ICLs are expected to have a higher risk of default due to the uncertainty of future repayments. Based on previous literature, this study attempts to derive detailed and comprehensive implications by considering current student loan issues.

3.1 Data collection

This study aims to develop a prediction model for DLs and ICLs from the Korea Student Aid Foundation (KOSAF), a public institution for student aid in Korea, and to identify the main factors of default. A number of loan accounts, i.e., 6,551,349, were analyzed and the structure of the data is shown in Table 1. All the variables used were categorized as data. Tuition level and student information by school system (college, university, graduate school) and type of school foundation (national, public, and private) differed. Because the data were sufficiently large, they were separated and analyzed for accuracy.

Table 1.

Structure of Analysis Data

The meaning of each variable is as follows: The variable, ‘Type of School Foundation,’ shows whether the customer’s affiliated university was a national or private university at the time of application for student loans. Likewise, the variable, ‘School Location,’ describes the categorized area where the customer’s affiliated university is located at the time of loan application. The ‘School System’ is categorized as follows: junior colleges were classified into three undergraduate years or shorter and universities were classified into four-year undergraduate years or more. The ‘Academic Field’ and ‘Day and Night’ classification code means the division of the customer’s affiliated department and day-night classification at the time of application for student loans. The variable, ‘School Year,’ is categorized into Entrants for first year students, and Enrolled Students for others at the time of application. The ‘Grades for previous semester’ variable represents the average grade of students in the semester preceding the time of application. ‘Income Section’ indicates the income level the customer’s household belongs to at the time of application. The higher its value, the more the converted income. ‘Age’ represents the age of customers at the time of application. ‘Fund usage’ is a variable that distinguishes the purpose for which the student loan will be used, primarily consisting of tuition and living expenses. Among the ‘Fund usage’ values, conversion loans are loans that convert Direct Loans, which were executed at relatively high interest rates in the past, to low interest rates of 2.9%,. ‘Employment status’ indicates whether the customer has commenced mandatory repayment by virtue of receiving an ICL repayment standard income or higher as of the end of 2022; this is used to analyze ICL.

All information, except employment and default, is available at the time of application for loans for each account. Student loan accounts are created for each semester according to the purpose of the loan. Therefore, even for the same customer, detailed information for each student loan account varies over time. If a change in customer status affects default, e.g., a long-term delay in student loans, it will be recognized in the customer’s recent account. However, if a customer’s financial condition deteriorates, accounts executed during better financial condition may also be overdue.

3.2 Data Analysis & Instrument

First, data pre-processing was performed. Missing values were identified in employment status data. The proportion of missing values in the total data was very small, at 0.08% for Direct Loans and 0.00% for ICL; therefore, missing values were excluded from the analysis.

Second, oversampling and undersampling were performed because the default value, i.e., the dependent variable, was concentrated in N. When the dependent variable is concentrated in a specific category, the model can be over-fitted for the major category and the predictive power of the model for new data can deteriorate. To solve this unbalanced data problem, this study used both oversampling and undersampling methods. Undersampling is a method that decreases the number of samples in the minority category, which accelerates the learning time. However, this causes a loss of sample data. Oversampling solves the problem of information loss by expanding the number of samples in multiple categories by the number of samples in multiple categories. The model performance was better than that of the undersampling method; however, it can cause overfitting and requires a long learning time[20, 21]. In this study, in consideration of efficiency, such as model learning time, DL with Y default and ICL data were oversampled, respectively, to 50% and 40% of the data in N. Additionally, the default N data were undersampled and adjusted so that the proportions of the two categories were the same.

Third, the study separates the data by loan product, school system, and school foundation. Loan products are divided into DL and ICL, the school system is divided into university - three years or shorter, university - four years or longer, and graduate school, and the type of school foundation is divided into national, public, and private schools. However, the data of national and public universities with three years or less are relatively small compared with those of private schools, which can reduce the significance of the analysis. Therefore, data with three years or shorter were integrated regardless of the school foundation. The ICL for graduates was first implemented in 2022; therefore, limited data were available, but this study included them for consistent analysis. The final data used in this study are shown in gray in Table 2.

Table 2.

Final data(this study used grey colored, unit: number of accounts)

Fourth, one-hot encoding was performed for categorical data analysis. For the measurement of model performance, 30% of the data were separated into test data. The models used were Random Forest, XGBoost, LightGBM, and CatBoost, as described above. The parameters were adjusted using a random search method.

Fifth, the predictive power of the constructed model was measured with the F-1 Score, which is a harmonized average of sensitivity and precision and is a balanced evaluation index that considers model overfitting. Recall that the ratio of accounts predicted by default among actual defaulted accounts can be used for the conservative prediction of default accounts; however, due to the nature of student loans, it was decided to use the F-1 Score instead of sensitivity to prevent excessive collection and promote customized financial support[23]. The calculation method for the F-1 Score is presented in Table 3. and Figure 1.

Table 3.

Confusion Matrix

Figure 1.

F-1 Score formula

Sixth, SHAP was applied to the model with the highest F-1 Score among the ensemble models used. Subsequently, for each data point, separated by loan product type, school system, and type of school foundation, the size and direction of the importance of the variables were verified using SHAP’s summary plot.

4.1. Student loan default rate prediction model

In order to construct a model for predicting the student loan default rate, four machine learning models were developed, based on the loan product type and school system. The machine learning models have the advantages that The predicted value was calculated using the test data and the F-1 Score was calculated by comparison with the actual value. Although the performance of the models was not significantly different, the Random Forest and CatBoost models for DL, and the XGBoost and CatBoost models for ICL, were relatively superior to the others. The CatBoost model, with an advantage in categorical data analysis, demonstrated a stable performance for all loan products and school system data. Table 4 shows a detailed comparison of the model performance by establishing classification and interdisciplinary systems.

Table 4.

The modeling results

The Random Forest model demonstrated the best performance for universities with three or fewer years. The F-1 Score reached a maximum of 79.11, indicating a relatively excellent predictive power. In the case of national and public universities, the Random Forest model demonstrated superior performance for general loans, and the XGBoost model, for ICL. The model performance for DL was 75.71%, which was higher than that for ICL. The CatBoost model performed the best in the case of private universities, with a F-1 Score of 73.60%, and the general loan model was superior to the ICL model. In the case of national and public graduate schools, the Random Forest model was found to be excellent for general loans, while the XGBoost model for ICL was 90.91%, which was higher than that of general loans. However, it was judged that attention to model interpretation was necessary because the ICL for graduate students in Korea was implemented in 2022, and the data were not sufficient. As a result of modeling private graduate schools, the performance of the CatBoost model was the best, and that of the ICL model was 80.27%, which is superior to that of general loans.

4.2 Key explanatory factors analysis based on eXplainable AI

To analyze the main explanatory factors of the default rate predicted by the model, this study applied SHAP to the Cat-

Boost model, which showed stable performance for each loan product and a summary plot. Because the size of the dataset was large, 10% of the training data were extracted for plotting. The summary plot was expressed as a point of how each variable affected the SHAP value of the individual predicted values, and the higher the frequency, the thicker the plot. Additionally, they were arranged in descending order of SHAP value. This allows us to determine whether the model generally applies the variable as a factor that increases or lowers the risk of default[21].

Figure. 2 shows the SHAP plot derivation results for the loan products, school systems, and types of school foundations. The summary plot is interpreted on the basis of color and direction. For variable values of Y(1) and Y(0), the colors are red and blue, respectively. The plot distributed in the left (right) direction indicates that the variable lowers (increases) the default rate. For example, for DL borrowers in university with 3 or lesser years, the default rate tends to decrease if the university is in a metropolitan area. If the grade at the time of application is less than 80 points (B), the default rate tends to increase (The full figures are presented in the appendix.).

Figure 2.

Result of SHAP Summary Plot(DL / Univ. 3-yr or shorter)

Figure 3. summarizes the common factors with the maximum effect on the default rate. It was found that for students, under the age of 24, in the medical field, in metropolitan area schools, with grades of 90 points or more, the default rate tends to decline. On the other hand, for students in the humanities and social education/ arts and sports field, male, with grades of less than 80 points, and income deciles of less than four sections, the default rate tends to increase.

Figure 3.

Common Factors for Falling and Rising Defaults

Table 5. presents the results of the analysis of the main factors affecting default by loan product and school system type. In the case of universities with a tenure of at most three years, school location and grades appear to significantly influence the default rate, regardless of the loan product. The key factor reducing the default rate is the school being located in a metropolitan area, and that increasing the default rate is the grades in the previous semester being lower than 80 points.

Table 5.

Top 3 Factors of Default Rate by Loan Product and School System

In the case of a DL borrower at a national and public university, the default rate decreases when the grades of the previous semester are 90 points or higher, and increases when the income section is under four. For ICL borrowers, like in case of DL, the default rate increases when the income section is below four. If they receive living expense loans, their default rates decrease.

In the case of a private university student and a DL borrower, the default rate decreases when the grade of the previous semester is at least 90 points, and increases when the score is less than 80 points, similar to the case in national and public universities. In the case of an ICL borrower, the default rate decreases when the age of the applicant is less than 24 years old, and the default rate increases when the income section is less than four or when the major is in the arts and sports field.

In the case of national and public graduate students and DL borrowers, the default rate decreases when the major is in the medical field and the default rate increases when the applicant is over 35 years old. In the case of ICL, the default rate decreases when the age range is 25–29 years. If students are in graduate schools located in the Gyeongsang region or are 30–34 years old, the default rate increases.

In the case of private graduate school students and DL borrowers, the default rate decreases when the major is in the medical field. If the applicant’s age is 35 years or older, the default rate increases. For ICL borrowers, if the applicant’s age is 30-34 years, the default rate appears to increase.