A Credit Scorecard That Means What It Says

Preview mode. Browse milestones and edit starter code here. Sign in to run and submit.

0/8EDA

Python

advanced~8h8 milestones

The problem

A lender approves personal loans and prices them off a predicted default probability. That number does not just rank applicants, it sets the interest rate and the amount held in reserve, so being right about the ordering is not enough: when the model says 8%, roughly 8 in 100 of those loans have to actually default. You will build the scorecard, discover that the standard fix for class imbalance destroys exactly this property, and then decide where to set the approval cutoff in dollars rather than in F1.

Your task · Base Rate and the Missing Column

Establish the default rate and check whether the missing delinquency dates are missing at random.

Implement explore_credit(train_df) returning a dict with:

row_count: number of training rows

default_rate: overall default rate, rounded to 6 places

missing_delinquency: count of NaN in months_since_last_delinquency

missing_share: that count over row_count, rounded to 6 places

default_rate_never_delinquent: default rate among rows where months_since_last_delinquency is NaN, rounded to 6 places

default_rate_prior_delinquency: default rate among the rest, rounded to 6 places

missingness_is_informative: True when the two rates differ by more than 0.02

The last three numbers are the point. The column is not missing because of a data-collection failure; it is missing because the event never happened, and those applicants are the safest ones in the book. Filling it with the median moves the safest cohort into the middle of the risk distribution.

Constraints

▪About 13% of funded loans default
▪months_since_last_delinquency is missing for the 66% who have never been delinquent, and that cohort defaults at roughly half the rate of the rest
▪applicant_region correlates with income, so any score will differ across regions
▪A performing loan earns $420, a default loses $2,600, a decline earns nothing
▪Hidden test set is scored server-side, so you cannot tune against it

Graded on

ROC-AUC >= 0.72 with expected calibration error <= 0.03
The class-weighted baseline shows ECE >= 0.15 at the same ROC-AUC
Brier score <= 0.105 on the hidden test set
Expected profit >= $300,000 at the chosen cutoff, against $282,540 for a naive 0.10 cutoff

Evaluated server-side against a hidden test set.

Skills you'll demonstrate

Probability CalibrationInformative MissingnessClass WeightingExpected Calibration ErrorCost-Sensitive ThresholdsSubgroup AnalysisLogistic Regression