The missing evidence sits behind a business decision
Reject inference addresses a practical lending problem: the lender sees repayment on loans it makes, but not repayment on a proposed loan it declined. A model trained on funded customers therefore inherits the effects of earlier marketing, application, pricing and approval choices. Extending that model to a different group changes the evidence problem as well as the business opportunity.
The 2021 study Reject inference methods in credit scoring finds no uniformly dominant approach among the methods it examines. The cited 2020 deep-generative research reports results within particular experiments. Neither makes rejected applicants’ unobserved repayment behavior a known fact. The accessible author-paper copy is included with the original journal reference. [1][2][3]
Why an approved-only test can mislead
Suppose an existing policy approves only applicants above a score threshold and with verified income. The training sample contains relatively little evidence about people below that threshold or without that verification. A new model may perform well within the approved sample while extrapolating poorly outside it. Its measured accuracy does not automatically justify a broad expansion of approvals.
Selection can also involve human judgment or information that was not retained in the modeling dataset. If loan officers used an unrecorded document or policy exception, the model developer may not be able to explain why some applicants were financed. Reweighting the observed sample cannot fully repair missing knowledge about the original selection mechanism.
A hypothetical expansion decision
Imagine 10,000 applicants, of whom the old policy finances 6,000. After a year, 300 of those financed borrowers meet the lender’s default definition, a 5% rate. A proposed model identifies 1,000 previously declined applicants for approval. Their default rate on the proposed loan is unknown. Assigning them the approved sample’s 5% rate would assume the very relationship the lender needs to investigate.
Even if some later obtained credit elsewhere, that outcome may involve a different amount, price, maturity or lender policy. It is useful evidence, but not a direct observation of performance on this lender’s intended offer. The hypothetical illustrates why an approval-growth estimate should present uncertainty about the newly included population rather than treat inferred labels as measured repayment.
What common methods assume
Reweighting gives observed borrowers different importance to approximate a target population, but it needs adequate overlap and assumptions about selection. Parceling or assigning likely outcomes to rejected applicants introduces modeled labels. Semi-supervised approaches can use information about the rejected population’s characteristics, while still depending on assumptions linking those characteristics to repayment.
These techniques can be useful tools for sensitivity analysis and model development. The danger is presenting their output as newly discovered ground truth. A model trained on labels produced by an earlier model can repeat the earlier assumptions and appear more certain because the synthetic training set is larger. Additional rows do not necessarily add independent information about outcomes.
Design an honest evaluation
Recommended analysis begins by documenting the population flow: applications, approvals, accepted offers, funded accounts and accounts with sufficiently mature outcomes. Attrition at each step can create another selection mechanism. Preserve the policy and offer terms associated with the observation. A borrower who declined an expensive offer is not equivalent to a borrower denied credit.
Evaluate the existing approved population separately from the proposed expansion group. Use sensitivity ranges for uncertain outcomes and compare alternative assumptions. Where appropriate and permissible under the lender’s policy, a carefully controlled pilot can produce direct evidence from a limited expansion. It should have defined exposure limits, monitoring and a stopping rule; the analytical problem is not solved by indiscriminately funding applicants previously considered unacceptable.
Governance and practical costs
Model documentation should label observed, externally obtained and inferred outcomes distinctly. Validators need access to the assumptions and the effect of changing them. Report whether the apparent benefit comes from a better model among familiar borrowers or from an untested expansion into a new population. Those are different business decisions with different evidence requirements.
The cost of obtaining better evidence may include a slower rollout, additional data, manual review and controlled credit exposure. That cost can be justified when an ambitious growth claim rests on weak extrapolation. Conversely, an excessively narrow demand for certainty can preserve an outdated policy. The goal is to make uncertainty explicit and proportionate to the amount of credit being committed.
Approval, usable credit and customer take-up differ
Analysis: a model may identify more potentially eligible applicants without producing offers they accept or can use. The rate, limit, loan size and repayment schedule help determine both take-up and performance. Measuring only approvals misses whether the expanded group receives a suitable offer, abandons the process or chooses another lender.
For a mortgage lender, dealer or point-of-sale finance provider, the application population also reflects referral and routing arrangements. An applicant rejected by one lender is not a random sample of the market. Compare the complete offer and channel before transferring a result, and avoid treating better repayment on a smaller or differently priced loan as proof about the original rejected offer.
Learning has a cost and a timetable
An expansion decision should identify what observed outcomes could resolve the uncertainty, how long they take to mature and what exposures the business can support while learning. A carefully designed pilot may create evidence, but its terms, eligibility and monitoring need independent review of applicable consumer and fair-lending obligations. Statistical exploration is not a legal exemption.
Keep a separate record of model forecasts, inferred labels and actual customer outcomes. Early payments can inform the view without establishing lifetime performance. Funding, acquisition, servicing and expected losses belong in the business case alongside any modeled approval lift. A forecast that expands access may be promising even when the evidence is incomplete; it should be presented as a testable proposition.
What would make an expansion claim persuasive
The strongest support comes from observed performance in the population and products actually added, evaluated after enough time and against a credible comparison. Improvements that disappear when terms, selection or economic conditions change deserve narrower claims.
Reject inference can help organize uncertainty and guide evidence collection. It cannot turn a simulated approval increase into a measured improvement in financial access or portfolio returns.
Sources
- Reject inference methods in credit scoring; Journal of Applied Statistics, 2021; DOI 10.1080/02664763.2021.1929090SourceBack to text: ↑
- Deep generative models for reject inference in credit scoring; Knowledge-Based Systems, 2020; DOI 10.1016/j.knosys.2020.105758SourceBack to text: ↑
- Author article: Reject inference methods in credit scoring; accessible full text, 2021 articleOfficial sourceBack to text: ↑