What is retrospective validation?
Retrospective validation tests a model against historical data where the real outcomes are already known, scoring past leads as if they were new and comparing predictions to what actually happened.
A retro study takes a partner's historical lead file along with the outcomes — who enrolled, who did not — and scores those leads using only information that would have been available at the time. The resulting predictions are compared against the known results to measure AUC, lift, and calibration.
Done properly it is a strong test, because the outcomes are real rather than simulated. Done carelessly it is misleading, with data leakage the most common failure: if any feature encodes information that only existed after the outcome, the model will appear far better than it is.
Retrospective validation is not the same as a live result. It demonstrates that a model would have ranked past leads well. It does not establish that acting on those scores would have produced the modelled benefit, since changing bidding behaviour changes which leads are won and how they are worked.
How RiskLync uses this term: Performance figures published by RiskLync come from retrospective studies against partner historical data.
Related terms
- AUC (Area Under the Curve) — AUC measures how well a model separates converters from non-converters across all possible thresholds. 0.5 means no better than chance; 1.0 means perfect separation.
- Decile lift — Decile lift compares how often leads in the best-scoring tenth of a population convert versus the worst-scoring tenth. A lift of 10x means the top decile converts ten times as often as the bottom.