Football model backtesting: accuracy is only the start
'Highest accuracy' is not a complete evaluation. A system that labels 51% events as 99% may hit often while remaining a poor probability forecaster.
Updated 2026-06-22
Prevent information leakage first
A backtest may use only data that was available when the prediction was made. Revised rankings, injury reports or odds cannot leak backwards into a historical forecast.
Use metrics designed for probabilities
Alongside accuracy, Brier score and log loss penalise overconfidence. Calibration asks whether events assigned 60% actually occur about 60% of the time over many cases.
Split by time and competition
Train, validate and test by time, then inspect performance by league, probability band and match phase. A single headline score can hide weak regimes.
FAQ
Does high accuracy prove a model is good?
No. Accuracy is sensitive to class balance and thresholds and says little about whether probabilities are calibrated.
What should a transparent backtest disclose?
At minimum: sample period, competition coverage, prediction timestamp, metric definitions, baseline and limitations.