This paper was motivated by evaluating real-time forecasts of home-team win probabilities in the NBA. The authors report that ESPN’s published forecasts are generally well-calibrated and that their forecasts show improved skill over several naive models, but not significantly better skill than simple logistic regression models using team strength and score difference.

The authors develop machine‑learning models to predict NBA game winners using player‑type clusters. They report that their best model "achieve[s] a prediction accuracy of ~76% over a period of five NBA seasons and a prediction accuracy of ~71% over a season not used for model training." They explicitly compare to human experts, noting that "our best models outperform human experts on prediction accuracy" and that human subject‑matter experts "have prediction accuracies in the range of 65%–68%." The paper also cites prior work where statistical models match or exceed expert predictions, suggesting algorithmic approaches can at least rival human intuition for game outcomes.

To create the calibration graph we divide the predictions into quintiles, twenty evenly sized groups based on the win probability implied by the model. For each quintile we calculate the average implied win probability and the actual win probability. The graph shows these odds in a scatter plot format. If the odds are well calibrated, teams with a 30%-win probability will win 30% of the time and the points on the calibration chart will align on the diagonal. Figure 1 is the calibration plot for the base Elo model. The model appears to be reasonably well calibrated as none of the points diverge from the diagonal by much. Later we show the calibration chart for the ESPN FPI model. It exhibits a similar pattern to the Elo models. Their models are reasonably accurate, picking the winner as the favorite more than 60% of the time. The models are reasonably well calibrated though it appears that each model is biased in that they tend to underrate underdogs.

This 2026 preprint uses ESPN’s NBA win-probability estimates as input data to construct a Referee Impact Metric over the 2021–2022 through 2024–2025 seasons. The authors treat the ESPN in-game win probabilities as a reasonable proxy for the underlying state of the game, stating that “using ESPN game-summary and win-probability data for NBA seasons 2021–2022 through 2024–2025, we show that RIM is empirically distinct from existing referee metrics.” While the paper does not audit ESPN’s calibration directly, its methodology implicitly relies on ESPN’s probabilities being sufficiently accurate to detect subtle referee effects at the possession level.

In discussing their NBA prediction system, FiveThirtyEight notes that its forecasts are designed to be probabilistic and calibrated: “When we say a team has a 70 percent chance of winning, we mean that in the long run teams in that situation should win about 70 percent of the time.” They explain that they check calibration by grouping predictions into probability buckets and comparing predicted against realized win rates, and report that their NBA and NFL models are generally well‑calibrated over multi‑season samples, though not perfect on small samples or at extreme probabilities.

After this play, ESPN Analytics estimated the Eagles' win probability at 67%, meaning that we would expect the Eagles to win from this game situation in about two-thirds of comparable games. The most paradoxical and counterintuitive result of our study is that high win probabilities are not uncommon among teams that ultimately lose. In both simulated and real NFL games between evenly matched opponents, the losing team reached a win probability of at least 66–67% in half of all cases. This finding does not mean the probabilities are wrong; rather, it shows why human intuition can misinterpret win probability graphs and view blown leads as something unusual.

The post argues that win probabilities should be checked against future outcomes and that reasonable probabilities should match observed game results. It recommends model checking and notes that over-precision and lack of updating can make sports win-probability models misleading.

The author compares an independent NBA win-probability model with ESPN’s model using Brier scores and reports that the ESPN model is only slightly worse on average, with Brier scores of 0.166 for ESPN versus 0.162 for the competing model. The post says ESPN’s model appears too reactive early in games, but still broadly tracks actual outcomes.

This project contains some code for analysis I've been doing on the ESPN college football win probability model. The goal is to scrape ESPN's win probability graphs for college football games, reconstruct the underlying probabilities, and then assess properties such as calibration and sharpness. By comparing predicted win probabilities at various points in games to actual outcomes, we can check whether, for example, situations where ESPN assigns a team a 70% chance of winning do in fact result in wins about 70% of the time.

In probabilistic forecasting, calibration means predicted probabilities match observed frequencies over many cases. A model can be well-calibrated even if it is not the most skillful predictor, and simple logistic models often serve as strong baselines in sports outcome prediction.

In this video, we look at ESPN Analytics' Matchup Predictor tool. We'll be comparing the estimates from the Matchup Predictor to see how they compare to sports book odds and to actual game outcomes over a sample of games. The analysis shows that ESPN's matchup predictor is nowhere close to perfect, but it does seem like there is a possibility that it's able to capture true win probabilities reasonably well in aggregate, even if some individual game predictions look strange to fans.

This video discusses using ESPN BPI ratings to build betting and probability tools, but it is not a study of ESPN NBA win-probability calibration. It is only weakly relevant background and does not provide rigorous evidence about calibration.