Every week, LOWFI runs on Tuesday morning to make its best assessment of every team in the NFL and then it simulates the remainder of the season 10,000 times to assess where things are headed. It then breaks down every team and shows how it came up with its rating. Those updates will be posted here each week, along with a companion post breaking down the full NFL.

Enjoy, engage, and ask questions!

-Jake

HOW TO READ A TEAM PROFILE

Every profle contains the same information

• The header includes the record, the team’s overall rank by net rating, and how much the rating moved since last game. Under that, both models’ seasons projections: playoff odds, division odds and total wins, from LOWFI/1 in teal and LOWFI/0 inmagenta. Those two colours mean the same two models everywhere else in the profile.

• Scoring is the season to date points for, points against and point differential, each shown a distance from the league average , with the team’s rank beside it. Raw scoring totals are in the subtitle.

• Four ways to read this team is the same team through four lenses, all converted to a projected 17-game record so they share units.

  • Record is the team’s current winning percentage, extended over 17 games.

  • Pythagorean uses points scored and allowed instead of wins and losses using the Pythagorean win expectation model

  • LOWFI/0 is the original model, carrying a full distribution over how good the team actually is rather than a single number.

  • LOWFI/1 is a more sophisticated model: it uses points scored and allowed, adjusted for opponent, on a preseason prior built from the market with non-linear and interaction effecs.

• All 10,000 seasons shows the distribution of win-totals from each model’s simulated universes. The wider the distribution, the more uncertain a model is about a team’s overall strength.

• LOWFI/1 strength is the model’s rating itself: offense, defense and net, in expected points per game against an average opponent, each with a 90% band. Defense is shown as points *prevented*, so up is better on all three. The subtitle carries strength of schedule, in opponent net rating, split between games played and games remaining.

• Where the rating comes from breaks each side of the rating into what the model used to estimate it. Bars run left to right in the order the model accumulates evidence - what it knew before the season, then what the season has told it - and each bar picks up where the last left off. Green helps the rating, orange hurts the rating, and the final bar is the final rating those pieces add up to.

AFC EAST

AFC NORTH

AFC SOUTH

AFC WEST

NFC EAST

NFC NORTH

NFC SOUTH

NFC WEST

Share

Every week, LOWFI runs on Tuesday morning to make its best assessment of every team in the NFL and then it simulates the remainder of the season 10,000 times to assess where things are headed. Those updates will be posted here each week, along with a companion post breaking down each team’s LOWFI assessment.

Enjoy, engage, and ask questions!

-Jake

THE RATINGS

LOWFI/1 rates every team’s offense and defense, and it expresses its assessment of team quality in expected points per game against an average team. The net rating is these two ratings together and can be thought of as how many points this team would be expected to win or lose by against a league-average opponent on a neutral field.

The ratings above are point estimates, but a key aspect of the LOWFI system is that it understands that the NFL is full of uncertainty, and it explicitly models it. This can be thought of as what LOWFI doesn’t know about the team - the uncertainty left in the estimate after this many games - and not a suggestion of how much a team will vary week to week.

As the season progresses, LOWFI/1 updates its estimate of “strength” - as measured in expected net points against an average team on a neutral field - and it tracks that estimate over time.

OFFENSE AND DEFENSE

Up and to the right is good. Defense is plotted as points prevented, so both axes run the same direction.

THE SEASON PROJECTIONS

The tables below show each LOWFI model’s results across 10,000 simulations of the remaining schedule. In each simulation, a team’s rating is drawn from its estimated distribution and held as “true” for that universe. Then the results are counted to determine the probability of various outcomes.

LOWFI/0 works the same way, but it draws a single “true strength” number for the simulation, and the table shows where it agrees and disagrees with LOWFI/1’s more sophisticated model.

Similar to the ratings, the projections tables above are point-estimates, but LOWFI simulates 10,000 seasons - below is the distribution of wins per team across every one of those simulations, according to LOWFI/1

The LOWFI system shares the same foundational concepts and model, but LOWFI/0 and LOWFI/1 are distinct models with distinct opinions - the chart below shows how the models differ in their current projection for each team’s win total.

THIS WEEK

Looking ahead, you can find every game’s distribution of possible outcomes, according to LOWFI/1.

And the distributions, in summary form, and comparing how LOWFI sees the game compared to the Tuesday line.

Reading this table:

• LINE is LOWFI/1’s projected margin next to the market’s, both from the home team’s perspective.

• EDGE is the difference between LOWFI/1’s implied line and the market line.

• WIN PROBABILITY runs all three opinions side by side, LOWFI/1, LOWFI/0, and the market de-vigged implied probability.

• HOME COVER and PUSH are LOWFI/1’s opinion against the posted spread

• STAKE is a quarter Kelly strategy expressed in units per hundred, and reads zero wherever LOWFI/1 doesn’t want to make a bet.

LOWFI/0 and LOWFI/1 not only disagree on the full season, but often on the upcoming week.

EVERY WEEK IS IMPORTANT

The NFL discourse normally begins discussing how important individual games are when division races and playoff seeding takes shape, but every game has the potential to shift a season. This can be expressed through the concept of playoff leverage, and LOWFI/1 uses its 10,000 simulated seasons to produce its own calculation of the importance of a single game on the likely playoff picture.

The solid bar is percentage of simulations in which a team makes the playoffs following a loss this week, the muted bar is the same but following a win, and the gap between them is the leverage itself.

THE SCOREBOARD

The LOWFI system wasn’t built to beat the market - and it usually doesn’t! The comparison gets published every week anyway, through the lens of the probability that the home team wins. The scoring uses the rules from FiveThirtyEight’s old NFL forecasting game1. Each game is worth 25 − 100 × (P − R)² points, where P is the probability the model or the market gave the home team and R is whether it won. A coin flip scores nothing. Being certain and right is worth 25 - being certain and wrong costs 75.

WHAT LOWFI DOESN’T SEE

No play-by-play, no personnel, no injury report. LOWFI reads box scores, so it learns a quarterback is hurt only once the team starts scoring like a team whose quarterback is hurt - in backtesting the 2025 Colts stayed “good” for weeks after Daniel Jones went down. Week 18 is its worst week of the season by a wide margin, for the same reason: the market knows who’s resting and LOWFI doesn’t.

Share

Every week, LOWFI runs on Tuesday morning to make its best assessment of every team in the NFL and then it simulates the remainder of the season 10,000 times to assess where things are headed. It then breaks down every team and shows how it came up with its rating. Those updates will be posted here each week, along with a companion post breaking down the full NFL.

Enjoy, engage, and ask questions!

-Jake

HOW TO READ A TEAM PROFILE

Every profle contains the same information

• The header includes the record, the team’s overall rank by net rating, and how much the rating moved since last game. Under that, both models’ seasons projections: playoff odds, division odds and total wins, from LOWFI/1 in teal and LOWFI/0 inmagenta. Those two colours mean the same two models everywhere else in the profile.

• Scoring is the season to date points for, points against and point differential, each shown a distance from the league average , with the team’s rank beside it. Raw scoring totals are in the subtitle.

• Four ways to read this team is the same team through four lenses, all converted to a projected 17-game record so they share units.

  • Record is the team’s current winning percentage, extended over 17 games.

  • Pythagorean uses points scored and allowed instead of wins and losses using the Pythagorean win expectation model

  • LOWFI/0 is the original model, carrying a full distribution over how good the team actually is rather than a single number.

  • LOWFI/1 is a more sophisticated model: it uses points scored and allowed, adjusted for opponent, on a preseason prior built from the market with non-linear and interaction effecs.

• All 10,000 seasons shows the distribution of win-totals from each model’s simulated universes. The wider the distribution, the more uncertain a model is about a team’s overall strength.

• LOWFI/1 strength is the model’s rating itself: offense, defense and net, in expected points per game against an average opponent, each with a 90% band. Defense is shown as points *prevented*, so up is better on all three. The subtitle carries strength of schedule, in opponent net rating, split between games played and games remaining.

• Where the rating comes from breaks each side of the rating into what the model used to estimate it. Bars run left to right in the order the model accumulates evidence - what it knew before the season, then what the season has told it - and each bar picks up where the last left off. Green helps the rating, orange hurts the rating, and the final bar is the final rating those pieces add up to.

AFC EAST

AFC NORTH

AFC SOUTH

AFC WEST

NFC EAST

NFC NORTH

NFC SOUTH

NFC WEST

Share

Every week, LOWFI runs on Tuesday morning to make its best assessment of every team in the NFL and then it simulates the remainder of the season 10,000 times to assess where things are headed. Those updates will be posted here each week, along with a companion post breaking down each team’s LOWFI assessment.

Enjoy, engage, and ask questions!

-Jake

THE RATINGS

LOWFI/1 rates every team’s offense and defense, and it expresses its assessment of team quality in expected points per game against an average team. The net rating is these two ratings together and can be thought of as how many points this team would be expected to win or lose by against a league-average opponent on a neutral field.

The ratings above are point estimates, but a key aspect of the LOWFI system is that it understands that the NFL is full of uncertainty, and it explicitly models it. This can be thought of as what LOWFI doesn’t know about the team - the uncertainty left in the estimate after this many games - and not a suggestion of how much a team will vary week to week.

OFFENSE AND DEFENSE

Up and to the right is good. Defense is plotted as points prevented, so both axes run the same direction.

THE SEASON PROJECTIONS

The tables below show each LOWFI model’s results across 10,000 simulations of the remaining schedule. In each simulation, a team’s rating is drawn from its estimated distribution and held as “true” for that universe. Then the results are counted to determine the probability of various outcomes.

LOWFI/0 works the same way, but it draws a single “true strength” number for the simulation, and the table shows where it agrees and disagrees with LOWFI/1’s more sophisticated model.

Similar to the ratings, the projections table above are point-estimates, but LOWFI simulates 10,000 seasons - below is the distribution of wins per team across everyone of those simulations, according to LOWFI/1

The LOWFI system shares the same foundational concepts and model, but LOWFI/0 and LOWFI/1 are distinct models with distinct opinions - the chart below shows how the models differ in their current projection for each team’s win total.

THIS WEEK

Looking ahead, you can find every game’s distribution of possible outcomes, according to LOWFI/1.

And the distributions, in summary form, and comparing how LOWFI sees the game compared to the Tuesday line.

Reading this table:

• LINE is LOWFI/1’s projected margin next to the market’s, both from the home team’s perspective.

• EDGE is the difference between LOWFI/1’s implied line and the market line.

• WIN PROBABILITY runs all three opinions side by side, LOWFI/1, LOWFI/0, and the market de-vigged implied probability.

• HOME COVER and PUSH are LOWFI/1’s opinion against the posted spread

• STAKE is a quarter Kelly strategy expressed in units per hundred, and reads zero wherever LOWFI/1 doesn’t want to make a bet.

LOWFI/0 and LOWFI/1 not only disagree on the full season, but often on the upcoming week.

EVERY WEEK IS IMPORTANT

The NFL discourse normally begins discussing how important individual games are when division races and playoff seeding takes shape, but every game has the potential to shift a season. This can be expressed through the concept of playoff leverage, and LOWFI/1 uses its 10,000 simulated seasons to produce its own calculation of the importance of a single game on the likely playoff picture.

The solid bar is percentage of simulations in which a team makes the playoffs following a loss this week, the muted bar is the same but following a win, and the gap between them is the leverage itself.

WHAT LOWFI DOESN’T SEE

No play-by-play, no personnel, no injury report. LOWFI reads box scores, so it learns a quarterback is hurt only once the team starts scoring like a team whose quarterback is hurt - in backtesting the 2025 Colts stayed “good” for weeks after Daniel Jones went down. Week 18 is its worst week of the season by a wide margin, for the same reason: the market knows who’s resting and LOWFI doesn’t.

Share

LOWFI/1 is the advanced Latent Outlook Weekly Forecasting Interval model. It estimates separate offensive and defensive ratings for every team, in points per game against an average opponent, from scoring trends and a preseason market prior, but is never explicitly told if a team is winning or losing.

This page describes how it works. For why it exists, see the introduction.

Subscribe now

Stage 1: Target

A team scoring 30 per game either has a strong offense or has faced weak defenses. The box score does not distinguish them, so the two effects are modeled using a crossed random-effects model fitted across every game since 2004.

The response is points scored in a team-game. Random effects for the scoring offense and the conceding defense, crossed rather than nested, plus a game-level effect. Fixed effects control for home field, week, division games and the league’s scoring environment that season. Fitted via REML.

Two important conclusions from the fitting process. Offensive identity has approximately 2.2x the separable variance of defensive identity, so defensive ratings are estimated less precisely across the board. And residual variance of 81 means individual games are noisy, even if both teams’ true ratings are known.

BLUPs are extracted per team-season and centered within season. These are the prediction target: an end-of-season +/- points rating, which is not known during the season.

Stage 2: Preseason prior

LOWFI/1 “seeds” itself for the season with a prior that is built around betting markets and prior season performance.

First, each team’s Vegas win totals and over/under prices are used to construct an implied net rating (i.e. how many points a team is expected to win or lose by against an average team, per game).

This is done by first “de-vigging” the Vegas totals. The totals plus the over/under prices produce implied probabilities, which are then normalized to remove the bookmaker’s margin, and the residual imbalance is converted to a shift in the total:

adj_total = line + sd_wins · Φ⁻¹(p_over)

sd_wins is estimated from the historical dispersion of actual wins around projected totals. A 9.5 at −150/+125 and a 9.5 at +125/−150 are implied to be fairly different teams; the adjustment is worth roughly a third of a win at the extremes.

Once the win totals are adjusted, they are converted to an implied winning percentage (adj_total / games ), which is inverted through a fitted win_pct ~ point_differential relationship to give the expected point differential (i.e. net rating) per game.

This gets close to a preseason prior, but LOWFI/1 tries to estimate offensive and defensive rating separately, so it behooves the model to have the net rating implied by the market split into these priors.

To do so, a ridge regression is used to predict each team’s eventual end-of-season offensive and defensive rating from the market-implied net rating plus their previous season’s offensive, defensive and net ratings. This effectively splits the market into historically accurate offensive and defensive decomposition.

The market determines net strength - prior season information contributes essentially nothing to it. The offense/defense split is entirely the prior season’s contribution.

Net prior correlation with realized strength is 0.518, so the preseason picture accounts for about 27% of variance in end-of-season rating.

Stage 3: In-season estimation

In season “best-guess” true offensive and defensive strengths are estimated using generalized additive models (GAMs), with smoothed terms derived from:

• preseason prior (own side and net)

• previous season’s ratings (own side and net)

• points scored and allowed to date, league-centered

• exponentially weighted recent scoring (ρ = 0.9)

• opponent quality faced: season-to-date average of opponent ratings, and of opponent preseason priors

• tensor interactions of week with the prior, the previous season, and current scoring

In-season terms carry a “has played games” indicator so they contribute nothing before games exist. Observations are weighted so early weeks dominate fitting - equal weighting would give late-season observations (which are near-complete) too much credit and drive a poor fit.

Week 1 uses separate models due to only having priors - fitting a GAM to the full season would lead to pooling across all eighteen weeks, and that would weight the preseason prior by its average usefulness across a season in which in-season scoring carries most of the signal. Measured out of fold, the pooled model was worse at week 1 than the raw prior it was given. The preseason models are therefore fitted only on observations with no games played and uses only four preseason predictors (market estimates and prior season ratings).

Predicted ratings are rescaled per week using coefficients from 5-fold season-blocked cross-validation, correcting systematic over-shrinkage. Post-calibration dispersion ratios run 0.98 at week 1 to 1.00 by week 12.

Standard errors are inflated by a factor decaying in games played - i.e. the ratings should become more certain as more games are accrued. The inflation value is fitted so that simulated win-total dispersion matches the observed dispersion of actual wins around preseason market lines (2.77 against 2.76).

Stage 4: Predicting games

Each game gets an expected spread that is derived from a simple OLS regression on observed home margin of victory using four ratings:

result ~ home_offensive_rating + home_defensive_rating + away_offensive_rating + away_defensive_rating

The intercept of the fitted model is the home field advantage - 2.26 in points. Residual standard error 13.46.

But the spread is not the same thing as the expected distribution of margin of victory - that distribution actually follows from the spread itself, and is not normally distributed around it (for example, margins of victory of +/- 3 or 7 are far more likely than a normal distribution would expect, given the weird point system of football). Instead, an ordered probit over integer margins from −49 to +59 is used to define the distribution, and the spread shifts the location within the distribution

he pmf is precomputed across a grid of spreads and sampled by inverse CDF during simulation to get expected MOV, which is what allows ten thousand seasons to run in seconds.

Stage 5: Simulation

10,000 simulations via nflseedR, with full seeding and tiebreaker rules. Each simulation draws one rating per team from its uncertainty band and holds it for the season, then samples each game from the margin distribution.

Holding the draw is required: redrawing weekly would average out rating uncertainty within each simulated season and understate win-total spread.

Held-out performance

Fitted through 2022, evaluated on 2023–2025 (816 games, unseen):

Correlation with the closing line: 0.85. Mean absolute difference: 2.47 points.

LOWFI/1’s reliability is better than the market’s, but resolution is 29% worse. The model is well calibrated, but insufficiently bold.

When examining randomized PIT across held-out games, the result is flat by decile (9.1, 9.7, 10.3, 11.3, 11.4, 10.3, 10.5, 8.5, 8.8, 10.0 against 10 expected), with mean 0.495 and standard deviation 0.282 against 0.289 for a uniform. The margin distributions are well calibrated.

When regressing actual margin on predicted margin out of sample, the slope is 1.02 (se 0.082), consistent with 1 (which is what’s expected for a good margin predictor).

Known limitations

Week 18: Log loss 0.684 against the market’s 0.586. Seeding-driven rest is unobservable from final scores, and so the model struggles to make great predictions.

Preseason compression: Projected win totals are approximately 20% too tightly dispersed at week 0, decaying to 7% by week 8. Costs roughly 1% of accuracy and primarily means there isn’t enough diversity of predicted records early.

Fixed margin shape: The distribution’s location moves with the spread but its scale does not, so equal spreads in high- and low-scoring games receive identical distributions. A location-scale ordered probit conditioning scale on projected total was tested and produced no improvement (CRPS change −0.02%, p = 0.45) because the total model’s R² is 0.053 — the scale covariate is effectively noise.

Neutral sites: International and neutral-site games receive full home field advantage. Approximately eight games per season.

Defensive precision: Defensive ratings are less precisely estimated than offensive ones in every week of every season, following from the variance decomposition in Stage 1.

LOWFI/0 is the original Latent Outlook Weekly Forecasting Interval model. It estimates one number per team and knows nothing else: not the score, not the opponent’s quality, not the margin, not even if a team has been winning or losing.

This page describes how it works. For why it exists, see the introduction.

Definitions

True strength (p): a team’s long-run probability of beating an average opponent on a neutral field. Conceptually, the winning percentage the team would settle at over an unbounded number of games.

p is never observed. LOWFI/0 estimates it via a Beta distribution. Beta is the conjugate prior for a binomial process and is supported on [0,1]; usefully, the sum of its two shape parameters (α + β) is interpretable directly as a number of games of evidence and as a point estimate (α/(α + β) = expected true strength).

Subscribe now

Preseason prior

LOWFI/0 is intentionally low-fidelity, so it has no external information to seed itself with - no market prior, no roster data - so it uses last season’s results in two ways:

Carryover: Each team’s season win rate is regressed on its own previous season across all available seasons. The fitted slope is 0.340 - this implies that a 12-win team should be expected to roughly win 9.5 games the following season, without any other information.

Prior width: Observed variance in season win rate is the sum of true talent variance and binomial sampling variance. The sampling component, p(1−p)/n, is subtracted before solving for the Beta parameters by method of moments:

σ²_talent = σ²_observed − E[p(1−p)/n]

This yields a true talent standard deviation of 0.1472, implying a prior concentration of 10.5 pseudo-games. Omitting the correction produces a prior roughly twice as wide, having treated seventeen games of sampling noise as ability.

These two metrics generate the preseason distribution around estimated true strength.

Game probabilities

For teams A and B with strengths p_A and p_B, the head-to-head probability is the log5 formula:

P(A beats B) = (p_A − p_A·p_B) / (p_A + p_B − 2·p_A·p_B)

Important properties of log5: returns p_A when p_B = 0.5; symmetric; multiplicative in odds space, so equal rating gaps at the extremes imply larger probability gaps than equal gaps near the mean.

Home field is applied separately and is fitted from a rolling history of home margin. Current estimate is 2.26 points, having moved across eras: 2.53 (2004–2015), 1.78 (2016–2019), 0.91 (2020–2021), 2.15 (2022–present). This estimate is applied in the margin mapping detailed below

In-season updating

The default update rule is rejection sampling (approximate Bayesian computation). Each week, per team:

1. Draw n = 4,000 candidate strengths from the team’s current estimated Beta distribution.

2. Draw the opponent’s strengths from theirs.

3. Compute log5 win probability for each paired draw; sample a Bernoulli outcome from that derived probability.

4. Retain only the strength draws whose simulated outcome matches the actual result.

5. Refit a Beta distribution to the retained draws by method of moments.

Margin mapping

LOWFI/0 does not model expected margin of victory. For simulation purposes and for comparison against a spread, its win probability is mapped to an expected margin by OLS regression of observed home margin on the log5 logit, fitted on training seasons only. This is a calibration mapping, not a forecast of points, and LOWFI/0 is excluded from margin-accuracy evaluation for that reason.

Season simulation

10,000 simulations per run via nflseedR, which implements NFL seeding and tiebreaker rules including divisional and conference records, common games, strength of victory and strength of schedule.

Each simulation draws one strength per team and holds it for the full season. Redrawing weekly would average the uncertainty about team quality within each simulated season and produce win-total distributions that are too narrow.

Projected wins, division odds, playoff odds and draft position are then calculated by counting up how many times different outcomes occur.

Held-out performance

Fitted through 2022, evaluated on 2023–2025 (816 games, unseen):

Log loss reference points on the same games: closing line 0.610, LOWFI/1 0.632, point-differential SRS 0.680, home field only 0.690. A constant 50% forecast scores 0.693.

By phase of season: weeks 1–6, 0.671; weeks 7–13, 0.616; weeks 14–18, 0.640.

Known limitations

No strength of schedule. Opponent quality is not an input. Early-season ratings are therefore biased in a predictable direction while schedules are unbalanced.

No margins. A one-point win and a thirty-point win are the same observation.

Week 18. Log loss deteriorates to 0.718 in week 18 against the market’s 0.586. Seeding-driven rest is unobservable to a results-only model.

Beta refit is a bit of a hack. The true ABC posterior after several weeks is not exactly Beta; forcing it back to one each week discards some shape. Not currently quantified.

See the detailed methodologies for LOWFI/0 and LOWFI/1

Back in 2018 or so, I was really getting into statistical programming with R, partly for work (I hated writing SQL), and mostly because the nflscrapR package went live. This beautiful little API gave analysts unprecedented access to NFL data, and public analytics space would never be the same. I wanted to get in on the action, and so I started exploring the data, joining in with a growing community of like-minded nerds in building analyses and visualizations and occasionally finding interesting insights.

But as the analyses and models proliferated (often with a bent on betting), I became obsessed with an idea - how well could I project an NFL season…with very little information? Like, what if I never told the model if a team was winning or losing?

And so, I set out to test this idea - first building a relatively simple model before tinkering a tweaking it to become more advanced, but always keeping that constraint in mind - the model(s) would never know if a team was winning or losing during the season.

I have been building some version of this model for about eight years. And while I’ve published bits and pieces about it on the internet (mostly on the current hellscape formerly known as Twitter), this is the first time I am fully publishing it. With a name and everything!

And so today I present to the internet a new NFL projection system - LOWFI!

LOWFI stands for Latent Outlook Weekly Forecasting Interval, and is an NFL rating and projection system that runs on data that can be found in box scores - points scored, points against, who’s playing whom, and preseason betting win totals. That’s it - no play-by-play data, no personnel, no injury reports, no passing information, no wins or losses.

The constraint is the point, and I will get to why. But first, how LOWFI came to be.

Subscribe now

Where it came from

The first version was built in 2018, and it barely knew anything. I wanted to see how accurately I could project a season if the model is never actually told the score.

So I gave every team a distribution of “true” strength1, drew thousands of possible values from those distributions, “played” the season out with those draws, and then - this is the part I still like - threw away every simulation where the wrong team won. The model was never told anyone had won a game. It just stopped carrying the versions of the world where they hadn’t. What was left over became the basis for the new distribution.2

It worked better than I ever expected. The model would pick up on trends like injuries and poor play without ever observing them, and did a pretty good job of getting to final win-totals pretty early in the season.

Over time, I wanted to add some concepts to the model - specifically, the original model only produced win probabilities in individual games, and I wanted to produce margin of victory predictions. So the next major version added points. That’s roughly what LOWFI is today, though almost every line of it has been rewritten a few times, with better exploration, visualization, and evaluation techniques, and now the system is fully automated. Both versions of the models exist, and they have now been christened LOWFI/0 (the (mostly) original strength model, and LOWFI/1 (the more advanced model that uses points and projects offensive and defensive strengths)

Why keep the constraint?

For two reasons.

The first one: nobody publishes what the extra information is worth. Nearly every published model uses more data than the last one, and every model reports that it is good. Almost none of them report what happens if you take major inputs away. I wanted a system where that comparison is inherent in the system - so LOWFI/0 knows only universes that agree with reality (though it doesn’t know what reality happened, at least explicitly), LOWFI/1 adds points, and both run every week, side by side, scored against the same games.

The second one: because I wanted to! The whole genesis of this side project was to see how well I could do with limited information, and that remains the goal. I have an idea of a more advanced, play-by-play derived system3, but I haven’t found the time to fully build it yet. Maybe next year!

What LOWFI actually is

Two models, run in parallel every week.

LOWFI/0 knows almost nothing. Each team is represented as a single probability (i.e. true strength) of beating an average opponent, carried as a distribution rather than a number, updated by discarding the simulations that disagree with reality.

LOWFI/1 knows points scored and points allowed and adjusts them for opponent4. Each team gets an offensive rating and a defensive rating, both in points per game against average.

Both then simulate the rest of the season ten thousand times, through the NFL’s actual seeding and tiebreaker rules5, which is where projected wins and playoff odds come from.

The system is called Latent Outlook Weekly Forecasting Interval because the things it estimates - true strength, offensive and defensive rating - are inferred rather than observed, re-estimated every week, and published as an interval. It is also low fidelity, at least relative to models like DVOA or FPI, though that doesn’t mean it’s bad.

How good is it?

The models are fitted on every completed season through 2025. To test them, I fitted a separate version that stopped at 2022 and scored it on 2023 through 2025 - 816 games it had never seen. Those are the numbers below:

Log loss is a standard measure for probability forecasts and lower is better; a model that says 50% on everything scores 0.693.

LOWFI/1 does not beat the market (and is not designed to). It finishes 0.022 behind the closing line and correlates with it at 0.85. I will publish the comparison anyway, because it is the only external benchmark that matters, and because a model that avoids being scored against the market is telling on itself.

The gap to Vegas is not calibration. Breaking the score into its components, LOWFI/1’s reliability - whether a stated 40% win probability happens 40% of the time - is actually better calibrated than the closing line. What the market has is resolution: it commits harder (LOWFI/1 might say 60% chance to win, the market implied 72%, etc). Whatever it knows that this model doesn’t shows up as confidence, not necessarily as correctness.

Surprisingly, LOWFI/0 is right there. Just a nat’s hair behind6, and slightly ahead on raw accuracy. The gap is real - I tested it statistically - but it is small, and it lives in calibration rather than in picking winners.

What to expect out of LOWFI

I’ve automated LOWFI to run every Tuesday morning. My goal will be two posts per week derived from that run.

The first is about the league and that week’s games. Updated ratings with their uncertainty, projections and playoff odds by division, this week’s games with the model’s line beside the market’s, and how far each team’s week moved their season. This is where you will come to see what the model thinks about the next week and the rest of the season.

The second will be thirty-two team cards, one per team, starting after Week 1. The cards will profile each team - current record and scoring, four different reads on how good a team is (their current record, their Pythagorean expectation, LOWFI/0 and LOWFI/1, all converted to a projected 17-game record), and a breakdown of how LOWFI/1 derives the offensive and defensive ratings.

Both come from one system, and every number is written to disk before a single chart is generated. Betting lines are snapshotted before kickoff, every week, whether the model has an opinion or not. If I ever claim to have beaten a number, the number will be on file from before the game.

Where LOWFI is right now

Preseason 2026, which means these ratings are the market’s opinion split into offense and defense using last season’s finish, and not much else. Twenty-seven percent of the variance in where teams actually end up, give or take.

The Rams, Seahawks and Lions open at the top. Miami, Las Vegas and Arizona open at the bottom. Top to bottom the whole league spans about seven points a game, and every single interval overlaps with most of the others. That is not the model hedging. Seventeen games is a very small sample, and in September nobody knows anything.

Baltimore actually opens with the best Super Bowl odds, though they won in less than 8% of simulations. Miami opens with the best odds at getting next season’s number one pick, with just about 11% of simulations ending with them in last place.

LOWFI/1 (and LOWFI/0, too, though I don’t plan on publishing all that detail every week!) also produces per game forecasts, with margin of victory distributions. Check out week 1.

LOWFI isn’t perfect

A projection system built on somewhat limited data is gonna have some flaws. Here are some of the major ones out of LOWFI.

Week 18 is bad. Once seeding is settled, good teams might rest starters. The market knows who is sitting. A model built on final scores has no idea, and week 18 is by a wide margin its worst week of the season.

Preseason projections are too tightly bunched, by about 20%. Teams sit closer to 8.5 wins than they should. It costs roughly a percent of accuracy and mostly affects how things look.

The preseason rating is the market’s opinion, not an independent one. At week 1, LOWFI/1 is largely restating a betting line. It stops being true fairly quickly, but it is true in September.

Defense is measured worse than offense, in every week of every season. Offensive identity carries about 2.2 times the separable signal. That is a property of football rather than a fixable flaw, and it is why every defensive rating here comes with a wider band.

Personnel is unknown, and the model only picks up on injuries, suspensions, and benching based on changes in the underlying performance of the team, rather than by knowing a QB has gone down (in backtesting, the 2025 Colts remained “good” in the model for a few weeks after Daniel Jones got injured, despite being terrible to watch).

What I’ll be watching

I know how this model performs on 2023 through 2025, because I held those seasons out and never touched them. I do not know how it performs on 2026, and neither does anyone else!

What I want to find out, in public, with the numbers filed in advance:

Does knowing the score keep mattering, or does the gap close? Does LOWFI/0 stay within a hundredth? Does the compression I can measure in the preseason actually cost anything by November? Ask me in February.

-Jake

Share

“Were the Seattle Seahawks the best team in football last season?”

A simple question, and one that you might answer simply: Yes - they won the Super Bowl!1

But how certain are you in that answer? After all, plenty of teams that win the Super Bowl don’t have the best record.2 In fact, the team with the best record has only won the Super Bowl 6 times out of the last 16 seasons…so what gives?

Beyond the fact that the goal of (most) NFL teams is not to have the best record but to win the Super Bowl, the 38% conversion rate from best record to SB winner highlights one of the most distinctive features of American Football - small sample sizes.

The NFL is unique in that teams play only 17 games, which might seem like a lot to a rabid fan, but it’s a statistically small sample. How confident can you be in that a theoretical 10-7 team truly represents a 59% winning percentage?

Not very! All samples, regardless of size, are subject to noise. A team's "true" win percentage3 isn't directly observable. What’s observed is a record, and a record is one presentation of an underlying process that would look at least a little different if you could rerun the season. The gap between what you saw and what's true is exactly what statistical analysis exists to quantify.

The simplest way to put a number on that gap is the binomial test. If each game is treated as an independent “trial” with some fixed but unknown probability of winning, then asking "what's the true win rate behind a 10-7 record?" is equivalent to asking "what set of underlying probabilities would make 10 wins in 17 tries a realistic outcome?" That's a question that can be answered mathematically - a 95% confidence interval derived from the binomial test will range from roughly 33% to 82%4. That's not a typo! The same 17-game record that looks like a 59% winning team is, statistically, consistent with a team that wins 5-6 games…or one that wins 13-14. The NFL's small sample problem, quantified.

But, wait, you say5, the games aren’t exactly independent. Besides, you have seen lots of NFL seasons before - the binomial test’s flaw is that it doesn’t know that a 5 win season and a 14 win season have some prior probability that can be estimated, just by looking at the distribution of winning percentage across the NFL

Why throw out that knowledge? You don’t need to - that's the job of the Beta-Binomial model. You have the same 17 games, the same 10 wins, but now the historical distribution of NFL win percentages acts as a prior6, guiding the uncertainty estimate back to reality, and tightening the confidence intervals to something more realistic - in this case, 37% and 75%, or a 6-11 to 12-13 win team. Not a huge difference, but it’s getting closer to truth. This is how each method looks across each possible record78

But statistical models are maybe too…theoretical. What about a different example? One drawn from sports with far more information - in fact, where teams play dozens of “NFL seasons” per year.

Subscribe now

The MLB plays 162 games per year, the NBA and NHL each play 82. Therefore, each team in each league plays a series of NFL length seasons - 17 consecutive game stretches that overlap. Games 1-17, 2-18, 3-19 and so forth,9 and ah screw it, HIT ME WITH THE VISUALIZATION!

Sorry, got excited there, back to the narrative10

Each of these seasons - let’s call them “NFL slices” - contains information that might help illuminate just how informative - or uninformative - a single NFL season is as it pertains to what a good record actually represents, all without resorting to complicated mathematics or statistics. For example, we can look at how each league winner’s best, worst, and average NFL slice looked, or how a Beta-Binomial model fit with another league’s NFL slices changes uncertainty.

But first, let’s start with the basics - how are wins typically distributed across these slices?

The general shape of an NFL slice across leagues is similar - wins are normally distributed11, centered around 9 wins. But there are some nuances - the MLB and NHL are noticeably skinnier, meaning that there is less team x NFL slice variance, while the NBA’s distribution is a close match to the NFLs. Said another way, an NBA team has roughly the same likelihood of going 13-4 over any given NFL slice as any NFL team12, whereas an MLB or NHL team is 40% less likely. Conversely, those latter two leagues are far more likely to go 10-7 (roughly 30% more likely). Skinnier distributions are also taller!

The shape similarity isn’t just a fun fact - it’s a statistical permission slip. If NBA, MLB, and NHL teams playing 17-game stretches produce win distributions that look roughly like what NFL teams produce over a full season, then those other leagues’ slice distributions are reasonable priors for the same uncertainty question that beta-binomial model answered above.

The “skinnier” distributions matter too - an MLB or NHL derived prior can be considered a more confident prior, since it already “thinks” extreme records are unlikely, and so therefore will pull estimates closer to the middle. So 3 more leagues, 3 alternatives on measuring uncertainty around a win-loss record. Yo, check out the pretty graphics:

These charts show how much more aggressive an MLB or NHL derived prior pulls “true” estimated winning percentage towards the mean13 and how much narrower (i.e. how much more confident) the intervals are.

The idea of confidence in uncertainty can be tested in a different way - since each team in other leagues plays many NFL slices per season, you can examine how a record in any given NFL slice compares to the team’s final winning percentage.

Wins in any given NFL slice are correlated with a team’s final winning percentage - bad teams are more likely to have an NFL slice with 1 win than good teams - but the correlation is weaker than you might expect.14 Sometimes, teams that had 1-win NFL slices finish above .50015, and teams going 15-2 in a slice may end with a sub .400 winning percentage.16 Now, how certain are you that the Steelers’ 10-7 record was really playoff worthy?17

As illustrated above, small samples aren’t worthless - they just carry uncertainty along for the ride. For example, if you isolate a team’s worst NFL slice, you likely can tell something meaningful about its’ best NFL slice:

In the NBA, the correlation is strong (.79). It’s still fairly strong in the NHL (.62), but is much weaker in the MLB (.45). What’s more interesting - at least to me - is that the best and worst NFL slice carry similar amounts of information about a team’s final winning percentage. Outside of some minor rounding, the best and worst NFL slice are equally (and strongly) correlated with the final win percentage across all leagues (.76 - MLB, .92 - NBA, .85 - NHL).

Okay, I promised that using the other leagues would be less statistically theoretical, and here I am talking correlation again. Another, less technical way to examine how much or little an NFL season tells us is to examine other leagues for how likely an “historic” run is - how likely is it that a team will go 15-2 during a season?

In the NFL, going 15-2 or better is a historically great season - it’s happened exactly twice since 2021. But if other leagues’ teams are playing dozens of NFL slices per season, how often should you expect to see a 15-2 or better stretch from any given team? The answer makes you rethink what “historic” actually means when your sample is small enough.

In the NBA, you should expect each team to roughly have one 15-2 or better NFL slice per season!

What if instead of looking at records across slices, you just ran the playoffs for each one? Take every NFL slice in a league season, roughly apply the NFL’s 14 team playoff structure to that slice’s standings, and count how many unique teams would have punched a ticket at some point during the year. If records were stable and meaningful, you’d expect roughly the same 14 teams every time. If they’re noisy, the field should look very different slice to slice. Let’s see how these “phantom playoffs” net out:

Yeah, there are 30 total teams in the MLB, and they all make the Phantom Playoffs, on average.

All of this analysis keeps pointing to a straightforward conclusion - an NFL season isn’t long enough to tell us much about the teams, and if you assume that an NFL team has similar variance to teams in other major sports, well, then the NFL season tells us very little indeed about who is “best”.

Taking this idea to its logical conclusion means there is only one thing left to do - simulate an NFL season, as if it were much longer! As an example, you can examine the 2025 Seattle Seahawks - Super Bowl champs! - and pretend like their 14-3 season was just one NFL slice in a much longer season. Earlier in the analysis, I developed a few models for what uncertainty can be drawn around an NFL Slice, and they can be used now - the Beta-Binomial model, and the observed final winning percentage based on a slice from each league. Do some math, dial up 10,000 simulated universes, and voilà - your 2025 Seahawks, but if they played18 in every other league:

Across all leagues, the Seahawks aren’t quite as good as their NFL slice suggests - finishing with an average win percentage of 70.6% according to the Beta-Binomial simulations and 64% according to the empirical model. That’s a 12-win and 10-win season, respectively. A fine-to-good outcome, but unlikely to be the best in the league.

But this can be taken further by asking “if the Seahawks played in these other leagues, how likely would it be for them to win the championship?” And this can be tested by training a series of simple models that use winning percentage as the lone predictor for likelihood to win the championship19. Pass the simulated final season “true” winning percentage, and now you’re talking - a distribution of simulated universes, and the likelihood that the Seahawk’s win that leagues championship

If you assume these20 simulations contain some truth, then the Seahawks only have somewhere between a 14% (empirical model) and 37% (Beta-Binomial) chance of winning a league’s championship, on average. Small samples, man.

So - were the Seattle Seahawks the best team in football last season? Maybe. Probably, even. But “probably” is doing a lot of work in that sentence, and it’s information that a win-loss record simply doesn't provide. The analysis above isn’t an argument that records are meaningless - they’re not, and the correlations show that clearly enough. It’s an argument that they’re noisier than they look, that the gap between “best record” and “best team” is wider than fans tend to assume, and that the NFL’s short season makes that gap almost impossible to close with wins and losses alone. The Seahawks earned their championship. Whether they were the best team in football is a question the schedule simply didn’t give us enough games to answer - maybe they were, maybe they weren’t, but analyzing their record alone won’t tell you that.

Thanks for taking the time

-Jake

Leave a comment

Share Jake Davis Analytics

Sources

NFL data from nflreadr package, all games excluding ties from 2021-2025

MLB data from baseballr package, all games excluding ties from 2002-2025

NBA data from hoopR package, all games excluding ties from 2002-2025

NHL data from hockeyR package, all games excluding ties from 2002-2025

A few weeks ago, during the NFL Wildcard matchup between the San Franscisco 49ers and the Philadelphia Eagles, Jauan Jennings threw a beautiful touchdown pass to Christian McCaffrey down the right sideline - Jennings hit him in stride while sprinting to his right, lofting a fantastic ball for the easy touchdown.

The only weird thing about the pass? Jennings isn’t a quarterback. He’s a wide receiver! And it’s not the first time he made a great throw in a high leverage situation - he hit McCaffrey for a touchdown in the freakin’ Super Bowl (a game in which he also caught a TD). That throw wasn’t quite so impressive, physically, but he looked off defenders, waited, and then delivered a strike on a perfectly set up cross-field screen.

@nfl#jauanjennings throwing dimes out there 🎯 #christianmccaffrey #sanfrancisco49ers #nfl @San Francisco 49ers

Tiktok failed to load.

Enable 3rd party cookies or use another browser

Watching Jennings’ throw made me think of Antwaan Randle-El’s 43 yard Super Bowl XL TD pass to Hines Ward (big yinzer over here #HereWeGo).

Which of course led me to the only logical end place - who is the greatest non-QB passer of all time?

When a non-QB tosses the pigskin, three things can happen, and two of them are bad. So using this as a baseline, I refined my question to something far simpler (and probably dumber, at least in terms of actual football value). What non-QB was the best at completing passes? And that rabbit hole of course took me to the inevitable question - who is the greatest completer of passes in NFL history?

“Jake, of Jake Davis Analytics, this is an easy question to answer” you might say. And maybe you’re right!1. We can just simply look at players that have the highest completion percentage of all time - obviously, the more passes you complete relative to your attempts tells us if you’re good at completing passes!

Okay, so let’s do this. I collected box score statistics for every game since 1950 for every player that attempted at least one pass, and then build a series of databases to summarize every game, every season, and then the career of all players that attempted at least one pass in NFL2 history. I calculated completion percentage as the number of completions divided by the number of attempts, and viola, we have our answer.

Boom - Bill Donckers and Cedric Wilson, Jr. - we’re done here.

But wait, you shout into the void3 - this doesn’t make any sense! How can Bill Donckers, a two year backup from the late 1970s, and Cedric Wilson, Jr, a spot-starting WR with 7 years in the league and mediocre receiving statistics, be the greatest completer of passes of all time? They’ve barely thrown the ball!

You’re right, I whisper. This doesn’t make any sense4, at least in the spirit of the question. After all, 249 players in my database have a 100% lifetime completion rate (77% of those players attempting only a single pass). So let’s turn to another method, and place a minimum attempts filter on the list. I’ll use 1,500, in accordance with Pro Football Reference’s database

Oh look at that, Joe Burrow (and Tua? and KYLER?? lol cute). There are some reasomable QBs on this list (Burrow, Brees, Dak, Mahomes), but it certainly feels a little…let’s call it recency bias-y. And I doubt anyone who’s watched Mac Jones play think he’s the 10th greatest completer of passes of all time! Hell, all but Brees are actively in the NFL, right now. Moreover, the crude instrument of this filter has eliminated the ability to answer the original motivating question regarding non-QB passers - running backs don’t throw the ball 1,500 times in their career.

Okay, so let’s look at this a different way - let’s examine the relationship between passign attempts and completion percentage, since the 1,500 threshold feels a bit arbitrary

it may seem counter-intuitive, but the relationship between attempts and completion percentage is postive5, which is only surprising because volume and rates tend to be mildly negatively correlated. But in this case, it makes perfect sense - if you’re bad at completing passes, you probably don’t get to have many passing attempts. So there is some selection bias at play here, which will make drawing a conclusion on the greatest completer of passes trickier than initially thought. One other thing to notice from the chart above is that the spread of dots doens’t seem to noticeably collapse at 1,500 attempts. So we might as well see how the greatest completer, in terms of raw percentage, changes based on different attempts thresholds.

Jake Browning, of all people, may be the greatest completer of passes…if you randomly choose a 315-attempt wide window6. While interesting, this breakdown doesn’t really tell us much new - the top completion percentage based on a minimum number of attempts tends to hover around 68% - and tends to be biased towards recent passers.

When completion percentage is examined longitudinally, an obvious and steep slope is apparant7 , growing from from sub-50% completion percentage in 1950 to a league average 64% rate in 2025.

But does this inherently mean modern day passers are better than the throwers of yesteryear? Not necessarily! For one thing, professional football has evolved dramatically over the decades - a game in 2020, while recognizably the same game as one played in 1970, is still a radically different form of the game. Passers are attempting (and completing) far more passes per game8, but those passes are going for far fewer yards9. While my dataset doesn’t include air yards (how far the ball traveled in the air), yards per completion is a fair proxy. And the decline suggests modern passers are attempting easier throws - perhaps due to scheme, perhaps due to better decision-making or some other factor - than older passers.

Moreover, modern passers have a very different training environment - Juan Jennings, for example, threw 377 passes in high school (222 in his senior year), compared to someone like Bart Starr, who threw 122 passes as a senior as a highly touted recruit. These dynamics play10 out in the form of a “rising tide lifts all ships” kind of way, as evidenced by the shrinking distance between the best and worst passers in any given season as time goes on. To illustrate this, on the top is a chart that shows the spread of completion percentage by season, and on the bottom is the summarized coefficient of variation11 of each season’s completion percentage.

If the chart above wasn’t clear enough, the CV chart makes it obvious - the standard “spread” of season-long completion percentage amongst QBs is shrinking. Stated in another way - the distance between good and bad QBs is shrinking over time.

So the motivating question of this post - who is the greatest completer of passes of all time - sure has gotten complicated. Position, the number of attempts, and the impacts of era all make what should be a simple counting and dividing exercise anything but. Thankfully, there is a tool for this - statistical modeling.

A statistical model is a useful mechanism for answering questions that appear simple on the surface but contain a myriad of complexities. To help answer this specific question, I fit a hierarchical statistical model12 meant to decompose the impact of postion, era, evidence in the form of attempts, and individual player impacts.

Philosophically, the model is designed to help make sense of a straightforward question: When a quarterback completes 75% of his passes over 50 attempts, should we believe he’s truly a 75% passer? Probably not. Small samples are noisy, and results tend to drift towards average as more data is gathered - a phenomenon called regression to the mean. Hierarchical models like the one I fit formalize this intuition through a concept called shrinkage - rather than taking each player’s stats at face value, the model blend their observed performance with what it knows about typical passers. A backup with 50 attempts at 75% gets pulled heavily toward the league average; a starter with 5,000 attempts at 68% stays close to his observed rate. The math borrows from Bayesian reasoning - start with prior belief about plausible completion percentage, then update as evidence accumulates. More data means more trust in individual results - less data means more reliance on the overall average. The result is a set of estimates that are skeptical of outliers but reward sustained performance - exactly what is needed to begin to assess a concept of the “best.”

But shrinkage alone doesn’t solve a deeper problem - a 62% completion rate meant something different in 1975 than it does today. Statistical models can address this by encoding the observed completion percentage into distinct components mentioned above. Effectively, the model is asking: Given when this player played, what position he played, and how many passes he attempted, what’s our best estimate of his underlying accuracy? Because the model specification contains both fixed effects - variables that don’t vary at the local level such as era and position - and random effects - variables that do vary at the local level such as individual player skills, you can strip away the fixed environmental factors and be left with the player effects themselves13. I call this the isolated completion effect: the expected completion percentage if every passer played in the same neutral environment. Now I can compare passers from any era (and from any position) on equal footing14, accounting for the inherent differences in position and era to credit and discount observed numbers where appropriately.

Phew - that was a lot of jargon, so it’s time to show the goods. So without further ado, the greatest passers of all time, at least according to my model.

Bart Starr - one of the OG “greatest QBs of all time” - rises to the top as the new greatest completer of passes. This makes some intuitive sense - Starr led the league in completion percentage four times15, was an outstanding postseason passer16, and had an average completion percentage 12% higher than the median passer of his generation - the third highest of all time when organizing passers by decade:

Starr’s Isolated Completion Effect benefits massively from his era adjustment - accourding to the model, the 1960s passing environment cost him about 6 percentage points compared to a more “neutral” environment. Using Drew Brees as a point of comparison, the model believes that had Starr had played in Brees's era, he'd have completed roughly 65% of his passes. If Brees had played in Starr's era, he'd have completed about 51%.

Now you might disagree with this assessment, and sure, the model isn’t the most sophisticated or well-specified1718, but it does start to reveal some interesting aspects of completing passes, as well provide tooling for additional “what-if” scenarios that simple counting and filtering metrics don’t allow. Let’s run through a few that stuck out to me.

Shinkage helps adjust observed completion percentage when accounting for evidence and era

One of the core elements of a statistical model is that it can still produce “expected” outcomes, even if it already has “seen” the observed outcome in the training data. These predictions can be thought of as shrinkage towards the mean when there isn’t enough evidence. As a secondary benefit, the model can also produce credible intervals in which the “real” outcome might sit. This is useful to try to understand what a “true” completion percentage might in the absence of information.

Let’s look at Cortland Sutton, who has gone 4 for 4 in his career passing on trick plays - his shrunken “true” estimated completion percentage is 49.4%, with a 95% confidence range of 42.3% - 56.5%. Or, for more fun, take fellow Bronco “QB” Kendall Hinton - who famously played an entire game at QB for Denver when the rest of their QB room was out with COVID. He went 2-10, but the model sees that 20% and thinks “that’s not that much evidence of skill, so let’s estimate the truth as something far closer to average” and suggests Hinton is roughly a 49.7% passer with a likely range of 42.7% - 56.7%. Looks pretty similar to Sutton!

The model is looking at both of those tiny stat lines and basically shrugging, guessing that without more evidence, it makes sense to just view them as likely average.

When revisiting the best passers by minimum attempts table from above, shrinkage and observation converge as the number of attempts increase - and uncertainty decreases to the point of non-existence. Tom Brady’s nearly 9,000 attempts leave little room for doubt that he was really completing 64% of them…even if that number isn’t fully attributed to him (Brady’s Isolated Completion Effect is 52.6%, in the 92% percentile). Browning, on the other hand, has a chance that his observed 68% completion percentage is “real”, but the model’s best guess is 66%.

Isolated Completion Effect massively re-configures historical rankings

This chart is pretty wild!

Isolated Completion Effect hates some passers I generally think of as pretty good! Andrew Luck, Kerry Collins, Trevor Lawrence become bottom 25 QBs19, and recent Super Bowl Winner and reclamation case Sam Darnold just misses. Meanwhile, HOF Norm Van Brocklin climbs 156 spots from 172 overall in completion percentage (53%) to 14th in Isolated Completion Effect (57%). I love this chart, and readers can explore the nuances further in the data downloads at the end of the post.

Models can time travel

One of the best aspects of a smart statistical model is that they allow for the concept of the “counter-factual” by changing certain parameters but not others. Let’s take our Isolated Completion Effect GOAT as an example…what if Starr was drafted the same year as Drew Brees? Or in any other decade? The model provides counterfactual’s for anyone to examine

The model tells a store in which Starr’s career completion percentage grows with era adjustments to be one of the best of all time - 61% in the 1970s, 63% in the 80s, 65% in the 90s, 67% in the 2000s, and over 70% in the 2010s. Brees’ career completion percentage through his first 16 years measures at 66.4%.

Is this “true”? Not, not really, since Starr only had one career and counter-factuals are precisely what the name says - counter to fact. But it’s still interesting and fun, and that’s what I’m here for.

The GOAT Completer of Passers still has uncertainty

The final major feature to discuss of any model is its estimate of uncertainty. I touched briefly on this in the shrinkage section, but uncertainty in the world is entropic - it propogates through systems and builts on itse;f. The uncertainty in the shrinkage estimates is around confidence in the point estimate - but it’s not the actual prediction interval, which accounts for uncertainty in the model’s estimates of its components. That’s a bit heavy for this post, so I’ll focus just on the uncertainty around the Isolated Completion Effect, and how that uncertainty allows us to test our conclusion20.

I’ll show this in two ways - first through the top 10 Isolated Completion Effect passers of all time, and then through the top players at each position.

With uncertainty about Isolated Completion Effect included, the top 10 list becomes far more nuanced. Otto Graham’s distribution doesn’t appear markedly different thatn Bart Starr’s, and eight of the nine pasers below Starr have uncertainty intervals higher than Bart’s estimated Isolated Completion Effect. This nuance suggests that, even with the era and evidence controls, it is still tough to be too declarative in drawing conclusions. In fact, this can be tested - there is only a 64% chance that Starr was definitevely better than Graham21. That sounds high…but how much would hypothetical you wager on it?

Or take the case of Jamie Martin, a career backup with only about 540 attempts. The model loves him, but it’s not quite sure he’s actually good: his “true” Isolated Completion Effect may be as low as 50% or as high as 64%. This implies about an 84% chance that Starr was better, but a 16% to Jamie Martin is pretty fascinating!

Lastly in the world of direct comparisons, let’s compare Starr to Brees once more - and in this case, Starr is clearly has the higher Isolated Completion Effect. The model estimates a 99% chance that Starr’s effect is higher than Brees. How? Because Brees accumulated so much evidence regarding Brees, it’s pretty certain that his Isolated Completion Effect is between 55% and 58% - and only a small portion of that distribution exceeds Starr’s distibution, despite their mean estimates being fairly close.

Examining the distributions of the top passers by Isolated Completion Effect by position is also interesting - according to the model Antwaan Randle El is credibly a better completer of passes than Starr22!

In the cases where the model has only 1 attempt to work from, the Isolated Completion Effect is massively wide - the model has no idea whether to assign the 100% completion percentage to skill, era, or luck. So it throws it’s hands up, estimates a massive distribution, and then picks the center for the estimate. But as attempts go up - shout-out to occasional QB Charley Trippi! - the estimates uncertainty declines. And that’s why I love statistical models!

So there you have it, 3,500 words later. A theoretically simple analysis that was actually very complex, answered through a statistical model. Congrats to Starr, our GOAT completer of passes, but credit where credit is due - Jauan Jennings with a 51% Isolated Completion Effect, good for 77% percentile in the database. I hope this lengthy analysis illustrated the power behind statistical modeling and the tools that models provide “for free” beyond the analysis itself. And hopefully this analysis illustrates how what often seems like simple questions can be deeply complex, if you take the time to think about them.

Thanks for taking the time,

-Jake

Leave a comment

Artifacts

Isolated Completion Effect database

Passing Seasons database

Passing Career database

Code (written in R)

Subscribe now