Inspired by Trey’s comment here: Ottoneu Surplus Calculator (2017 version), I went back and looked at eight of my FGPTS leagues where I had saved a screenshot of the Surplus Calc keeper surplus as of 1/31/16. I compared the ranking of Proj Val Infl (the projected total value the team would have after auction, assuming $ spent at auction returned value based on inflation) to the final standings. Before I share the results, a few caveats:
Unlike Trey’s suggestion, this is not looking at surplus rankings right before the season began (after auction), but is rather comparing the teams only based on the players they kept
If an owner did particularly well or poorly buying talent at auction, that would have a big effect on the quality of their team going into the season
Obviously another big factor that creates variance between the keeper rankings and final finish is the in season management of that team (via lineups, add/drops, trades, etc).
Finally, these surplus calc rankings were based on January projections, before ZiPS had even been published. You also have to factor in the natural variance that always exists between projections and actual results.
Here is the google sheet with the data: https://docs.google.com/spreadsheets/d/14kAQAqytVtfMIih4eLufV8b-6V2MSF1e1EoZgidgNmg/edit?usp=sharing
So how did my keeper rankings do compared to actual results? Overall I’d say fairly well, though they performed worse than I would have expected. Here are some of the interesting data points:
1st place keepers averaged 3.25 actual finish (two finished 1st, two finished 2nd, worst finished 8th)
2nd place keepers averaged 3.00 actual (two finished 1st, two finished 2nd, worst 6th)
3rd place keepers averaged 3.75 actual (one 1st, two 2nd, zero 3rd, worst 6th)
4th place keepers averaged 5.63 actual (one 2nd, one 3rd, worst 9th)
5th place keepers averaged 6.88 actual (one 1st, one 2nd, worst 12th)
6th place keepers averaged 8.00 actual (one 3rd, worst 11th)
7th place keepers averaged 5.88 actual (two 1st, one 3rd, worst 11th)
8th place keepers averaged 6.75 actual (two 3rd, worst 12th)
9th place keepers averaged 8.13 actual (best 6th, worst 12th)
10th place keepers averaged 9.75 actual (best 3rd, worst 12th)
11th place keepers averaged 8.13 actual (best 4th)
12th place keepers averaged 8.88 actual (best 4th)
So two teams projected as 7th best keepers won their league, one team projected 1st best keepers finished 8th, the highest rated keeper team that finished in 12th place was 5th best keepers, and the highest actual finish for a team with 12th ranked keepers was 4th.
*Edit- thanks to help from @TheEmbassy, I ran a correlation test on the data (Kendall’s tau) and the correlation between keeper surplus rank and actual finish was approximately 0.45, meaning there is a 45% chance that actual finish will be predicted by keeper surplus rank)