I can distill the differences in the 2015-16 and 2016-17 models pretty simply:
- 2015-16 took the in-season values for overtime and tie games into effect when assigning K factors and expected values. Tie games were assigned expected values 0.50-0.50, and overtime games were assigned 0.75 for a win and 0.25 for a loss.
- 2016-17 took the in-season values for overtime games into effect when assigning K factors.
Let’s take some examples from last season.
- Week 5: LSSU (EV of 62.54%) lost in 5×5 overtime. They lost 19 points (out of a possible 30). The previous season’s model would’ve had them lose 15. ABOVE 2016 was harsher to LSSU.
- Week 8: UAA (10.95%) hosted MTU. UAA got a shutout win, and it garnered them 9 points (out of a possible 10). The 2016-17 model only assigned a maximum of 10 points. In the 2015-16 model, that game would’ve been a tie, and UAA would’ve gotten 16 points (out of a maximum of 40). ABOVE 2016 was nicer to MTU.
- Week 15: LSSU (EV of 60.35%) won in 3×3 overtime, garnering them 8 points (out of a possible 20). The previous season’s model would’ve had them lose four points. ABOVE 2016 was nicer to MTU.
- Week 16: BGSU (EV of 57.99%) hosted UAF in a game that was decided in a standard 5×5 overtime. BGSU garnered 13 points (out of 30). The previous season’s model would’ve seen BGSU gain just seven points (out of 40). ABOVE 2016 was nicer to BGSU.
These comparisons are a little difficult, I admit. For one, 2015-16 did not have a goal differential bonus: all games had a maximum of 40 points to be assigned, and the teams that got most of them were teams that ran up big upsets as scored by BELOW differentials. But the ultimate result is that the goal differentials and the assumption that a win was a win was a win ended up making the system different, and I’m not sure it was for the better.
What I think that I’m going to try is something like the following:
- The 2015-16 overtime win/loss/tie expected value thinking comes back in. I think that the model should reflect the positive value of a BELOW-says-they’re-weaker team taking the game to the 61st minute, or perhaps a shootout.
- Remove the three-goal bonus, which I’ve half-heartedly defended. I think that there’s value in tracking multi-goal wins, but an upset of three or more goals has happened just four times last year in 34 games of that differential.
Before I dive back into writing the model, I’ll generate a canonical BELOW calculation from 2013 forward using this model:
- Teams get an initial BELOW based on their total winning percentage in the 2012-13 season.
- After each season, a team’s BELOW reverts to the mean by 50%. If you finished 2012-13 with a BELOW of 1400, you start 2013-14 with a BELOW of 1450.
- For every game, take their current BELOW rating to develop an expected value.
- Use the ABOVE model for 2017-18 to assign points as follows:
- Multi-goal wins have a maximum of 50 BELOW points.
- One-goal wins in regulation are assigned 40 BELOW points.
- Wins in a standard overtime are assigned 30 BELOW points.
- If the game’s start date is 2016-10-01 or later, assign 20 BELOW points to the winner in the bogus overtime sessions.
- Wins for recalculating BELOW in regulation time are from a 1.0 actual value. Losses receive a 0.0 actual value.
- Wins for recalculating BELOW in standard overtime are given a 0.75 actual value, while losses receive a 0.25 actual value.
- Any game still tied after 65:00 has each team assigned an actual value of 0.50.
- Repeat through the end of the 2017 Broadmoor title.
Then I can apply an eye test as to how well BELOW correlates with teams making the playoffs, including potentially changing the BELOW points assignments. Then I can put that model and make a script to do a Monte Carlo simulation.
Off to do some data entry.
- Michigan Tech: 62% of the time
- Bowling Green: 38% of the time
- Michigan Tech: 65% of the time
- Bowling Green: 35% of the time
I’ve been doing ABOVE projections with a randomizing element since late January. The reason for this is pretty basic: if you want to test that an estimate is right, you can use a group of estimates clustered around a single estimate and see if any of them are right or wrong. I’ve been using a randomization band of +/- 40, so in a case where I ran 81,000 cases (I run 100,000 to 1,000,000 based on how many games I’m estimating), I’d expect to see around 1,000 uses of each BELOW input.
That first estimate is the randomized BELOW estimate churned through the ABOVE calculation 1,000,000 times. Coming into the final, Michigan Tech is at BELOW 1701 and Bowling Green is at 1617, its highest mark in 2016-17. As a result, we could have a matchup of MTU 1741, BGSU 1577; in that matchup, Michigan Tech would be picked to win 72% of the time. We could also see MTU 1661, BGSU 1657, and that’s a dead heat.
When I noticed that BGSU was at its best mark of the year, I decided to additionally randomize between the best and worst marks for each team. MTU has been as high as 1739 and as low as 1586; BGSU is at 1617 now and was as low as 1479. This gives us a band of 54% BGSU favored to 82% MTU. Throw that in the wash and you get that second pair of numbers.
Which is better? It’s a single game. BG fans will probably rib me if their team wins on Saturday. MTU fans will probably accuse me of jinxing them. I’ll probably be frustrated either way, but it’s a certainty that I’ll be back next year.
Ed.: Post updated to fix the date of the final game.
Congratulations to Bowling Green for downing top seed Bemidji State, a team that hadn’t lost two league games in a row all season and lost both games in a series just twice (to #1 North Dakota and a streak-starting Princeton team).
Minnesota State pulled off a 1-0 win over Michigan Tech to push the 2-3 semifinal to a third game. Here are your projections:
- Michigan Tech is expected to win on Sunday 58.131% of the time. If they win, they are favored to beat Bowling Green in Houghton next weekend 67.802% of the time.
- In the 41.869% of trials where Minnesota State wins on Sunday, they beat BG in a game they host 60.981% of the time.
- Despite the fact that they are a strong underdog in both finals matchups, Bowling Green is projected to be the Broadmoor Trophy winner 35.054% of the time, second to Michigan Tech at 39.414% and ahead of Minnesota State at 25.532%.
If Michigan Tech wins on Sunday and Bowling Green wins next week, we will have the same combination of playoff seed wins as last year: 1-2-3-4 in the quarters, then 2-4 in the semis with a 4 in the finals. It would be truly interesting for the Falcons to win given their start.
Enjoy the games! We have just two left, and that is sad.
After a 5-1 Michigan Tech and a 4-3 overtime Bowling Green win, it’s time to update Friday’s projections.
Overall Broadmoor Trophy Projection:
- Bemidji State: 16.278%
- Michigan Tech: 48.734%
- Minnesota State: 11.848%
- Bowling Green: 23.140%
Bowling Green (1587) at Bemidji State (1640)
- Bowling Green sweep: 21.618%
- Bowling Green in 3: 42.469%
- Bemidji State in 3: 35.91%
Minnesota State (1642) at Michigan Tech (1700)
- Michigan Tech sweep: 58.282%
- Michigan Tech in 3: 21.428%
- Minnesota State in 3: 20.290%
Michigan Tech at Bemidji State final:
- Occurs 28.525% of the time
- Bemidji State wins 44.673% of those matchups.
- Michigan Tech wins 55.327% of those matchups.
Minnesota State at Bemidji State final:
- Occurs 7.332% of the time
- Bemidji State wins 48.213% of those matchups.
- Minnesota State wins 51.787% of those matchups.
Bowling Green at Michigan Tech final:
- Occurs 51.053% of the time
- Michigan Tech wins 64.545% of those matchups.
- Bowling Green wins 35.455% of those matchups.
Bowling Green at Minnesota State final:
- Occurs 13.091% of the time
- Minnesota State wins 61.508% of those matchups.
- Bowling Green wins 38.492% of those matchups.
First semifinal: Bowling Green (BELOW 1568) at Bemidji State (BELOW 1659)
- Bemidji wins 66.850% of the time: in 2: 41.926%; in 3: 24.924%
- BG wins 33.150% of the time: in 2: 16.591%; in 3: 16.559%
Second semifinal: Minnesota State (BELOW 1672) at Michigan Tech 1670
- Tech wins 49.614% of the time: in 2: 27.742%; in 3: 21.871%
- Mankato wins 50.386% of the time: in 2: 22.109%; in 3: 28.277%
- Bemidji-Tech: 33.145%.
- Bemidji-Mankato: 33.705%.
- Tech-BG Final: 16.460%.
- Mankato-BG Final: 16.681%.
- Bemidji State: 33.350%
- Michigan Tech: 24.850%
- Minnesota State: 25.242%
- Bowling Green: 16.558%
Once you get to the finals, all four are 50-50 matchups. Here’s why:
- Because Michigan Tech and Minnesota State are in a dead heat with Bemidji State close behind, the Tech-Mankato winner is going to be just a little bit better because that season is generally going to be close.
- If Bowling Green gets the upset, they’ll be pretty close to their finals opponent.
This is a lot like last year, where the top three teams were pretty clustered in terms of an estimate and Ferris broke through and were the better team that weekend. ABOVE, by randomizing BELOW estimates within a range, accounts for teams getting hot and going cold and, over time, averages that out. Now it makes more sense for a regular season projection, but it works here in the finals.
So what you see above is the projection of Bemidji State likely winning 2/3 of the time and the Tech-Mankato semifinal being a toss-up. I could run that 1,000,000,000 times and probably get something close to the same result as I did with 1,000,000.
Enjoy the games. The math means little when the puck drops.
2017 Post-Quarterfinal BELOWs
These are the BELOW estimates after the quarterfinal round of the 2017 WCHA Playoffs
I’m as surprised as you are, but it makes sense. Here’s why:
- Michigan Tech blew out Lake Superior, but LSSU was the weakest team in the tournament.
- Bemidji State went to three games with Northern Michigan and essentially stayed even in BELOW — they’re a team that was expected to win 70% of the time against a league-average team, which Northern Michigan was.
- Minnesota State played a good Alaska team and fairly well dominated them, winning by two and three goals.
I honestly wouldn’t be surprised to see Mankato win this tournament. Four of the league’s top 11 scorers wear purple. Goaltending is still a question, but Jason Pawloski put the Nanooks to sleep on a hyuuuuuuge pilla. As good as Angus Redmond was in his first two months, he’s never been the same since Alabama-Huntsville hung six on him in late January and has looked more medium than well-done.
I’ll be back to running numbers, but that Tech – Mankato series promises to be a treat.
I haven’t re-written all of the code (gotta love being on-call), but here are projections for the four quarterfinals and the combinations that result:
Quarterfinal #1: Northern Michigan at Bemidji State
- Bemidji sweeps 68.587% of the time and wins in three 20.975% of the time.
- Northern sweeps 4.335% of the time and wins in three 6.103% of the time.
Quarterfinal #2: Lake Superior at Michigan Tech
- Tech sweeps 65.791% of the time and wins in three 21.993% of the time.
- Lake State sweeps 5.191% of the time and wins in three 7.025% of the time.
Quarterfinal #3: Alaska at Minnesota State
- Mankato sweeps 51.005% of the time and wins in three 24.905% of the time.
- Fairbanks sweeps 11.244% of the time and wins in three 12.846% of the time.
Quarterfinal #4: Ferris State at Bowling Green
- BG sweeps 33.769% of the time and wins in three 24.982% of the time.
- Ferris sweeps 20.526% of the time and wins in three 20.722% of the time.
- Bemidji – Tech – Mankato – BG : 35.047%
- Bemidji – Tech – Mankato – Ferris: 24.603%
- Bemidji – Tech – BG – Fairbanks: 11.138%
- Tech – Mankato – BG – Northern: 9.278%
- Bemidji – Tech – Ferris – Fairbanks: 7.818%
- Bemidji – Mankato – BG – : 4.862%
- Bemidji – Mankato – Ferris – Lake State: 3.453%
- Tech – Mankato – Ferris – Northern: 2.864%
- Bemidji – BG – Fairbanks – Lake State: 1.557%
- Tech – BG – Fairbanks – Northern: 1.288%
- Bemidji – Ferris – Fairbanks – Lake State : 1.083%
- Tech – Ferris – Fairbanks – Northern: 0.908%
- Mankato – BG – Lake State – Northern: 0.566%
- Mankato – BG – Lake State – Northern: 0.398%
- BG – Fairbanks – Lake State – Northern: 0.175%
- Ferris – Fairbanks – Lake State – Northern: 0.121%
This is pretty standard ABOVE stuff:
- Take the final 2016-17 BELOWs and randomize them +/- 40 points to enhance their quality as estimates.
- Calculate expected values using the Elo-style formulas.
- Have a random dice roll for an outcome of the game.
- Determine a winner of the game based on the expected values.
- Re-calculate BELOW (based on the randomized estimates).
- Repeat steps 2-4.
- If one team has one two games, they’ve swept and are declared the winner.
- If the teams split the first two games, repeat steps 2-5 to get the third game.
- Perform steps 7 and 8 for each of the four quarterfinals and account for one of the sixteen possible semifinal matchups.
I just didn’t get to finishing the semifinal and final codes for this. I’ll look to do that tomorrow when perhaps I’ve slept.
2016-17 Week 21 BELOW
Above are the 2016-17 BELOW rankings after Week 21 of WCHA conference play (140 of 140 games played).
It’s important to note that Northern Michigan is the 8th seed, facing Bemidji State, but is 6th in the final 2016-17 WCHA regular season BELOW ratings. If BSU thinks that they have an easy setup for the first round, they have another think coming.
I’ll go polish the playoff model up a bit and have the projections out soon. This is the same model that I used back in September to revisit how the playoffs looked. When we look at probabilities in the projections, please re-visit that post: what we had in the 2016 playoffs, with the tournament pretty much going chalk, was still just a 1% probability. With eight teams, the probabilities are pretty wild — and FSU won with a 11% chance at the start.
|Bemidji State||1 - 100%||
|Michigan Tech||2 - 100 %||
|Minnesota State||3 - 100 %||
|Bowling Green||4 - 65.91%||5 - 34.09%||
|Ferris State||4 - 34.09%||5 - 65.91%||
|Alaska||6 - 100%||
|Lake Superior||7 - 84.25%||8 - 14.60%||9 - 1.16%
|Northern Michigan||7 - 14.60%||8 - 53.85%||9 - 31.56%
|Alabama-Huntsville||7 - 1.16%||8 - 31.56%||9 - 67.29%
|Alaska-Anchorage||10 - 100%||
I’d like to thank the league for putting out a press release so I didn’t have to work on this at work, and I’d like to thank the teams for making it easy on me — we know half of the results already. Here’s what’s left:
- Bowling Green and Ferris State are tied at 41 points, and the Falcons own the tiebreaker.
- Lake Superior is at 8-3-3-13/33, Northern Michigan is at 9-1-2-15/31, and Alabama-Huntsville is at 9-0-3-15/30.
- Lake Superior makes the playoffs as the 7th seed if they get a win tonight in regulation or overtime. They also get 7th as long as they have more points than Northern Michigan, who wins at B.
- Northern Michigan makes the playoffs at the 7th seed if they pass or equal Lake Superior in the standings unless Alabama-Huntsville ties the two at 33, which takes the Lakers losing and Northern winning in 3×3 or the shootout and the Chargers winning over Bowling Green.
- Lake Superior finishes in 8th if Northern passes or ties them without Huntsville doing the same. UAH winning the three-way tiebreaker is the only way that LSSU misses the playoffs.
- Northern Michigan finishes in 8th anytime they stay ahead of Huntsville but don’t pass Lake State.
- The NMU-UAH tiebreaker is fun: at 33, UAH wins at B; at 32, at D; at 31, at C. That’s just wacky.
2016-17 Week 21, Series 70: Michigan Tech (1666) at/home Northern Michigan (1488)
|MTU 5 points (W, OTW)||7.71%
|MTU 4 points (W, OTL or OTW, OTW)||5.05%
|NMU 5 points (W, OTW)||3.39%
|NMU 4 points (W, OTL or OTW, OTW)||2.52%
This is our lop-sided matchup of the week, and it’s a shame, really. The Wildcats’ resurgence has been remarkable, but facing the league’s best team, even on rivalry weekend, is a tough task. They have to like their chances of keeping ahead of the pack with a split or better — about 41% — but the long losing streak they had has made the playoffs a dicey proposition.