Is BELOW Working?
What a wacky weekend. If you missed it, here are the details:
- Minnesota State’s two-goal win over Bowling Green on Saturday night was the only regulation win, and it included an empty-net goal.
- Two more games were decided in standard overtime: Alaska – Ferris State on Saturday and Bowling Green – Minnesota State on Friday.
- Three more games were decided in 3-on-3 play on Friday: Alaska – Ferris State, Alaska-Anchorage – Michigan Tech, and Lake Superior – Northern Michigan.
- Two games went to a shootout on Saturday: Alaska-Anchorage – Michigan Tech and Lake Superior – Northern Michigan.
Before this weekend, 71 of 84 games were decided in regulation (85%). Now it’s 72/92 (78%). We saw these kinds of blips last season, and sometimes they make sense and sometimes not. That UAF-FSU and LSSU-NMU were close and required OT for both games isn’t surprising, and BGSU-MSU having a close series wasn’t either.
That UAA-MTU result, though…
I will argue that BELOW is working, and here’s why:
[table “116” not found /]
The key thing to note is those last two columns. If a team’s EWP from BELOW is higher than its point percentage (current points / games played to-date), it’s underperforming BELOW. The true over-performers are Bemidji State and Alabama-Huntsville. The under-performers are Minnesota State, Northern Michigan, and Bowling Green.
Masked in this, though, is who these teams have played.
When the going gets tough …
2016-17 Week 15 BELOW Revisited
BELOW: Current BELOW (Week 15).
CBELOW: Weighted average of a team's opponents' BELOW, to-date.
RBELOW: Weighted average of a teams' future opponents' BELOW.
CBELOW is the weighted average of BELOW of a team’s opponents to-date. For example, UAH has already played FSU four times this season, so the Bulldogs’ BELOW counts four times in the average; conversely, the Chargers haven’t played BSU at all. As a counter-example, UAA has played a murderers’ row of teams to-date but plays just one team in the top-half of the league down the stretch.
The hardest schedules down the stretch belong to Huntsville, Ferris, Northern, and Bemidji.
This week was an outlier
Yes, the results of ABOVE-based predictions were spotty this week. In defense of the model, going from 13 overtime results to 20 was a big shock to the system and isn’t something that you’d expect very often — 0.00018% of the time, in fact.
The model will adjust in lots of ways:
- I’ll modify the coefficients for going into standard, 3×3, and shootout overtimes. These will be a lot higher than what they currently are. This will collapse ABOVE’s predictions a bit.
- UAA is up and MTU is down after last week. The other series were pretty close in terms of expected winning percentages, so mixed bags and overtimes didn’t move BELOW much there.
I’d like to take a moment and address some feedback. There are two things to note here.
First, BELOW is an estimate of what we think a team’s value is based on the results that it’s had to-date. I even went so far as to re-evaluate the very basis of BELOW: the pre-season estimate.
2016-17 BELOW, Week 15, Revisited Again
Above are the 2016-17 BELOW rankings after Week 12 of WCHA conference play (92 of 140 games played).
|Team||BELOW 2016||BELOW 1500||EWP2016||EWP1500||Pts%
This is a busy chart, so I’ll explain:
- BELOW 2016 uses the baseline numbers from last season as a starting point.
- BELOW 1500 assumes that all teams are league-average going into the season.
- EWP 2016 is the expected winning percentage based on BELOW 2016 values.
- EWP 1500 is the expected winning percentage based on BELOW 1500 values.
- Pts % is the number of points divided by the number of league games.
When I look at that table, I don’t see a conclusion that I can draw. Yes, EWP 1500 is closer to PTS% for every team, but that’s to be expected because the teams are all fully reverted to the mean, rather than partially reverted. That said, I don’t think that BELOW 1500 is the answer because it’s even worse on BSU, UAA, and NMU, farthest from that mean.
What can we validate against? I can re-run every games from 2012-13 forward and see what works historically.
A Change to BELOW
Lastly: I have implemented a change to BELOW. I had not differentiated one-goal wins in overtime and regulation, but now I’m going to. The multipliers are now as follows (the change is in bold):
- Winning in the shootout gets a maximum of 10 points. This has occurred 7.61% of the time in 2016-17 to-date.
- Winning in 3-on-3 overtime gets 20 points. (7.61%)
- Winning in 5-on-5 overtime gets 30 points. (7.61%)
- Winning in regulation by one goal gets 40 points. (30.43%)
- Winning by two goals gets 50 points. (22.83%)
- Winning by three or more goals gets 60 points. (23.91%)
I think that it’s fair that winning in overtime shouldn’t have the same value as winning in regulation, as a team who holds a one-goal lead in regulation has arguably done a tougher job than the one gaining the win in overtime.
I’m still not sure about variable multipliers that take into account the BELOW estimate of the teams. I haven’t come up with a good way to do it, though. My response is that a team with a 10% expectation of winning that blows out the other team deserves the 54 points that they get, and that’s 48 points more than the 90% team.
What’s very clear is this: teams go on runs. Bemidji was on a great one to start the year, and their strong lead in the standings is proof of that. Northern was on a similar trend, which is why BELOW thinks that they should have a couple of more wins than they actually do.
I’ll revisit this again next week. Those eight games may be an outlier, or they could be an indication that the league is closer than we thought and that maybe those multipliers are too big.