Follow me on Twitter!

Thursday, May 22, 2014

Regional Predictions, Week 3

I've said before that I have no doubt the CrossFit Games is a totally viable spectator sport. My opinion on the matter hasn't changed, but I'm beginning to think that it's the Regionals that should be on ESPN. And I doubt anyone who followed the coverage on the Games site this past weekend would disagree with me.

There was high drama all weekend, and with 10 different competitions to follow at various time zones across the globe, events were broadcast basically non-stop. We had the reigning fittest woman on Earth fighting just to make the Games in Europe, possibly the most competitive men's competition ever in Central East, a changing of the guard in Australia and the first real challenge to Camille and Michelle Letendre's dominance in Canada East. So before we move on to predictions for this week, let's get to some quick thoughts on week 2:
  • The handstand walk claimed another victim this week in Sam Briggs. I'm sure HQ would never admit that the programming was anything less than perfect, but anyone following that competition knows that Briggs was one of the three fittest women there, and likely the fittest. She dominated events 3-6 and placed decently on the two heavier workouts (1 and 7).
  • That being said, she will almost assuredly get a special invite, and the three women who did qualify certainly deserved it. Along with Southern California, Europe looks to be one of the two strongest women's regions in the world (Australia also looked sneaky-tough last weekend, too).
  • I certainly hope that HQ also extends a special invite to Graham Holmberg. We'll have to wait to see how the rest of the competitions play out, but I'd be he would have qualified in every other region (and likely won many of them). He came through when it counted most, smashing the event record in event 7, but unfortunately, four other men in his region also beat the event record (including third-place Will Moorad). He was also the only man in that region other than Rich Froning to take an outright first place, and he did it twice.
  • Super-impressive performance by Moorad to grab a spot in the Central East. As disappointed as I was to see Graham fall off, it is nice to see someone else break through in that region.
  • If the Games were programmed like the Regionals, Camille Leblanc-Bazinet might be the most dominant athlete in the sport. If the event has barbells and gymnastics, she's basically guaranteed a spot in the top 10 in the world. Hopefully she can fare a bit better with the more unorthodox Games events this year.
  • While I'm not convinced that event 7 is as good a test of fitness as, say event 4 or event 6, but I can't dispute that it is great for the viewers. Those 8 overhead squats have derailed more than one athlete's shot at the Games and it's given several others the chance to make up ground in dramatic fashion.
  • Event 6 looks like an absolute beast of a workout. Unlike prior years, I haven't been able to test out the workouts (due to a back injury), but I don't recall seeing Rich Froning that gassed in a workout since the rope climb/sled push workout at the 2012 Games. 
So how did my predictions do last week? Well, despite a couple of shockers, things actually pretty well. The charts below show how well the predictions were calibrated this year compared to last year. The athletes were bucketed based on their predicted rank, and each bucket was plotted as a poitn on the blue line.  The location on the x-axis represents my predicted chances of qualifying and the location on the y-axis represents the actual chances of qualifying.  The red line represents perfect predictions, so when the blue line is below the red line, my predictions over-estimated the chances of qualifying for those athletes.  As you can see, this year, the blue line tracks much more closely to the perfect predictions.

Of course, calibration only tells half the story. We could easily create a well-calibrated model by predicting an even chance for each athlete. If there are 30 athletes in a region with 3 qualifying spots, we could estimate that each athlete has a 10% chance of qualifying, and indeed, we would be right in some sense. It is sure that 10% of them will qualify. But we are also trying to be accurate. A perfect model would predict 100% chances for the 3 athletes that do qualify and 0% for all others. To measure our accuracy, we can calculate the means square error across all our athletes. In that respect, I did about as well as last year, and considerably better than the perfectly calibrated model with even predictions for each athlete.
  • 2014 Week 2, CFG Analysis Predictions - 4.38%
  • 2013 Weeks 3-4, CFG Analysis Predictions - 4.43%
  • 2013-2014 Equal Chance Predictions - 6.40%
So, with that in mind, I really didn't make any changes to the model this week. The only region where I had to deviate significantly was for the women in Asia. There are no returning Games qualifiers, hardly any returning regional competitors and just a handful of athletes in the top 2000 in the Open worldwide. My default model would have basically given all athletes the same chance, so I modifiied it as best I could, but the predictions are still pretty weak in that region.
[UPDATE 5/23/2014: In the Asia region, I just realized Candice Ford was Candice Howe last year, which meant I originally assumed she did not compete at the regionals last year. The predictions below are now fixed to account for this.]

Anyway, without further ado, here are my week 3 predictions. Enjoy the Regionals everyone!

Note that Asia only has one qualifying spot. All other regions this week have three.


  1. Rich was sick all weekend and sounds like it turned into bronchitis. I was there and noticed him coughing up a storm more than once after finishing events. You could tell he was doing just enough to stay in front and didn't sprint to the finish lines or else he would have won every event. His wife tweeted that anyone else as sick as he was wouldn't have been competing!

    Have you seen this site?

    1. Nick,

      Thanks a bunch for the link. I had used that site last year but I forgot about it. It mostly jives with what I expected to see, but I think I'll actually add a couple observations to this post later on today based on what I'm seeing from that site. It should also let me get an early look at the correlations between events, which I'm curious about.

      As for Rich, I know he had said he felt sick, but I wasn't sure how bad it really was. The guy is a machine, but I also don't want to take anything away from Panchik for hanging right with him. Rich wasn't exactly sandbagging out there.

    2. Panchik is definitely bringing it. You could see how much he wanted it when he made sprints at the end of some of those workouts.

  2. Anders,

    Thanks for this post. It's been interesting to consider the viability of CrossFit as a spectator sport by comparing Regionals coverage to the clips released by the NPFL as it ramps up. I think the NPFL is imagining a certain degree of pleasure in the sport coming from the sort of statistical perspective that you exemplify here, akin to baseball and football stats.

    To follow up on my last comment, Google+ pretty much sucks as an alert mechanism, so I'm going to make a second plea:

    1. I think I got that set up now. Expect a new post Thursday, so we'll see if it worked.