A Closer Look at the 2014 Games Programming

Today's post will basically put a bow on the 2014 CrossFit Games season.  What's that, you say?  Didn't the Games end 2 months ago?  Haven't we moved onto the Team Series already?  Isn't the 2014 Games season in the distant past?  NO!  It's not over until I say it's over, and since I've been too busy to wrap it up until now, it's not over yet!

Sorry.  Anyway...

Like last year, I'll break things down based on the five goals I think that should be driving the programming of the Games, in order of importance:

1. Ensure that the fittest athletes win the overall championship
2. Make the competition as fair as possible for all athletes involved
3. Test events across broad time and modal domains (i.e., stay in keeping with CrossFit's general definition of fitness)
4. Balance the time and modal domains so that no elements are weighted too heavily
5. Make the event enjoyable for the spectators

For each goal, there will be some brief discussion and analysis, and I'll conclude by pointing out suggestions for improvement, because simply identifying the problems only gets us halfway there. Additionally, I'll point out things that I felt worked out particularly well.  For those who've been following this site for a while, this is basically the same way I broke things down at the end of last season.

So let's get started.

1. Ensure that the fittest athletes win the overall championship

I'll start by saying that I do think the two fittest athletes won the titles this year.  Rich Froning came through when it counted, and despite looking more human than in years past, I think there is no doubt that he deserved his fourth straight title.  Camille was ridiculously consistent the whole season, and her combination of Olympic lifting prowess and gymnastic ability is pretty much unrivaled.  Also, as discussed in this post from August, several different scoring systems would have produced the same champions.

That being said, if we're being honest, we have to acknowledge that Sam Briggs realistically could have won the Games if she qualified.  She dominated the Open, and aside from one event (an extremely specialized event at that), she was among the best in the world at Regionals.  While maybe not the most skilled, I feel she has the best conditioning in the world.  Looking at the programming that came out at the Games I think she would have fared well, but it would not have been easy for her.

Looking at the 36 events she competed in during 2013 and 2014, she's only finished outside the top 20 worldwide in 7 of them: 2013 Regional Event 2 (3-RM OHS), 2013 Games Zigzag Sprint, 2013 Games Clean & Jerk, 2014 Open Event 2 (C2B and OHS), 2014 Regional Event 1 (max hang squat snatch), 2014 Regional Event 2 (max HS walk) and 2014 Regional Event 7 (pull-ups and heavy OHS).  The theme here is heavy Olympic lifts and extremely short time domains.  Other than that, she's at the very top of the world.

Applying that to this year's Games, she potentially would have struggled on the 1-RM OHS, the Clean Speed Ladder, the Midline March (due to HS walks) and Thick and Quick.  Other than that, I'd expect her to be top 10 in basically everything else.  Would that have been enough to win?  It's hard to say.

How We Can Do Better: Avoid programming extremely volatile events at Regionals (like a single attempt at a HS walk).  Remember, the goal is to find the fittest athletes for the Games, not just to see who can avoid screwing up (this isn't American Ninja Warrior).  Also, I hope the qualification system gets tweaked to allow more athletes from the elite regions to make it (Central East men, for instance).
Credit Where Credit is Due: The Games test was not overly grueling this year (see discussion in my last post), and while we did have a couple of top contenders fall out due to injury (Kara Webb and Anna Tunnicliffe), it seemed like most of the athletes were competing at their best the entire week.

2. Make the competition as fair as possible for all athletes involved

I think HQ learned from the mistakes of 2013 as they programmed the Open this year.   Judging was pretty straightforward on every event in the Open, and really judging wasn't a major issue throughout the season.  To me, that's a huge key for this sport moving forward.  The less we can have spectators talking about the judging, the better.

I also like the improvements in the way ties were handled at the Games.  We saw very few big logjams the way we did in the in Cinco 1 and 2 in 2013.  All the athletes were able to separate themselves throughout the field on every event.

How We Can Do Better: Though I personally liked the workout, the inclusion of a rower in Open Event 4 goes against the fact that HQ has consistently said that the Open is for anyone in the world.  Rowers are far more expensive than the other pieces of equipment that have been required in previous years.

Credit Where Credit is Due: Judging is becoming less and less of a factor throughout Open, Regionals and Games.  Also, there were fewer massive ties in the standings at the Games.

3. Test events across broad time and modal domains (i.e., stay in keeping with CrossFit's general definition of fitness)

Like last year, let's start by looking at a list of all the movements used this season, along with the movement subcategory I've placed each one into. I realize the subcategories are subjective, and an argument could be made to shift a few movements around or create a new subcategory. In general, I think this is a decent organizational scheme (and I've used it in the past), but I'm open to suggestions.

As we've seen in recent years, the season as a whole is testing a very wide variety of movements.  This year saw 27 different movements, compared with 29 last year.  The only movements that did not appear at all this year that have appeared in at least two other seasons are: bike, KB swing, wall climb-over and push-up. I don't think leaving those out are too much of a concern.

Another key goal is to hit a wide variety of time domains and weight loads. Below are charts showing the distribution of the times and the relative weight loads (for men) throughout the entire 2013 and 2014 seasons. The explanation behind the relative weight loads can be found here.  Two notes: 1) some of the Regional and Games movements had to be estimated because I don't have any data on them (such as weighted overhead lunge and pig flips); 2) the time domains for workouts that weren't AMRAP were rough estimates of the average finishing times.

What was lacking this year, as discussed previously, were the extremely long events.  There were no events beyond 45 minutes this season, and there were twice as many sub-5:00 events as last year.  You'll also notice that there were a smaller number of lifts in all of the ranges except the very low (0.4-0.8) and the very high (2.4+).  Part of this is just the fact that there were more lifts last season, though that was not the case at the Games.  We have previously noted that the Regionals were particularly bodyweight-focused this season, but the Games made up for that by being particularly heavy.

How We Can Do Better: I'd like to see us go really long at least once (though probably not more than once).

Credit Where Credit is Due: The season really didn't miss out on much.  It's really not likely that an athlete could finish well these days with a hole in any area of their fitness.

4. Balance the time and modal domains so that no elements are weighted too heavily

Based on the subcategories of movements I've defined above, below is a breakdown of movements in each segment of the 2014 Games Season. As in the past, these percentages are based on the importance each movement was given in each workout, not simply the number of times the movement occurred (so an OHS in the 1 RM at the Games was worth more than the OHS in Open 14.2).

We see this year that there was consistently focus on the Olympic lifts at all phases of the competition, but we saw changes in the other categories.  Pure conditioning (running/rowing/double-unders) increased in value steadily throughout, whereas high skill gymnastics increased dramatically after the Open.  Conversely, basic gymnastics decreased in value steadily and was not much of a factor in the Games.  As is typical, we didn't see many of the uncommon CrossFit movements until the Games.

In my opinion, the Olympic lifts were maybe a touch overvalued (a third of the competition seems like a bit much).  Also, I'd like to see the three phases of competition be a little more similar, so that you don't have athletes qualifying for the Games without being competent in a particular area, only to be exposed at the Games.  For instance, we continue to see running have little value until the Games, which means we may be letting in athletes who are poor runners.  Of course, those athletes won't win the Games anyway, but you may be keeping out athletes who would have been better Games performers.

That being said, there's no such thing as perfect balance here.  We know the Open can't have a ton of high skill gymnastics movements, and we know that Open-style workouts might not be spectator-friendly at the Games.  You could also argue that powerlifting-style lifts are undervalued, but I'm of the opinion that the Olympic lifts are a much better test, and HQ seems to agree, so I doubt we will see that change.  I don't see any glaring problems in the chart above, so that's good news.

As far as time domains, I think there was pretty good balance, even though a bit more weight was given to shorter workouts than in years past.  Like I mentioned previously, I can't complain too much about this, because it seems to keep the athletes fresher throughout the Games.  As long as HQ continues to throw in a few nasty workouts in the 15-25 minute range, plus one or two really long ones, I think that's sufficient.

Another way to see if we're not weighting one area too much is to look at the rank correlations between the events. If the rankings for two separate events are highly correlated, it indicates that we may be over-emphasizing one particular area.  As I did last year, I focused only on the Games, since it's not really a problem if we test the same thing in two different competitions since the scoring resets each time.  It's the overemphasis within the same competition that's a problem.

The chart below shows all the combinations of men's events at the Games, excluding the finals, since the entire field did not compete.

The cells highlighted yellow had a correlation above 50%, and the cells in red had a correlation below 0% (for some reason, the "-" sometimes doesn't come through well on the picture, but those are negative numbers).  Only two combinations had a correlation above 50%, and one of those were the two sprint sleds (each of which were worth only 50 points, so this actually isn't really a problem that they're highly correlated).  The other combination was event 2 (1-RM OHS) and event 9 (clean speed ladder) - this shouldn't be surprising.

For women, the results were similar, although there were also high correlations between Event 1 (The Beach) and Event 3 (Triple-3) and between Event 2 (1-RM OHS) and Event 6 (21-15-9).  Not sure about the OHS/21-15-9 combination, but the other one also makes sense intuitively since both events are very long.

It's also interesting to note the negative correlations, most notably the -31% correlation between Event 3 (Triple-3) and Event 9 (Clean Speed Ladder).  Women showed a -14% correlation here as well.

All in all, this is similar to what we saw last year, but probably a bit more balanced.  Between men and women, there were 6 correlations above 50%, compared to 8 last year, and as mentioned, 2 of those (sprint sled events for men and women) were only given half-weight.  

How We Can Do Better: I've said it before, but it bears repeating: We need to test running earlier in the season (and more than just 50-yard jogs between rope climbs).  Also, I'd prefer we lighten up just a bit on the Olympic lifting emphasis.

Credit Where Credit is Due: Things were really balanced overall, and cross-correlations between events at the Games were lower, indicating we weren't testing the same things a bunch of different times.

5. Make the event enjoyable for the spectators

I was lucky enough to attend the Games in person for the third straight year (thank you to my lovely wife), and I can say without a doubt that they are improving the spectator experience each year.  I was originally concerned about the fact that the morning and afternoon events would all be in the soccer stadium, but I think they made it work.  There's still no doubt that the tennis stadium is the more exciting venue, but the issue there is that you can't fit as many athletes in each heat, so the sheer number of heats tends to make things drag on.  In the soccer stadium, they were able to keep most events to 3 heats, which really is a big improvement over 4 heats.  Not to mention that all the silver ticket holders get to watch those events as well.

Another change was that the team events were all done in the morning prior to any individual events, so it really separated that out from the individual competition.  While some of the teams may disagree, I liked this move because it shortened the breaks between individual events.  Some of the team events can really drag on, and while the top heats are often exciting, the majority of fans probably don't want to sit through tons of the team competition if they don't have to.  And of course, those who do want to watch can show up in the morning and get great seats.

I think ultimately, the Games are going to have to have all of the events in the soccer stadium, or some other large venue.  There is simply too much demand to continue limiting the Gold tickets to only 10,000 fans.  From my understanding, they sold out in a matter of minutes this year, and even some people who were logged in prior to the tickets going on sale could not get them.  Let's hope HQ can find a way to satisfy the demand while still keeping the event as exciting as possible for those who do get tickets.

How We Can Do Better: Do not let Swizz Beatz perform again.  In fact, replace the musical act completely with those crazy gladiator dudes.

Credit Where Credit is Due: The experience continues to improve each year.  The views in the soccer stadium were improved, there were fewer lulls in the action and the spectators were more engaged than in previous years.  The conclusion of that men's Push-Pull event with Bridges narrowly holding off Froning was still the highlight of the weekend for me.


And now... the 2014 CrossFit Games season is in the books.

2014 Regional Review

Every year, watching the Regional competition, seeing how incredibly difficult it is to qualify for the Games even for the most seasoned of athletes, it's hard to imagine that those who do qualify have any weaknesses. Yet in a few short weeks, we will once again see the programming elevate to yet another level, exposing any deficiencies and allowing the best of the best to shine. Even with a few big name athletes out of the mix, it's going to be another tremendous competition, possibly the best to date (the women's side in particular is WIDE open).

But before we move fully into Games mode, let's take a more thorough look back at this season's Regional competition.

First, we'll start with the programming. Although I thought the programming was solid this year, I still preferred last season's events a bit more. A big reason for that is the handstand walk event - I simply cannot get over the fact that a single attempt at walking on one's hands was worth the same amount of points as classic CrossFit metcon like event 3, event 4 or event 7. It just seems to me that if you're evaluating an athlete, those other events tell you a lot more than a handstand walk. Part of that is opinion, but it is also backed up by the numbers. Below is a chart showing the correlation between an atahlete's ranking in each Regional and Open event this season and the athlete's ranking across all other events. Higher numbers indicate that the best overall athletes are typically finishing high on the event, which generally means the event was a good test of fitness.

As I noted last week, the events that really stood out on the positive side were Regional Even 3, Regional Event 7 and Open Event 4. All three had correlations near 80% for both men and women. I've admitted already that my doubts about Regional Event 7 were unfounded, but I'll state it again: this was a pretty great event. On the flip side, you can see that the handstand walk (Regional Event 2) had one of the lowest correlations for both men and women, although the run/rope climb workout (Regional Event 5) had the lowest correlation of any event. My theory here is that since most other events generally favored smaller athletes, many top athletes on the other workouts struggled a bit here since the height was a clear advantage.

Which brings me to a second topic on the programming: did the Regional programming really favor smaller athletes, as many (including me) expected it would? This came up on the Update Show last week, and predictably it was dismissed as being a myth. The evidence? Well, since Tommy Hackenbruck, Jason Khalipa and Elizabeth Akinwale all qualified, the programming must not have favored smaller aithletes, according to Pat Sherwood. Needless to say, as a stats guy (and just a fan of the sport in general), hearing this topic brushed off so casually was irritating for a number of reasons:

• Anyone who knows anything about data knows that you can't prove a point like this simply by cherry picking a few oddball cases. Can we assume height is not advantageous in basketball because Nate Robinson is in the NBA? Of course not.
• Tommy Hackenbruck is not even that tall. The man is 6'1", so let's keep things in perspective here. And Elizabeth Akinwale is only 5'7"!
• As much as HQ talks about data points and all the insights they can gain from the numbers, we so often get responses like this instead of actual analysis. HQ has the numbers at their disposal to provide better evidence to support their point, yet they choose not to. I love this sport, but we lose credibility when the organization in charge so rarely admits any fault.

None of this is to say that Sherwood was necessarily wrong. His opinions were simply not supported by facts. That's what is irritating.

However, the numbers are available to see if the HQ company line is in fact correct. I spent a couple of hours compiling the heights and weights of male Games qualifiers from the past three years (I'd like to get the female numbers at some point when I have some more time). Here are the raw averages for the field in those years:

• 2012 - 69.9 inches, 198 pounds
• 2013 - 69.9 inches, 197 pounds
• 2014 - 69.4 inches, 192 pounds

Clearly there is a drop in size in the past year, and if we throw out the bottom and top two values from each year (including 6-foot-5 Aja Barto in 2012 and 2013), the trend actually becomes a bit more apparent.

• 2012 - 69.9 inches, 199 pounds
• 2013 - 69.7 inches, 196 pounds
• 2014 - 69.4 inches, 192 pounds

Now, is this a drastic drop? No. But the fact that the average size of the Games athletes is a half-inch and 5 pounds lighter than a year ago is not meaningless. Perhaps you could argue that the programming in prior years was too favorable towards larger athletes (I wouldn't argue that, but you could). But to just ignore the numbers altogether is a bit disingenous.

Something else that came out of this investigation (which I hope to expand on at a later date) was that I decided to look at how the distribution of heights for Games qualifiers compared to the general population. My theory was actually that since the programming is generally so balanced, we'd see that the Games athletes have a distribution of heights that are pretty typical. Below are the comparisons (Games athletes distribution based on the qualifiers from the past three years combined, general public data from the Census bureau).

Although the shape of the distribution isn't terribly different, we do see that Games athletes are more concentrated around the average (approximately 5-10) than the general public. In fact, this has been a trend in recent years. The standard deviation in height among Games qualifiers has dropped from 2.4 inches in 2012 to 2.1 inches in 2014, and the standard deviation in weight has dropped from 15.9 pounds in 2012 to 13.2 pounds in 2014. I expect this will continue in the future as the field grows and it becomes harder and harder to overcome any natural disadvantages.

This brings me to my final topic for today: who were the most impressive athletes at this year's Regionals? Obviously, Rich Froning remains the dominant athlete on the men's side, staying well clear of the field despite (reportedly) battling a nasty cold all weekend. And for sure, Scott Panchik again showed that he's a podium contender by hanging right with Froning (and also finishing well ahead of the third-place athlete in the Cross-Regional Comparison). On the women's side, clearly Camille showed up big-time, as she typically has in past Regionals.

But considering what we saw above regarding the concentration of "average size" athletes, it does make performances by athletes like Hackenbruck, Nate Schrader (6'1") and Becca Voigt (5'9") even more impressive. In fact, I'd venture to say these athletes will do even better at the Games, since the programming could be more favorable to their size. Hackenbruck and Schrader were both in the low 20s in the Cross-Regional Comparison (the best of any athletes over 5-11), and I could easily see both in the top 10 at the Games.

That's it for today. In the next few weeks, we'll get into adjusting those Cross-Regional Comparisons, and of course, assessing who actually could win the CrossFit Games. Until then, enjoy your training, and I'll see you back here soon.

Regional Predictions, Week 4

Although last week's regional competitions had their share of drama, I'll admit it felt like a bit of a letdown after the amazing weekend prior. Thankfully, this fourth and final weekend looks like it has some fantastic stuff in store. I mean, how could Northern California's men's competition not be insane? We have seven former Games competitors vying for three spots, including three men who finished in the top 10 at the Games last year. Like we saw in the Central East, there will be some men not heading to the Games that probably would have gone in virtually any other region in the world.

With that in mind, let's get to some assorted topics before we move onto the predictions for week 4:

• Taken in a vacuum, I don't have a problem with Dave Castro's statement (speaking for HQ I presume) that there will not be any wild cards given out this year. However, in context of this season, I'm not a fan.
• Why even announce that wild card spots will be available (which they did earlier this year) if you are going to rule out that possibility before the Regionals have even finished? I cannot conceive of a scenario where wild cards would make more sense than they do for Sam Briggs this year. She is the reigning Fittest Woman on Earth, she had a single bad event in one of the most volatile events ever programmed at Regionals (1-attempt handstand walk) and she still finished fourth in a stacked region. If you're not going to use a wild card in that situation, then you're never going to use it.
• Talent is so clearly bunched in a few regions (and has been for a few years). I can understand the argument that the regionals are set up with a limited number of spots in each region to increase drama and make things more exciting. However, I find it difficult to accept the argument that this system is ideal for finding the fittest athletes in the world. I get that cross-regional comparisons are not perfect, but I challenge anyone to argue that Graham Holmberg (4th in Central East) is not among the 40 best CrossFitters in the world. As it stands now, he is ranked ahead of the champions from 9 other regions. For Castro to argue that "the right athletes" are going to the Games seems a bit disingenuous. If you're just setting it up this way for drama, that's fine, but let's just call it what it is.
• Although we have one week to go, the data from across all regions has allowed me to get a sneak peak at some interesting things from this year's regionals.
• In terms of correlation with success across all Regional and Open events, it appears that events 3 and 7 are the top events at this point. I'll admit when I was wrong, and I was wrong on event 7. The top athletes are all crushing it, and it is damn exciting. Event 3 is a bit surprising, but again, look at the athletes who are doing well there, and they're usually dominating across the board.
• On the other end of the spectrum, event 5 for the men actually has the lowest correlation with overall success. My guess here is that this is the one event this season that truly favors taller athletes, and so you are seeing some athletes with huge performances who otherwise are struggling. For the women, this event is not so bad, mainly because there are no athletes jumping 10-11 feet in the air and getting to the top of the rope in a couple pulls.
• Not surprisingly, the two single-modality events (1 and 2) are among the least correlated with overall success for both men and women. Event 2 is slightly worse than event 1, but not by as much as you might think.
• Events 4 and 6 are kind of middling in this respect. I expected event 6 to really bring out the top all-around athletes, but it might just be so grueling that it heavily favors the endurance specialists.
• If we look at Open events in this context, 14.3 has the lowest correlation with overall success among Regional athletes (as it did for the entire Open field). On the other side, 14.4 was the highest correlation with overall success among Regional athletes (as it did for the entire Open field). In fact, it is basically neck-and-neck with Regional event 3 for the top spot across all events this season.
• Some have suggested that results in the handstand walk might be correlated with success in event 4 (which has tons of handstand push-ups). It doesn't appear that way; ranks on those two events are not particularly correlated (52% for men, 44% for women - both of those figures are middle of the road this respect). The only combination of events that really stands out is events 1 and 7, which were 77% correlated for women and 68% correlated for men.
• Last week I posted a chart and some statistics regarding the accuracy of my predictions (I should note that these are after removing athletes who withdrew prior to event 1). After week 3, the calibration plot looks about the same, but the mean-square error has dropped from 4.38% to 3.93%. For reference, last year's model was 4.43% and a model giving each athlete an equal chance would be about 6.40%. Below is the calibration plot (read last week's post for an explanation):

Alrighty... with all that out of the way. Let's get onto the predictions. This week, the only athlete for whom I made a manual adjustment to the model was Jason Khalipa. This year's events might not really favor him, but the guy has been so freaking consistent over the past 6 years that I felt he warranted special consideration.

With that said, here you go. Enjoy the final week of Regionals, everyone!

[Update 5/31: I've made a couple fixes to account for women's name changes since last year, as well as making the adjustment for Andrea Ager that I suggested in the comments a couple nights ago. I treated her as if she did not compete at Regionals last year, rather than as if she finished very low. Her low finish was due to a DQ in the OHS event, not due to a poor performance overall.]

Note that Africa only has one qualifying spot. All other regions this week have three.

Also note that the pictures look prettier this week because I'm posting from a Mac. Excel is terrible on a Mac, but at least it exports nicely to pictures.

Regional Predictions, Week 3

I've said before that I have no doubt the CrossFit Games is a totally viable spectator sport. My opinion on the matter hasn't changed, but I'm beginning to think that it's the Regionals that should be on ESPN. And I doubt anyone who followed the coverage on the Games site this past weekend would disagree with me.

There was high drama all weekend, and with 10 different competitions to follow at various time zones across the globe, events were broadcast basically non-stop. We had the reigning fittest woman on Earth fighting just to make the Games in Europe, possibly the most competitive men's competition ever in Central East, a changing of the guard in Australia and the first real challenge to Camille and Michelle Letendre's dominance in Canada East. So before we move on to predictions for this week, let's get to some quick thoughts on week 2:

• The handstand walk claimed another victim this week in Sam Briggs. I'm sure HQ would never admit that the programming was anything less than perfect, but anyone following that competition knows that Briggs was one of the three fittest women there, and likely the fittest. She dominated events 3-6 and placed decently on the two heavier workouts (1 and 7).
• That being said, she will almost assuredly get a special invite, and the three women who did qualify certainly deserved it. Along with Southern California, Europe looks to be one of the two strongest women's regions in the world (Australia also looked sneaky-tough last weekend, too).
• I certainly hope that HQ also extends a special invite to Graham Holmberg. We'll have to wait to see how the rest of the competitions play out, but I'd be he would have qualified in every other region (and likely won many of them). He came through when it counted most, smashing the event record in event 7, but unfortunately, four other men in his region also beat the event record (including third-place Will Moorad). He was also the only man in that region other than Rich Froning to take an outright first place, and he did it twice.
• Super-impressive performance by Moorad to grab a spot in the Central East. As disappointed as I was to see Graham fall off, it is nice to see someone else break through in that region.
• If the Games were programmed like the Regionals, Camille Leblanc-Bazinet might be the most dominant athlete in the sport. If the event has barbells and gymnastics, she's basically guaranteed a spot in the top 10 in the world. Hopefully she can fare a bit better with the more unorthodox Games events this year.
• While I'm not convinced that event 7 is as good a test of fitness as, say event 4 or event 6, but I can't dispute that it is great for the viewers. Those 8 overhead squats have derailed more than one athlete's shot at the Games and it's given several others the chance to make up ground in dramatic fashion.
• Event 6 looks like an absolute beast of a workout. Unlike prior years, I haven't been able to test out the workouts (due to a back injury), but I don't recall seeing Rich Froning that gassed in a workout since the rope climb/sled push workout at the 2012 Games. 

So how did my predictions do last week? Well, despite a couple of shockers, things actually pretty well. The charts below show how well the predictions were calibrated this year compared to last year. The athletes were bucketed based on their predicted rank, and each bucket was plotted as a poitn on the blue line.  The location on the x-axis represents my predicted chances of qualifying and the location on the y-axis represents the actual chances of qualifying.  The red line represents perfect predictions, so when the blue line is below the red line, my predictions over-estimated the chances of qualifying for those athletes.  As you can see, this year, the blue line tracks much more closely to the perfect predictions.

Of course, calibration only tells half the story. We could easily create a well-calibrated model by predicting an even chance for each athlete. If there are 30 athletes in a region with 3 qualifying spots, we could estimate that each athlete has a 10% chance of qualifying, and indeed, we would be right in some sense. It is sure that 10% of them will qualify. But we are also trying to be accurate. A perfect model would predict 100% chances for the 3 athletes that do qualify and 0% for all others. To measure our accuracy, we can calculate the means square error across all our athletes. In that respect, I did about as well as last year, and considerably better than the perfectly calibrated model with even predictions for each athlete.

• 2014 Week 2, CFG Analysis Predictions - 4.38%
• 2013 Weeks 3-4, CFG Analysis Predictions - 4.43%
• 2013-2014 Equal Chance Predictions - 6.40%

So, with that in mind, I really didn't make any changes to the model this week. The only region where I had to deviate significantly was for the women in Asia. There are no returning Games qualifiers, hardly any returning regional competitors and just a handful of athletes in the top 2000 in the Open worldwide. My default model would have basically given all athletes the same chance, so I modifiied it as best I could, but the predictions are still pretty weak in that region.
[UPDATE 5/23/2014: In the Asia region, I just realized Candice Ford was Candice Howe last year, which meant I originally assumed she did not compete at the regionals last year. The predictions below are now fixed to account for this.]

Anyway, without further ado, here are my week 3 predictions. Enjoy the Regionals everyone!

Note that Asia only has one qualifying spot. All other regions this week have three.

Regionals Predictions, Week 2

Welcome back, everyone. The first week of the 2014 Regionals is in the books, and in many ways, I think it played out like I expected. The handstand walk derailed one heavy favorite (Stacie Tovar), all of the athletes I picked to fare well did (Lucas Parker, Elizabeth Akinwale, Talayna Fortunato) and the weekend as a whole seemed to favor athletes with strong gymnastic abilities and pure strength, rather than those with the biggest engines. I was pleasantly surprised by event 7, which appeared to be more balanced than I expected and provided some exciting shake-ups in a few regions.

This week looks to be one of the most (if not the most) exciting weeks of the Regional schedule. We all know that the Central East men's region is the toughest in the world, but the European women's region and the Central East women's region are also super-competitive as well this year. It's also a little intriguing to see which "records" will fall now that the bar has been set on all the events. Personally, I think every single one of the men's records will go down (potentially all in the Central East?) and many of the women's records will go down (doubtful that Akinwale's records in events 1 and 2 will fall).

With that in mind, let's get down to the business at hand. Last week, I made a bunch of excuses for why I wasn't able to get formal predictions finished in time, but this week I was able to make it happen. I've been able to estimate the odds of qualifying for each athlete in all 10 competitions, and at the bottom of this post I've shown the odds for the top contenders in each region.

But first, here's a recap of the methodology, which is largely similar to what was done last year:

• Learn from prior results
• Separate the 2013 Regional competitors into various categories, based on their performance in the 2013 Open, 2012 Games and 2012 Regionals.
• See how frequently athletes in each category posted a very high (top 20 worldwide) or relatively high (20-50 worldwide) regional performance in 2013 (based on the cross-regional rankings last year).
• Repeat the first two steps one year further back. Combine the results with what I came up with in the first two steps. This helped me get a bigger sample size and hopefully improve the predictions.
• Apply learnings to what we know this year to make predictions
• For each athlete this year, place them in one of the categories based on their performance in the 2014 Open, 2013 Games and 2013 Regionals.
• Depending on their category, randomly generate a worldwide ranking for each athlete this year. The category affects this randomized worldwide ranking, i.e. those who had better results in the past year will generally get a better randomized worldwide ranking this year.
• Re-rank all athletes within a region based on these randomized worldwide rankings.
• Repeat 200 times and see how often each athlete qualifies for the 2014 Games.

Now, for those so inclined, here are a few details on this process:

• To get a large enough sample size to build this model, I combined men and women.
• The process for creating the categories of competitors was not straightforward. There was quite a bit of judgment on my part to make sure that each category had sufficient athletes to be credible and that the categories produced results that made sense with each other. For instance, I wanted to separate out the top 2012 Games competitors (I chose top 15), but that meant I could not further break those athletes down based on 2012 regional rank, because there just would not be enough athletes there to get a credible sample.
• The process for randomly generating the numbers is as follows:
• Generate a uniform random number between 0 and 1 (=rand() in Excel). If the first is lower than the athlete's chance of finishing in the top 20, assign him or her to the top 20. If not, then if it is lower than the athlete's chances of finishing in the top 50, assign him or her to be between 20-50. Otherwise, the athlete is assigned to be between 50 and 100.
• Once we have assigned the athlete to the a range of ranks, generate another uniform random number between 0 and 1. Multiply this by 20 to get the athlete's exact place within the range (multiply by 30 if they are in the 20-50 range or multiply by 50 if they are in the 50-100 range). Generally, you'll need to be in the top 50 worldwide to qualify, but depending on how other athletes fare, it's possible to end up in the 50-100 range and still be in the top 3.
• Here is a lifting of the categories I used to break down the athletes:
• Top 15 at prior Games
• Below 15 at prior Games, top 40 worldwide at prior Regionals
• Below 15 at prior Games, below 40 worldwide at prior Regionals
• Did not make prior Games, top 50 worldwide at prior Regionals, top 100 in current Open
• Did not make prior Games, top 50 worldwide at prior Regionals, below 100 in current Open
• Did not make prior Games, 50-100 worldwide at prior Regionals
• Did not make prior Games, below 100 worldwide at prior Regionals, top 250 in current Open
• Did not make prior Games, below 100 worldwide at prior Regionals, below 250 in current Open
• Did not make prior Games, did not compete at prior Regionals, top 75 in current Open
• Did not make prior Games, did not compete at prior Regionals, 75-150 in current Open
• Did not make prior Games, did not compete at prior Regionals, below 150 in current Open

Last year, my predictions weren't bad, but they generally overestimated the chances for the athletes on the low end and high end, but I underestimated the chances for the athletes in the middle. Consider:

• Athletes predicted 0-10% - 2.5% expected to qualify, 0.4% qualified
• Athletes predicted 10-50% - 20.7% expected to qualify, 39.5% qualified
• Athletes predicted 50-100% - 66.3% expected to qualify, 57.1% qualified

I have more data to train the model this year, which should help to calibrate things a little better, and I was a bit more liberal in applying some manual adjustments to some elite athletes. For instance, Julie Foucher did not compete last year, but I treated her as if she had finished in the top 15 in the Games. As a three-time top 5 athlete, I think this is only fair. Other athletes for whom I made at least some adjustment included Rich Froning, Samantha Briggs, Annie Thorisdottir, Frederick Aegideus and Camille Leblanc-Bazinet.

So with that in mind, below are my predictions for week 2 (athletes with less than 5% chance are not shown). As always, keep in mind that this is all in fun, and it's all simply based on the numbers. I'm not making any sort of judgment about the effort these athletes have put in, I'm simply reflecting how athletes in similar situations have performed in the past. Enjoy week 2, everyone!

*Note that Canada East only has two qualifying spots. All other regions this week have three.

Quick Regional Programming Thoughts

As much as I was hoping to have my regional predictions set up to go this week, I wasn't able to make that happen. A confluence of events over the last month - a 5-hour actuarial exam, a trip out of town to have my son baptized, my wife's birthday and numerous trips to the chiropractor/ART to try to fix some back issues that flared up after the Open - pretty much made that impossible for me. By next week, I do expect to be able to produce my stochastic regional predictions (like I did for the final two weeks of last year's season). But for this week, your guess is as good as mine about who will claim the first 24 spots at the Games.

That being said, I did have time to do a bit of work assessing the programming for this year's Regionals. Here are my thoughts:

• Overall, I like the programming. In particular, I think events 3-6 seem to me to be well-balanced workouts that should be fun to watch.
• Events 1 and 2 introduce a ton of volatility into the situation. Only getting 3 attempts on the hang snatch means we could see some top athletes get burned by taking a gamble on those 2nd and 3rd lifts. And a max handstand walk on a single attempt gives plenty of opportunity for a catastrophic failure that could cost an otherwise fit athlete a shot at the Games. In my opinion, I don't think this event really helps us find the fittest athletes, and it may end up preventing some really stellar athletes from making it.
• Event 7 is really a wait-and-see event for me. It seems like a really weird design for a workout to have just 8 reps of the overhead squat, since it doesn't seem like it gives enough time for athletes to make up ground on that movement. But hopefully I'm wrong and this event turns out to be a better test of fitness than it appears to be on paper, especially considering it's the finale.
• This year's regional is in some ways the heaviest regionals to date and in some ways the lightest. When weights are involved, the average relative load (1.55 men, 0.98 women) is the highest in the past four years. However, the programming is only 37% lifting, the smallest percentage of any Regionals. In fact, the 2011 Games is the only HQ competition with a lower percentage (34%).
• When you combine those two factors, you get an load-based emphasis on lifting (LBEL) of 0.58 for men and 0.36 for women, both slightly lower than 2011 and 2013 and much lower than 2012. The chart below shows the progression of each of these metrics at the Regionals since 2011.

• All-in-all, I believe this year's Regionals look more like the Games than in any past year. And what that means to me is that the emphasis is on strength, both in terms of weightlifting and very challenging bodyweight movements (like legless rope climbs or strict handstand push-ups). Don't get me wrong, you can't do well here without a high level of conditioning, but you will be punished much harder for lacking in strength.
• Look for handstand push-ups and legless rope climbs to completely decimate the women's leaderboard. With the legless rope climbs, we saw how much variability there was at the Games last year. As far as handstand push-ups, keep in mind that before kipping became common (think 2011 and earlier), this was an extremely difficult movement for many top women, even at the Games.
• With that in mind, here are some athletes that should do well this programming: Lucas Parker, Josh Bridges, Lacee Kovacs, Chris Spealler, Matthew Fraser, Elizabeth Akinwale, Talayna Fortunato, Annie Thorisdottir, Camille Leblanc-Bazinet.
• Beyond those names, all the podium athletes from last year's Games should do fine. I'm just not sure they will fare any better because of this programming.

That's it for today. I hope you all enjoy the opening weekend of Regionals, and I'll see you again next week.