Saturday, April 19, 2014

A Look Back at the 2014 Open: Part I

As I did last year around this time, today I'll be starting my review of the 2014 CrossFit Games Open. Of course there will be limitations to what this analysis will be able to cover, partly due to the data I'm able to get at this point (I am hoping to eventually pull down all the individual stats, like Fran time, max deadlift, etc.). Still, I think there is enough data out there to help us further our understanding of the current state of our sport and where we may be headed. Like last year, I'll be breaking this post up into two parts. To start, here is a list of topics I plan to cover, followed by a list of things I will not be touching on in this post:

Will cover:
  • Breakdown of the programming of this year's Open, much like my "What to Expect from the Open" posts from fall 2012 and fall 2013 (Part I)
  • Correlations between events this year, compared with last year (Part II)
  • Comparison of performance by new competitors vs. returning athletes (Part II)
  • Comparison of 11.1 and 14.1 results (Part II)
  • Attrition in this year's Open, compared with past years (Part II)
Will not cover:
  • Comparison between regions (don't have region information on the data at the moment)
  • Breakdown by age group (don't have age information, either)
  • Predictions for regionals (coming in the next few weeks)
  • Probably lots of other subjects that I simply didn't think of. If you have suggestions for future analysis, by all means, post to comments or email me.
Finally, here are some notes on the data set I am using for any work dealing with the results of the Open (thanks again to Andrew Havko for helping me pull this data, along with the 2011 Open results, which I've been wanting to get my hands on for some time):
  • Excluded any athletes who did not complete all 5 events. This simply makes for fairer comparisons. I did look at all scores in order to calculate the number who dropped off each week, but that is it.
  • Masters competitors (54 and under, since older groups are scaled) are lumped in with everyone else. As mentioned above, I don't have age information this dataset since I pulled it straight off the worldwide leaderboard.
  • I have re-ranked athletes on each event among the athletes in this dataset.
  • Athletes were identified as returning athletes if their full name was in last year's dataset. There are multiple athletes with the same exact name, but I had no way around this without region or age information. I assume any impact here is minor. The one manual fix I made was to make sure the Ben Smith at the top of the leaderboard was matched up with the correct Ben Smith from last year's data. 
So let's get started.

We'll start with the programming this year. I generally liked the programming this year (although it certainly didn't play to my strengths), mainly because HQ finally threw us some curveballs and some workouts that I didn't expect. Among the things we saw this year that hadn't occurred in previous Opens:

  • Rowing
  • A workout for time (rather than AMRAP)
  • A workout with more than 3 movements
  • Weights over 300 pounds for men and 200 pounds for women
  • Pull-ups and thrusters not in the same workout

Having said that, the Open is still the Open, and many things remained the same or similar as prior years. For instance, the loading was still much lower than the Regional and Games level. In fact, by my measurements, this was the lightest Open yet. Below is a basic comparison of the average loading* used each year in the men's competition (the pattern is the same for women).

The average relative weight was down from all prior years and there less than 50% lifting, which meant that the load-based emphasis on lifting (LBEL) was down about 10% from the historical average**. An investigation for another day is whether this Open favored smaller athletes because of that lower LBEL.

Now let's take a look at which movements have been used across the three years, and how they have been valued. This is presented slightly different than last year: each value represents how much that movement was worth as a percentage of the total for that season. The reason for presenting it this way (as opposed to counting total events) is that 2011 had six events and the other years had five, so this accounts for the fact that each event was not worth as much in 2011.

We see that this year, the programming hit almost every movement that has been used at any point in the past (push-ups and jerks were the exceptions) and added one new movement (rowing). Not surprisingly, we see the same movements being emphasized as in prior years: snatches, burpees, thrusters and pull-ups. One interesting note is that this is the first year in which no movement has accounted for more than 10% of the total points. However, the caveat here is that this methodology assumes all movements in a given workout are valued equally. In reality, there are instances where this is not necessarily true: for instance, most people would agree the deadlifts were valued far more than the box jumps in 14.3

As in past years, you'll also notice that the Olympic-style lifts and derivatives (thruster, overhead squat), as well as basic gymnastics movements, were the biggest keys to Open success. However, we did see a bit more value placed on other areas, such as powerlifting (e.g. deadlift) and pure conditioning (double-unders, rowing). Although Castro did surprise us with a few things this year, I still think it's a safe bet that the more advanced movements you might see at the Games (e.g. ring handstand push-ups, heavy medicine ball cleans) are not going to be tested in the Open. That's not to discount the usefulness of these other skills in training; it's just that you're not likely to see that tested until at least the regionals.

Finally, here's a chart I put together showing the relationship between loading, the number of movements, and the length of workout in the past three years of the Open. This chart was shown last year, but I have added the 2014 workouts, which are represented by the red balls. In the chart below, the x-axis represents the time domain, the y-axis represents the number of movements*** and the size of each bubble represents the LBEL of that particular workout (roughly how "heavy" was each workout). A plus-symbol indicates the weight varied during the workout and the arrows indicate the time varied for the workout****.

This year's workouts, although unique in how they were programmed, still didn't stray too far from what we've seen in the past in terms of loading and time domain. Keep in mind that for the above chart, I'm using averages for the variable-weight event (14.3) and variable-time events (14.2 and 14.5). For many CrossFitters, they did see a very long workout in 14.5 (which took many people beyond 20 minutes) and a very short workout in 14.2 (which only lasted 3-6 minutes for a majority of the field).

You can still observe some general trends here, which are often true of CrossFit programming in general. The shorter workouts tend to involve fewer movements and can occasionally go heavy, while longer workouts can potentially involve 3 or more movements but generally have light-to-moderate loading.

That's it for Part I. In Part II, which I expect to be out this week, I'll be focusing more on the results of this season's Open. See you soon.

*For background on these metrics, please see my post "What to Expect from the 2013 Open and Beyond." You may notice that the loading for prior years has changed slightly, which is due to me updating the relativities between lifts as I gather more data.
**For any workout with a variable element, such as the weight in 14.3 or the time in 14.2 or 14.5, I used the average of the top 1000 athletes. This is consistent with prior years.
***I considered the 11.3 a single-modality for this chart even though it technically included cleans and jerks.
****Workout 13.5 is actually hidden from view here because 14.3 is covering it up (same time domain and number of movements). Workout 13.5 was also a variable-time workout and would have had the arrows on the ball.


  1. I doubt there is the data set for this, but I would be interested in some type of comparison of the training programs developed by some of the big coaches in the sport. For example, I'd be interested to see a comparison of how athletes who follow the competitor style program offered by Invictus, Outlaw, and CFNE fared in the open.

    Were there any clumping points in this years Open? A few years ago, the bar burpee, OHS, and muscle up wod had a lot of competitors clumped around either 0 muscle ups or a single muscle up. I don't think that happened as much this year, but am I wrong?

    1. I think it would be really challenging to get the data to support an analysis of your first question. Unless you could know of EVERYONE who followed each program, you'd be setting yourself up for a lot of selection bias if you just looked at people who you KNEW were on a certain program. People who didn't do as well probably aren't as likely to say they followed a particular program, and the programmers also wouldn't be as likely to claim them. That being said, it would be really interesting to see. Maybe for just the Games athletes, something like this could be done.

      For your second question, that's the type of thing I'll be covering in Part II. Obviously 14.4 would probably have some of those same types of issues at the muscle-up point. One thing I'll look into is whether a higher % of people who reached the muscle-ups were able to actually get one. I will say that HQ's tiebreaker method makes the clumping a lot less of an issue than in 2011 and 2012.