Follow me on Twitter!

Thursday, April 18, 2013

A Look Back at the 2013 Open: Part I

After a somewhat longer-than-expected break, I think it's high time to get look back at the 2013 CrossFit Games Open, in its entirety, and see what we can learn. For sure, there will still be plenty of stones left unturned, in part because of limitations in the data I've been able to get to this point, but also because I believe the Regionals and Games can give further insight into the Open. For instance, last year I looked at athletes who competed in all 6 events at regionals and compared their results across all 11 events to that point. This allowed me to see which individual events were predictive of success across a wide variety of events. Obviously, we can't do that quite yet, at least not to that extent.

That being said, I think there is enough data out there to break things down adequately and understand more about the state of our sport and possibly where we're headed. Due to the amount of material, I'll be breaking this post up into two parts. To start, here is a list of topics I plan to cover, followed by a list of things I will not be touching on in this post:

Will cover:
  • Breakdown of the programming of this year's Open, much like last year's post "What to Expect from the 2013 Open and Beyond" (Part I)
  • Correlations between events this year, compared with last year (Part II)
  • Comparison of performance by new competitors vs. returning athletes (Part II)
  • Comparison of 12.4 and 13.3 results, in a fair amount of depth (Part II)
  • Attrition in this year's Open, compared with last year (Part II)
Will not cover:
  • Comparison between regions (don't have region information on the data at the moment)
  • Breakdown by age group (don't have age information, either)
  • Predictions for regionals
  • Probably lots of other subjects that I simply didn't think of. If you have suggestions for future analysis, by all means, post to comments or email me.
Finally, here are some notes on the data set I am using for any work dealing with the results of the Open:
  • Excluded any athletes who did not complete all 5 events. This simply makes for fairer comparisons. I did look at all scores in order to calculate the number who dropped off each week, but that is it.
  • Masters competitors are lumped in with everyone else. As mentioned above, I don't have age information this dataset since I pulled it straight off the worldwide leaderboard. This is not ideal, but I made sure to do the same when looking at last year's data to make comparisons. Only about 20% of the field are in the Master's divisions, with only 2% in ages 55+ (where the workouts are slightly scaled).
  • I have re-ranked athletes on each event among the athletes in this dataset.
  • Athletes were identified as returning athletes if their full name was in last year's dataset. There are multiple athletes with the same exact name, but I had no way around this without region or age information. I assume any impact here is minor. The one manual fix I made was to make sure the Ben Smith at the top of the leaderboard was matched up with the correct Ben Smith from last year's data. 

OK, with that out of the way, let's get rolling.

We'll start with the programming this year. As I mentioned in my prior posts, I felt this year's programming better was an improvement over last year, if for no other reason than we eliminated the single-modality events. I also felt the events this year were balanced, with specialists unlikely to finish particularly high on any given event, but yet there was enough diversity that we weren't testing the same thing over and over again. As I started looking into the programming further, it became clear that 2013 was, in many ways, a blend between 2011 and 2012. First, here is a basic comparison of the average loading* used each year in the men's competition (the pattern is the same for women).

The average relative weight was down slightly, but very much in the same neighborhood as previous years, while the percent of points from lifting and the load-based emphasis on lifting (LBEL) were both basically equal to the average of the previous two years. Now let's take a look at which movements** have been used across the three years, and how they have been valued (1.00 equals one full event).

You may notice that we have not introduced a single new movement since the Open began in 2011. I was glad to see we brought back the clean and deadlift this year, but notice which movement is at the bottom: overhead squat. In my mind, this is the quintessential CrossFit lift, and yet it's accounted for only 2% of the points in the Open over the past three years. Sad but true.

You'll also notice that in general, the Olympic-style lifts and derivatives (thruster, overhead squat), as well as basic gymnastics movements, are the biggest keys to Open success. Running, rowing, powerlifting, kettlebells, wall balls, high skill gymnastics, strongman lifts - these are all of minor importance until you reach beyond the Open. If you want to make Regionals, work on your snatch, clean and jerk, burpees, thrusters and pull-ups. If you can't do those things extremely well, it doesn't matter if you can bang out 25 consecutive ring handstand push-ups or lift a 300-lb. atlas stone. It doesn't even matter what your time is on a 5K run. That's not to discount the usefulness of these other skills in training; it's just that you're not likely to see that tested until at least the regionals.

Finally, here's a chart I put together showing the relationship between loading, the number of movements, and the length of workout in the past three years of the Open. In the past, I probably haven't spent as much time as I should looking into the time domains in workouts across the Games season, partly because after the Open, the workouts often have a set workload, not a set time. But for the Open, we know the time domain exactly***. In the chart below, the x-axis represents the time domain, the y-axis represents the number of movements and the size of each bubble represents the LBEL of that particular workout (roughly how "heavy" was each workout). Note that I considered 11.3 a single-modality despite technically being a clean and jerk.

What we see is fairly typical of CrossFit programming. The fewer movements are involved, the shorter the workout is likely to be. In training, this is generally true because of issues like Rhabdo that come into play when you hammer one muscle group too much. Also, the relationship isn't quite as strong, but typically the heavier the load, the shorter the time domain. Again, I think part of this is simply being smart and safe, since going heavy for an extended period of time lends itself to potential injury.

That's it for Part I. In Part II, I'll be focusing more on the results of this season's Open. See you soon.

*For background on these metrics, please see my post "What to Expect from the 2013 Open and Beyond." Just as I did last year, the average weight load on the burpee-snatch workout was calculated based on the average score from the regional-level competitors. Therefore, that workout was considered fairly heavy, despite the fact that many beginner and intermediate athletes would not lift more than a 75/45 lb. snatch.
**For 13.4, I considered the toes-to-bar to be worth 50% of the workout, the clean to be worth 25% and the jerk to be worth 25%.
***For 13.5, which had a varying time limit, I used the average time spent for athletes in the top 1,000 worldwide (roughly the regional competitors). This turned out to be 8:00 for men (seen in the chart) and about 5:20 for women.


  1. An earlier analysis you gave estimated the number of points an athlete could surrender and still place top-48 in their region. That estimate ended up being far off; the best from each region were very consistent each event.

    Would you say someone can have one bad (relatively speaking .. maybe 85th percentile) event and still make Regionals?

    In your opinion, are the high correlation coefficients of individual event finishing order and overall finish order indicative of dominant top-tier athletes in a balanced test of fitness or is it indicative of an unbalanced test of fitness? Unbalanced because the same individuals are placing high in each event. For example, what would happen to overall order if a 10k and power lifting total were added to the mix (logistics notwithstanding)?

    1. Matt,

      In response to your first question (could someone have one bad event and still make regionals?), it depends on the region, but generally speaking, no. In the Central East, there were roughly 4800 men to start, so finishing 85th percentile in one event would give you 700+ points and give you no shot to recover. Even in 13.5, after the field slimmed down, 85th percentile was still worth about 500 points. Now, if you're asking whether I think an athlete SHOULD be able to get away with that, I'd say probably not. I was 300th-ish in my region at the end of the day and never finished below 506 in an event, and I don't consider myself a legitimate threat to make regionals, so I'd think our regional competitors should have to be at least 90th percentile across the board. Of course, last year you certainly could have gotten away with finishing 85th or even 80th percentile on an event.

      Anyway, I think the second part of your question is more intriguing. Were the point totals of the top athletes low because the Open was unbalanced? I personally say no. I didn't write about it (maybe I should have), but I looked at the correlations between each event to see if any two events were too similar. Among the top 1000 athletes, no two events were more than 26% correlated for men and 40% for women (13.1 and 13.4). So it's not like we were repeating the same thing over and over again. However, there weren't any events that were significantly NEGATIVELY correlated either - last year, 12.1 and 12.2 were -26% correlated among the top 1000, meaning that those who did well on one typically did worse on the other. I don't have necessarily have a problem with that on its own. But to me, the fewer skills you test per event, the more events you have to have. So if the Open had 10 events, then sure, throw in two single-modalities that are totally opposite. But with only 5 events, let's try to pick the 5 events that tell us the most about the athletes. I think this year HQ did a better job of that.

      If we threw a 10K run and powerlifting total in ON TOP of the five events we had, I think it wouldn't be a bad thing. But to replace two of the events we had with those would probably not be an improvement, in my opinion.

      Great question, though. Feel free to let me know what you think on the matter as well.