Quick Update - Predictions Coming Soon

Hello all. I just wanted to drop a quick post to say that it's been a busy week, but my predictions for the Games will be forthcoming soon, likely this weekend. I'll be estimating the likelihood of each individual athlete winning the whole thing, placing in the top 3 and placing in the money (top 10). The process is a bit more complex than last year, but I think it should be pretty neat.

I'm also curious to see what you guys are thinking about this year's Games. Seems pretty clear that Froning will be the favorite on the men's side, but the women's side is wide open. Feel free to post thoughts to comments here or on my next post, after I make my predictions.

The Games are coming up on us soon (potentially under 2 weeks, depending on when the competition actually starts), and I'm pumped to get out to L.A. to watch. In the meantime, good luck with your training!

Which Have Been the "Best" Events This Season?

If you've been reading my blog for any length of time, you know that one of the ways I like to evaluate the effectiveness* of a CrossFit event is by looking at how well the results from that event correlate to results from a variety of other events. I laid out the theory behind this in my post from last year titled "Are certain events 'better' than others?", but I'll recap it here:

• In a competition setting, what we are trying to do is learn as much about each athlete's overall fitness level as possible.
• We only have a limited number of events to do this, particularly in the Open and at Regionals. Therefore, we need to maximize the information we get from each event.
• If athletes who score well on a particular event tend to score well across the board, then that event is probably a good indicator of overall fitness. Conversely, if the results from that event don't correlate at all with results in other events, then maybe that particular event did not really tell us much.

Overall, this year's regionals and Open were set up well for me to do this type of analysis. I did this same analysis after last year's regionals, but due to the cuts after regional event 5, I was left with only about 250-300 athletes of each gender who completed all the Open and Regional workouts, and this limited the analysis to only the very elite athletes. I also did this analysis after the Open this year, which gave me a huge sample of athletes, but with only 5 events, I didn't really have a "wide variety" of events to evaluate. 

This year, because we did not have any cuts at Regionals, I was left with 673 men and 512 women who have completed all 12 events this season. Although these are all still very solid athletes, I got a lot more of the borderline regional competitors in the mix than I did last year. Remember, a "good" event for the Games might not make a "good" event for a competition within your own box. So keep in mind with this analysis that we are evaluating these events based on how well they predict overall fitness for Regional competitors.

The methodology for performing this analysis this year was the following:

• For athletes who completed all events this year, compile their results (not just their ranking) for each of those events. The Regionals results I used are those that have been adjusted to account for the week in which week each athlete competed. See my previous post for more info on those adjustments.
• Rerank the entire field in each of those events.
• For each event, calculate the sum of each athlete's ranks in all other events.
• Calculate the Pearson correlation (referred to from here on out as simply "correlation") between the ranks in each event and the sum of ranks in all other events. Higher correlations indicate "better" events.

Below are the results for both men and women.

The pattern that emerges is one I've noticed pretty much across the board since I've been doing this type of analysis: events with more movements tend to be better tests of fitness. This makes sense intuitively, since they test more things, by definition. That doesn't mean we shouldn't have single-modality events in competition, it just means they probably should be used sparingly and only for movements that are deemed very important.

You may notice that Open Event 2, which had three movements, is bucking this trend by falling quite close to the bottom. The concerns with this event were pretty well documented during the Open: judging was very difficult, the weights were extremely light for top competitors, and the option of step-ups was utilized by a lot more athletes than was probably expected. So simply because you have 3 or 4 movements in an event doesn't make it a great one.

To get a visual interpretation of the concept I'm getting at here, below are scatter plots for some of the best and worst events from this year. On each graph, the x-axis represents each athlete's rank on that event and the y-axis represents each athlete's combined rank on all other events.

It should be fairly clear that for the first two graphs, there is a clear relationship between the x- and y-axis. Athletes who did well on these events generally did well across the board. On the third graph, the points are much more scattered, indicating that there were plenty of athletes who did well on Open Event 2 who didn't fare well across the board, and vice versa.

Although I consider the results using the entire Regional field to be the most useful, I also tested performed this analysis on three other subsets of the regional field:

1. Games athletes only
2. Top 292 men and top 258 women (same number as I had in my analysis last year)
3. Random sample of 20% of the entire regional field

What I was interested in was how volatile my results were. For instance, is Regional Event 1 really better than Regional Event 4 for women (76% vs. 74%)? Would that hold up if I changed the group of athletes a bit?

In general, the events near the very top and the very bottom stayed in that vicinity, with a couple of exceptions. Here are the main takeaways:

• For men, Regional Event 4 was in the top 3 across all the samples and Open Event 1 was in the top 5 across all the samples. For women, Regional Event 4 was in the top 4 across all the samples, Regional Event 1 was in the top 5 across all the samples, Open Event 1 was in the top 5 across all the samples and Open Event 4 was also in the top 6 across all the samples.
• Considering HQ is programming the same events for both men and women, I would conclude that Regional Event 4 ("The 100s") and Open Event 1 (AMRAP 17 of burpees and snatch) were the best events this season. In one of my 2013 Open recap posts, I noted that 13.4 was generally the best event of the Open. I still feel that it was a very good event for the entire Open field, but it wasn't quite as strong when we look at just these stronger athletes. 
• Across both men and women, Open Event 2, Regional Event 5 and Regional Event 3 were each in the bottom 4 in all but one sample. I would conclude that these were generally the three weakest events this season. 
• I mentioned some issues with Open Event 2 above.
• For Regional Event 5, I think the issue is that we saw a lot of athletes near the bottom of the field do well on this simply because they could deadlift a house. If you could handle the deadlifts easily, you could generally do well even if you weren't particularly great at box jumps or had sub-par aerobic capacity.
• For Regional Event 3, I think the issue was that the burpees did not really factor into this much, making it basically just a muscle-up test. As far as single-modalities go, this wasn't too bad of an event. But personally, I think this event and the overhead squat ladder (a true single-modality) should have been worth only 50% of the other events.  
• Two of the events featuring box jumps turned out to be relatively modest tests of fitness. Throw in all the complaints about achilles problems we've seen popping up recently, and I think HQ may want to look into adjusting how they program box jumps. I think box jumps are a good test of fitness in general, but I'd personally love to see us go to box jump-overs (onto and over the box with an option to jump straight over) in the future.
• With the exception of Open Event 2 and Regional Event 5, I think the rest of the events were generally solid. As I mentioned above, I might consider adjusting the point value for a couple of the other ones.

I also did one final analysis, primarily out of curiosity. Using the entire Regional field, I looked at the correlation between each pair of events. Some of the interesting findings:

• In general, the most highly correlated pair was Regional Event 4/Regional Event 1 (71% for women and 67% for men). This is somewhat surprising given how the time domains were completely different, but both involved pull-ups and a light thruster-type movement (bar thrusters or wall-balls).
• The two muscle-up workouts (Regional Event 3 and Open Event 3) were highly correlated (68% for women and 55% for men).
• Regional Event 5 and Regional Event 7 were highly correlated (56% for women and 68% for men). Both were extremely heavy.
• The least-correlated pair was Regional Event 3/Regional Event 5 (17% for women and 19% for men). Shouldn't be a surprise considering one was bodyweight only and one involved extremely heavy deadlifts. The pair of Regional Event 2/Regional Event 3 were also not very correlated (38% for women and 22% for men). Remember those occurred within 2 minutes of each other.

That's about it for today. This is always one of my favorite analyses to work on, but with the Games fast approaching, I suppose it's about time to start tackling the tough questions and making some predictions. Will Froning three-peat? (Probably) Who will emerge on the women's side with Annie out? (It's wide-open) Will the first event of the Games take more or less than 4 hours? (God, I hope so) What bizarre contraption will Rogue unveil this year? (Potentially a flying bicycle, similar to the one in E.T.) Who will wear the shortest shorts this season? (Stacie Tovar still the champ until proven otherwise)

Anyway, until next time, good luck with your training!

*I am referring to the effectiveness of this event as it relates to competition. In other words, is this event a good test of fitness. This does not necessarily mean the event is good or bad for training purposes. For instance, I feel that 13.2 was not a good workout for testing (due to a lot of factors, like how difficult it was to judge and how light it was for the top competitors), but in training, I think it would be a good workout for building aerobic capacity (and it definitely left me hurting).

A Fairer Regional Comparison: 2013 Edition

Today's post will mark an anniversary of sorts for my blog. Roughly a year ago, I put up my first post, in which I produced adjusted worldwide Regional rankings, accounting for the advantage gained by athletes in the later weeks. The post went up with little fanfare, and I imagine many people who started coming to this site via Rudy Nielsen's Outlaw Way blog may not have even read this original post. In hindsight, it was probably overly technical, but give me a break: it was my first post.

Well, today I'll be doing essentially the same thing for this year's regionals. But first, let me clarify what I believe these rankings really mean. They are not my predictions for the Games, and they are not a ranking of the best CrossFitters in the world. Rather, what they are is an attempt to understand who would win if the exact same Regional events played themselves out at the Games. If all the Games qualifiers competed together, at the same time, using the Games scoring system, who do we believe would win? Certainly this gives us a good starting point to discuss who may win the Games (hint: Rich Froning has a shot!), but it's not a prediction for the Games.

Many of you have probably seen a version of these rankings at http://crossfitregionalshowdown.com/leaderboard/men and http://crossfitregionalshowdown.com/leaderboard/women. These are certainly informative, and in fact I grabbed all the original results from this site, so I really appreciate the work they've done. But my feeling has always been that the athletes in the later weeks have an advantage that isn't captured in these rankings. It stands to reason that having additional weeks to prepare, watch other competitions and game plan the events has to give athletes some advantage.

To get an idea of the advantage across the entire field, I ranked all athletes that finished all 7 Regional events based on their worldwide Open rank. Then I ranked them by their worldwide Regional rank (from the sites listed above). For each region, I looked at the average change from Open rank to Regional rank, then plotted them along with the week in which each region competed. This is shown in the chart below, with lower numbers representing improvements from the Open to Regionals.

You can see that while there is variation, the latter weeks clearly tended to have better scores from athletes, relative to their Open performance. This isn't a perfect metric: for one, some elite athletes don't take the Open as seriously as others, but also, it gets skewed at the each end because the top athletes in the Open can only get worse and the worst athletes in the Open can only get better. Still, this tells us that something is probably going on. For women, the same effect is there, but it's not quite as pronounced.

To attempt to account for this, I performed a series of two-variable linear regressions. For each event, I used the event result (the actual result*, not the ranking) as the dependent variable, and for one of the independent variables, I used the week of the athlete's region. For the other independent variable, I used either the athlete's 2012 Regional ranking (if he/she competed individually in 2012) or the athlete's 2013 Open ranking (if he/she did not compete individually at Regionals in 2012). Then I looked at the coefficient for the week of competition - this would give an indication of the impact of the week of competition, after controlling for the athlete's ability.

I then divided the coefficient by the average result on that event for all athletes to get the percentage impact of the week of competition. Depending on some of the summary statistics and using a bit of judgement, I arrived at a final adjustment factor for each event (some were 0 if the results didn't appear significant enough). If the adjustment factor for event 1 was -1.0%, then for every week beyond the midpoint (week = 2.5), I adjusted the athlete's score up by 1.0%. If the athlete competed in week 4 and his score was 6:00, then his adjusted score becomes (6:00 / (1 - 1.5%)) = 6:05. If the athlete competed in week 1, his adjusted score becomes (6:00 / (1 + 1.5%)) = 5:55.

Below are the adjustment factors I used for each event for men and women:

• Event 1: -0.6% (men), -0.5% (women)
• Event 2: +0.7% (men), +0.8% (women)
• Event 3: -0.7% (men), none (women)
• Event 4: -0.2% (men), none (women)
• Event 5: none (men), none (women)
• Event 6: -1.5% (men), none (women)
• Event 7: -1.8% (men), -2.2% (women)

As you can see, these are not particularly aggressive adjustments. I think with the first wave of athletes having several weeks to prepare, the advantage is not dramatic each week beyond that. And for women, certain movements like muscle-ups and handstand push-ups are simply so troubling for many athletes that no amount of game-planning in a few weeks can make a significant difference.

For those who care, a couple notes on differences from my modeling last year, as well as some limitations:

• This year, I used all Regional athletes. Last year I only used those who completed all made the cut to the final event, but since there were no cuts this year, I used everyone.
• This year, I performed the regression across the individual athletes. Last year, I summarized up to the region level first before performing the regression. I think this year's method is preferable; last year I summed up first because I didn't have access to everyone's Open results, so I just counted up the number of athletes in each region in the top 180 worldwide as a proxy for region strength.
• This year, I included Asia, Africa and Latin America in running the regressions. Last year, I excluded them.
• I made no adjustments for the weather conditions at the outdoor regions. I realize that the elements may have made things more difficult, but with only a couple regions competing outdoors, it is difficult to assess just how much impact this had.
• I used no tiebreakers. If two athletes tied, they just stayed tied. Sue me.

OK, let's cut the chit-chat and get to the results. The tables below show my adjusted worldwide Regional rankings, along with the rankings if I had not made any adjustments for the week of competition and the rankings if I had not made any adjustments and used Regional scoring.

My adjustments were generally smaller on the women's side, so there wasn't a ton of shifting on that side of the leaderboard. For the men, we definitely saw some big jumps. Among the biggest winners with my adjustments were:

• Josh Bridges (7th to 4th)
• Lacee Kovacs (28th to 18th)
• Travis Mayer (32nd to 23rd)
• ZA Anderson (34th to 24th)
• Mikko Salo (30th to 22nd)
• Daniel Tyminski (18th to 14th)
• Marcus Filly (26th to 20th)
• Valerie Voboril (20th to 17th)
• Kristan Clever (31st to 28th)
• Katrin Tanja Davidsdottir (35th to 32nd)
• Talayna Fortunato (14th to 12th)

That's all for now, folks. Next week, I plan to look into all 12 events we've seen so far this season and see which ones were "better" than others. Until then, good luck with your training.


*For events 3, 4, 6 and 7, I added additional time for each rep still left at the time cap, since the 1 second per rep they have listed is not realistic. For events 3 and 7, I added 15 seconds/rep. For event 6, I added 10 seconds/rep. For event 4, I added 5 seconds/rep.

Quick Hits: Regional Reaction and Upcoming Schedule

It's been quite a ride the last four weeks of regionals, and to be sure, there is plenty of analysis to be done before we get to the Games in a few weeks. But for now, I just want to get some general thoughts together about the Regionals and my experience watching the Central East. I'll also look briefly at how my projections turned out, as well as what I plan to post over the coming weeks.

Let's get started:

• Watching the Central East regionals this past weekend solidified in my mind that CrossFit as a sport has staying power. The stands were legitimately packed all weekend, and some of the roars from the crowd were unlike anything I've heard at a CrossFit event before. In particular, one moment that stands out was the end of men's event 6 with Rich Froning chasing down Dan Bailey on the lunges. When Dan put the bar down halfway through, the crowd just sensed that Rich had a shot, took their energy to another level, and of course, Rich delivered, gutting out the entire 90 feet to take the event. Few other sports let the spectators really see that type of will and determination to endure pain, and on a regular and repeated basis. It's one of the reasons I've always enjoyed watching track and field, but I think the variety of CrossFit pushes the athletes to places they have never been to before. I think any fan of sport can appreciate seeing that.
• There's no question that the Central East men's region was hands-down the most competitive region. What is up for debate is whether HQ should do anything to remedy the situation. I think inviting athletes solely based on a worldwide regional ranking is not a smart idea - not unless the Regionals are all held on the same weekend and under identical conditions. That means no more outdoor events, and it probably means we all get to watch less of the action. But perhaps some sort of last-chance qualifier might be worthwhile for those in the top 10 of their region? Seeing athletes like Nick Fory and Gerald Sasser miss out on the Games despite putting up scores that would win several other regions is frustrating (see http://crossfitregionalshowdown.com/leaderboard/men).
• After making some adjustments based on the actual results that came in for the overhead squat workout, I recalculated the average relative weight and the load-based emphasis on lifting (LBEL) for this year's regionals. For both men and women, the average relative weight was higher than both 2011 and 2012, but the LBEL was lower. The way I interpret that is that the minimum strength required to be competitive at regionals was very high this year (high relative weight), but for those that could handle the required weights, the competition generally favored the smaller athletes (low LBEL).
• I'd still like to see running more involved in the regional competition. I know there are logistical issues, but it seems silly that in selecting the fittest athletes in the world, no one is required to run more than 800 feet (and even that 800 feet of running, in event 7, is little more than a light jog to recover). How about at least some shuttle sprints? Or a longer run on Friday when fewer people are able to attend anyway?
• On the men's side, it's going to be almost impossible to pick against Rich Froning for the Games. The guy is an absolute machine with virtually no weaknesses. Unless every event is under 4:00 or over an hour, I don't see a lot of scenarios where he doesn't win.
• For women, things are wide open. If Annie comes back, she's got a great shot (after the inevitable backlash against HQ for inviting her). Either way, Sam Briggs and Camille Leblanc-Bazinet look awfully solid all-around, and some of the stronger athletes like Elizabeth Akinwale and Linsday Valenzuela have become more well-rounded in the past year. There are others in the mix as well (Talayna Fortunato, Amanda Goodman, Kara Webb, Rebecca Voigt, etc.).

As for my predictions last week, things turned out fairly well. I totally whiffed on two athletes who made the Games (Lindy Wall and Jordan Troyan). For Wall, at least I have an excuse: last year, she was Lindy Barber, so I did not account for the fact that she was 6th at regionals a year ago. Had I known that, she would have been given a 16% chance instead of the 6% I originally gave her (I didn't even list her among top contenders on the site). With Troyan, although he was 10th at Regionals last year, he didn't have a great Open performance this year (20th in his region), so I just didn't see that one coming.

If I give myself a pass on that, then I ended up with a mean square error of 0.044 across the final two weeks (I predicted a total of 8 regionals). This was significantly better than any of the "default" estimates I mentioned last week. The best of those estimates would have been giving my top 3 athletes in each region a 100% shot, which would have produced a MSE of 0.052. There's always room to improve this modeling, but I think it was a decent start in my first year attempting to predict the regionals.

OK, finally, here's a quick listing of topics I plan to touch on between now and the start of the Games:

• A look at all the events thus far to see which are most correlated with success across a wide variety of events (i.e. which events are "better"). I did a very similar analysis last year and did a shortened version of it after this year's open.
• Adjusted worldwide regional rankings to account for the advantage gained by competing in a later region (and possibly other factors). This will be very similar to my inaugural post on this site approximately a year ago.
• Games predictions. I'll definitely do some sort of stochastic prediction to give each athlete odds of winning the Games (and potentially the odds of finishing second, third, etc.). I may separately do a best estimate of who will finish in the top 5 or 10.
• Perhaps a scientific wild-ass guess (SWAG) about some of the events we could see at the Games. This seemed to be pretty popular during the Open, and of course it requires little-to-no hard work on my part.
• Maybe a bit more analysis on the Open now that I've got a more complete dataset (thanks again to Michael Girdley at girdley.com for hooking me up with that data). This could wait until after the Games, depending on time.

If you've got other ideas, by all means let me know. Looking forward to the Games in just six short weeks. I've got my gold ticket, and I hope you all do, too!

Week 4 Predictions: Let the Big Dogs Eat

OK, so I may be a little biased living in Indianapolis, but I think I'm justified in saying that the Central East men's competition is really the crown jewel of the Regional season. I'll be headed down to Columbus, OH to watch Saturday and Sunday in person, and I have to say I'm pretty pumped. The numbers for this region are staggering:

• The winner of the last three CrossFit Games has come from the Central East (Graham Holmberg 2010, Rich Froning 2011-12). Both men are competing this weekend.
• The Central East produced five men in the top 10 of the 2012 CrossFit Games. All five are competing this weekend.
• Two other men competing this weekend qualified for the Games in 2011: Nick Urankar and Joseph Weigel. Urankar was 6th at the regional last year.
• Four of the top 20 Open finishers worldwide and six of the top 43 are competing in this region this weekend.

Thank goodness for the rule allowing extra spots for former champions, because limiting this region to only three spots would be damn near inhumane. Five doesn't even seem quite enough, frankly. Eight competitors from this region were in the top 31 of the worldwide regional standings last year. There are some other great athletes competing this weekend (Ben Smith, Nate Schrader, Lucas Parker, Christy Phillips, etc.), but you can't deny the strength of the Central East men's region as a whole.

The competitiveness of this region also forced me into making some adjustments to my modeling this week. For background on the process of generating these projections, please see my prior post.

Change one this week was the fact that I decided I could not treat Froning like the other competitors. Realistically, the only way he's not making the Games is due to injury. My solution was to give him an 80% chance of finishing top 10 worldwide and a 20% chance of finishing 11-25 worldwide. This leaves open the chance that he could get beat, but only in a scenario where several other guys put up crazy numbers.

Change two was to adjust all my projections to give more of an advantage to the top competitors. My solution was to do the following:

• For each category, look at last year's results and calculate how often athletes in that category finished in the top 25 worldwide. Last week, I calculated how often they finished top 50.
• Additionally, calculate how often they finished between 26-75 worldwide. Last week, I calculated how often they finished 51-100.

The reason for making this change was that in really competitive regions, too many athletes were getting put into that top 50 range. From there, it was just a crapshoot to see who drew better random numbers and qualified for the Games. Now, it's less common for athletes to get into that top 25 category, so doing so gives you a big advantage. For instance, athletes in the top category (top 15 at Games last year) have an 83% shot at finishing top 50 and a 65% shot at finishing top 25. The next highest category (16-50 at Games last year, top 40 worldwide in Regionals last year) had an 81% chance at finishing top 50 but only a 41% shot at finishing top 25. This gives us more separation for the top athletes.

Still, the Central East has five men who all qualified for my top category. The fact is, it's tough to separate them: who would you pick not to make it from those five? But then again, you also have Nick Urankar, who is really solid (26th worldwide in the Regionals last year), and there are others who could legitimately slip into the top 5. As a result, my projections don't look that great for some men that you'd generally consider to be locks for the Games. I'm not sure that's necessarily wrong.

One final note: I made the assumption that if Froning or Holmberg is in the top four, the region gets four spots. If both are in the top five, the region gets five spots. This means if Froning is 5th and Holmberg is 4th, they both still get in. Technically, this isn't how the rules are laid out, but after HQ invited 4th place Kristan Clever from SoCal, I think it's a safe bet that they'd do the same for these guys.

Before I move onto this week's picks, let's look at how last week's picks turned out. Overall, I think pretty well:

• Of the 12 qualifiers from regions I projected, 4 were given more than a 50% shot at making it. There were also 3 athletes who I gave more than a 50% shot who did not make it.
• According to my picks, the longest shots to qualify were Alex Nettey at 16%, Zach Forrest at 18% and Matt Hathcock at 18%.
• If we look at the mean square error (MSE) for my picks compared to some other naive estimates, they did pretty well. 
• The MSE for my picks was 0.045.
• If you had given everyone in each region an equal shot, your MSE would have been 0.064.
• If you had given a 100% chance to the top 3 in the Open in each region, your MSE would have been 0.078.
• If you had given a 100% chance to my top 3 in each region, your MSE would have been 0.057.

So overall, a decent showing. As mentioned above, I made some adjustments, so we'll have to see how they turn out.

As I did last week, I'm limiting these picks to the four regionals that will be broadcast. In this case, that's the Central East men and women and the Mid Atlantic men and women. Without further ado, here are the athletes with the best shot at qualifying from those regions this weekend:

Enjoy the weekend, everyone!

Week 3 Predictions: #Spealler and Stochastic Modeling

Welcome back for week 3 of the regional season. Looking at this slate of regions (Asia, North Central, North West, South West), it's pretty clear what sticks out: #spealler. What everyone wants to know is whether or not Chris Spealler can qualify for a 7th time, and if so, will he do it in as dramatic fashion as last year?

I'll attempt to answer that question in a couple of ways. One way is just based on experience watching the sport, sizing up the events and going by "feel" to some degree. The other way is to look at Spealler's chances the same way I'll be looking at everyone else's chances: with some stochastic modeling.

Before I get back to Spealler, I'll give an overview of how I did the modeling this week, because it's quite different than my week 1 and 2 predictions. This week, I wanted to look not only at who is most likely to qualify, but how likely they are to qualify. In other words, I wanted to estimate the probability of each athlete qualifying to the Games.

In short, this is done as follows:

• Based on their performance in the 2012 Open, 2011 Games and 2011 Regionals, separate the 2012 Regional competitors into various categories.
• See how frequently athletes in each category posted a very high (top 50 worldwide) or relatively high (50-100 worldwide) regional performance in 2012 (based on the cross-regional rankings last year).
• For each athlete this year, place them in one of the categories based on their performance in the 2013 Open, 2012 Games and 2012 Regionals.
• Depending on their category, randomly generate a worldwide ranking for each athlete this year. The category affects this randomized worldwide ranking, i.e. those who had better results in the past year will generally get a better randomized worldwide ranking this year.
• Re-rank all athletes within a region based on these randomized worldwide rankings.
• Repeat 1,000 times and see how often each athlete qualifies for the 2013 Games.

Now, for those so inclined, here are a few details on this process:

• To get a large enough sample size to build this model, I combined men and women.
• The process for creating the categories of competitors was not straightforward. There was quite a bit of judgment on my part to make sure that each category had sufficient athletes to be credible and that the categories produced results that made sense with each other. For instance, I wanted to separate out the top 2011 games competitors (I chose top 15), but that meant I could not further break those athletes down based on 2011 regional rank, because there just would not be enough athletes there to get a credible sample.
• The process for randomly generating the numbers is as follows:
• Generate a uniform random number between 0 and 1 (=rand() in Excel). If the first is lower than the athlete's chance of finishing in the top 50, assign him or her to the top 50. If not, then if it is lower than the athlete's chances of finishing in the top 100, assign him or her to be between 50-100. Otherwise, the athlete is assigned to be between 100 and 150.
• Once we have assigned the athlete to the a range of ranks, generate another uniform random number between 0 and 1. Multiply this by 50 to get the athlete's exact place within the range. Generally, you'll need to be in the top 50 worldwide to qualify, but depending on how other athletes fare, it's possible to end up in the 50-100 range and still be in the top 3.
• Here is a lifting of the categories I used to break down the athletes:
• Top 15 at prior Games
• Below 15 at prior Games, top 40 worldwide at prior Regionals
• Below 15 at prior Games, below 40 worldwide at prior Regionals
• Did not make prior Games, top 50 worldwide at prior Regionals, top .5% in current Open
• Did not make prior Games, top 50 worldwide at prior Regionals, below .5% in current Open
• Did not make prior Games, 50-100 worldwide at prior Regionals
• Did not make prior Games, below 100 worldwide at prior Regionals, top 1% in current Open
• Did not make prior Games, below 100 worldwide at prior Regionals, below 1% in current Open
• Did not make prior Games, did not compete at prior Regionals, top 0.2% in current Open
• Did not make prior Games, did not compete at prior Regionals, 0.2-1.0% in current Open
• Did not make prior Games, did not compete at prior Regionals, below 1.0% in current Open

OK, before we move onto the model results, I promised some good old-fashioned analysis regarding one Chris Spealler. As I mentioned when the regional events were announced, I thought these events favored smaller athletes much more so than last year. It seems that has been the case so far on the men's side (Spencer Hendel failing to qualify and Josh Bridges dominating his regional are two pieces of evidence for this). I think Spealler will take a big hit on the overhead squat and a slightly smaller hit on the deadlift-box jump, but I see him faring well on everything else. 

There are at least 5 really tough guys in that region, so he'll certainly need to be at the top of his game. I tend to think Hathcock will break through this year, which would mean Spealler would probably need to beat out Patrick Burke in order to qualify (doubt he can beat Matt Chan). You never know with Burke since he really struggled in the Open, but he has been in the Games 4 years running. But all in all, with a gun to my head, I say Spealler makes it. I don't exactly know how, but I say he makes it.

Onto the model results. Because this process was time-consuming, and because it's most definitely in the early stages with a few kinks to get worked out, I only produced predictions for the South West and North Central this week. 

This process tends to give a lot of solid athletes a good chance of qualifying, but will rarely give anyone an extremely high chance of qualifying. Obviously, for someone like Rich Froning, I may be inclined to make some manual adjustments so that his odds go up substantially. But frankly, the Regionals are so tough these days that there are only a handful of athletes that we expect to cruise through to the Games.

This week, I have just produced the results straight up, with no modification. Take them for what they are worth: these models only take into account the performances from the past year, and obviously I have no idea how an athlete is feeling or how they have been training. On a large scale, I feel good about them, but there are certainly instances where certain athletes probably should be assigned a better chance than they are here (Kasperbauer seems like one, as you will see below, but then again, that is a very deep region).

OK, finally, below are the athletes in each region with the best chances of making it to the Games.

Enjoy the weekend everyone!

How is Jackie being won?

Friday afternoon, I saw a quote in the recap of the South Central men's event 1 recap from Mike McGoldrick: "The race is in the thrusters and pull-ups for time, so the row doesn't matter." That's always been my feeling as I approached Jackie, but is that really the case? In particular, is that the case for the athletes at the top of our sport?

To answer this question, I took a similar approach to looking at this workout as I did for Open WODs 2-5 this year (for more detail on the theory behind all this, see my post "WOD Design and Why It (Usually) Pays to be Well-Rounded"). Watching several elite athletes, I timed their splits on each movement. Once I had average times for how long each portion of the workout took, I could look at how much an improvement or decline on each portion of the workout would impact an athlete's final time.

What I did for each of the Open WODs was this:

• Use the average pace for each movement to calculate a baseline score.
• For each movement, reduce the average pace by 20%.
• For the other movements, increase their pace by an amount such that the composite pace is still 1.00. If we have 3 movements, that means increasing the pace of the other two movements by 10% each.
• Re-calculate the time for the workout. The percentage reduction in the overall score from our baseline is the "leverage" for that movement. Higher leverage generally indicates that a weakness on this movement will hurt an athlete a lot.
• Repeat this for each movement.

Based on the archived live footage of the top men's and women's heats from the South Central and Northern California, I calculated splits for as many athletes as possible. Camera angles prevented me from getting the entire field, but in total, I got 11 men and 10 women (I only included athletes for whom I could get all three of their splits). Applying the approach listed above yields these results:

These results would indicate that actually, the row is the most important movement, by a longshot. A 20% reduction in row speed really hurts the athlete, even with an improvement in the other two areas. Problem solved, right?

Well, not really. This analysis is useful, but the underlying assumption here is that an athlete might actually vary by 20% on the row just as easily as they might vary by 20% on the thrusters or pull-ups. In reality, these athletes will all post row times that are much closer to each other than that. Our average row pace for men was a 3:24 (remember, these are elite athletes only). A 20% reduction in pace would give us a 4:15 pace - none of these athletes are rowing at that pace unless they're rowing for at least 10K. On the other hand, the average athlete took 38 seconds to complete the pull-ups. A 20% reduction in speed means taking 48 seconds - that's very possible, even for an elite athlete.

What I did to try to account for this is to calculate the standard deviation in the split time for each movement. For those unfamiliar, standard deviation is a measure of how much variation there is among a set of values. The higher the value, the more variation there is. Below is a chart showing the average time for each station, the average time with pace increased by 2 standard deviations and the average time with pace decreased by 2 standard deviations*.

You can see that even though the row takes much longer, there is much less variation. The coefficient of variation (standard deviation divided by average) was 3% for men's and women's row, 8% for the women's thrusters, 9% for the men's thrusters, 14% for the men's pull-ups and 24% for the women's pull-ups.

With these values in hand, instead of calculating the leverage as I described earlier, I used the following method:

• Use the average pace for each movement to calculate a baseline score.
• For each movement, reduce the average pace by 2 standard deviations.
• For the other movements, increase their pace by a number of standard deviations such that we composite to a 0 standard deviations moved. If we have 3 movements, that means increasing the pace of the other two movements by 1 standard deviation each.
• Re-calculate the time for the workout. The percentage reduction in the overall score from our baseline is the "normalized leverage" for that movement.
• Repeat this for each movement.

Applying that to Jackie, we get the following:

What we see here are two very different stories: 

• For the men, the workout is balanced. Thrusters are most important, but the row is critical as well. The pull-ups aren't vitally important for these athletes because nearly all of them are going unbroken.
• For the women, the workout is won or lost on the pull-ups. The row doesn't separate the ladies that much, nor do the thrusters. However, athletes who were strong on the pull-ups could make up 20 seconds or more on that station alone.

To understand how some of the top athletes hit this workout, consider Jason Khalipa and Pat Barber:

• Khalipa set the current record (5:04) largely based on his blazing row time - his row pace of 314 meters/minute was 2.0 standard deviations above average, his thrusters were right at the average and his pull-ups were 0.8 standard deviations above average (remember, these "averages" are for the elite of the elite).
• Barber finished 15 seconds behind Khalipa (5:19), almost entirely due to the row. His row pace of 274 meters/minute was 2.1 standard deviations below average, and even after going 1.8 standard deviations above average on the thrusters and 1.6 standard deviations above average on the pull-ups, he still couldn't make up all of the ground he lost on the row.

In my view, this is an excellent way to understand the strategy for each workout, but there are limitations. The biggest problem is that data like this doesn't always exist without the benefit of video footage. The process of gathering it is also time-consuming and challenging (I'd love a bigger sample size, but it takes about 15 minutes to get the splits for a single heat). And obviously, this particular analysis really only applies to elite athletes - for someone shooting for a Jackie time closer to 8:00, the row is probably less of a factor since it's likely the pull-ups or thrusters that are sapping more of the time. The averages and the standard deviations need to be calculated using athletes of a similar caliber for them to make sense. 

Still, I hope this has provided some insight into what's behind some of the times we're seeing these athletes put up at regionals. Enjoy week three everyone!

*You may notice that for the women's pull-ups in particular, the bars with plus and minus 2 standard deviations are not evenly spaced around the original average. That's because I calculated everything based on the pace (i.e. reps per minute) rather than the time (i.e. minutes per rep). When I converted it back to the time (because it's easier to visualize), the symmetry disappears. For example, 25 miles per hour converts to 0.040 hours per mile, 20 miles per hour converts to .050 hours per mile and 15 miles per hour converts to 0.067 hours per mile. The miles per hour are symmetrical, but not the hours per mile.