Regional Predictions Week 3: Central, Meridian and West Regionals

We're back for one final week of Regionals.  You can get some of my thoughts on the first two weeks of Regionals on the podcast I posted earlier this week.  Every year, we seem to have a few seemingly shoe-in athletes not make the Games, but last week was full of surprises.  There have been 18 men and women from last year's top 15 at the Games who have competed at Regionals in the first two weeks, and six (33%) have failed to qualify.  Those types of athletes generally get a 70-85% chance of qualifying in my modeling, so this is a bit atypical.  Two of those, Josh Bridges and Valerie Voboril, have each qualified at least three times in the past.

On the flip side, we've seen some athletes finally break through this year (e.g. Elijah Muhammad) or get back to the Games (e.g. Spencer Hendel).  Hendel is one of the few athletes to qualify for the Games, fail to qualify for multiple Games in a row, then qualify again.

I'm sure we'll have some surprises in store in week 3, so it should be a fun one to watch.  Enjoy the weekend everyone!

(Note: West Regional predictions revised 5/30 to remove Ben Stoneberg and Emily Carothers, who are not competing. All other athletes' chances increased as a result.)

Regional Predictions Week 2: California, Pacific and East Regionals

One week is in the books, and things went pretty well from a prediction standpoint and a viewing standpoint.  Before we get to predictions for week 2, here are some thoughts on what we've seen so far:

• From a viewing standpoint, I thought the events improved throughout the weekend.  "Randy" wasn't bad, but to me, any workout with just a single movement just isn't quite as fun.  "Tommy V" was simply too many rope climbs to really be a great spectator event.  The long chipper on Saturday was pretty entertaining once the run ended (Emily Bridgers and Anna Tunnicliffe had a nice battle in the Atlantic).  The handstand walk followed by the snatch was OK, but nothing great.  But Sunday was good for some high drama.  The handstand push-ups on event 6 really caused some serious shifts in the leaderboard and it gave us some good battles, like Camille vs. Natalie Newhart in the South.  Event 7, while in my opinion not quite as good of an event as last year's pull-up/OHS workout, provided some great theatrics.  The men's final in the Atlantic was the only event that I had to watch live, and I (along with with plenty of others it seemed) was rooting for Elijah Muhammad to finally get his trip to the Games.  His comeback on that event was exactly the type of drama the sport needs.
• In the past couple weeks, I complained a bit about the fact that only two attempts were given for the max snatch, but my reasoning was that it left athletes vulnerable to being penalized significantly for a single mistake.  But watching the event, I also felt like having only two attempts made for an awkward viewing experience.  I think we're trained to expect three attempts on something like this, and it felt odd that the event was over after only two tries.  It just moved too quickly, in my opinion.  If there were three attempts, I think you'd get more athletes really pushing the envelope on that third try, but generally we got some pretty safe lifts for most athletes.
• Speaking of the snatch, for men, the average load lifted was 236 lbs. in the Atlantic and 226 lbs. in the South, and that includes 2 lifters in each region who had no good lifts (those counted as 0 lbs.).  For women, the average were 151 lbs. in the Atlantic and 144 lbs. in the South, including 1 athlete in the Atlantic and 3 athletes in the South who had no good lifts.  By comparison, the averages on the hang snatch event last year were 224 lbs. for men and 137 lbs. for women.  Remember, those fields included about twice as many athletes, so you'd expect the loads not to be quite as high, not to mention the fact that the lift was required to be from the hang and power snatches were not allowed.  I think it's safe to say athletes were not as close to their maxes this year.
• The predictions were generally pretty decent in week 1.  The only major surprise qualifier was Whitney Gelin from the Atlantic, who I had pegged with a 1% chance.  Otherwise, every other qualifier had at least a 13% chance of qualifying.  Of the 9 athletes I predicted with greater than a 50% chance, 7 qualified.  After the regionals are over, I'll update the calibration plot that I showed last week.
• For this week's predictions, the only significant manual adjustments I made were to boost Mat Fraser's chances a bit (not quite the same amount as I boosted Camille's last week, but close) and to boost Kara Webb's chances (I treated her as if she was a top 15 Games finisher).

One final note: I'm planning to record a podcast this weekend with John Nail.  We'll be doing the podcast after the Memorial Day "Murph" workout at my old gym, and we'll be chatting about regionals, my (still unfinished) road back to being a full-fledged CrossFit athlete again, whether Hamm's is truly the best cheap beer available and other subjects of vital importance.

And finally, predictions are below.  Enjoy week 2 of Regionals, everyone!

Regional Predictions Week 1: South and Atlantic Regionals

In each of the past four years, fans who've watched the CrossFit Games have seen some really cool stuff. At the Games, you get to see athletes put up some insane weights (377-lb. overhead squats, for instance), you get crazy-brutal events like a 2-hour triathlon or the burden run, and you get to see some events with excellent finishes (like Josh Bridges holding off Rich Froning in "Push Pull" last year).  But what you don't typically get is a lot of drama.  With the exception of last season's men's competition, the men's and women's titles have basically been decided by the time the last event rolled around, and even in that men's final last year, you never felt like Mat Fraser could really pull it off.  You have to go back to the 2010 men's final to really get a big shift at the end, and even that was weird because it was hard to tell that Graham Holmberg was actually going to catch Rich as you were watching.

No, the Games is not where you go for drama.  That would be Regionals.  If you've been to the event in person on a Sunday, the tension in the air is unavoidable.  Watch Sevan Matossian's documentary "Only Three" about last year's Central East Regional and you'll see what I mean.  For that reason, the Regionals have long been my favorite part of the season as a spectator.

For the third year, I'll be posting predictions for each athlete's chances of making it to the Games.  In the past, these predictions have been fairly well-calibrated, meaning that, for instance, athletes with a 50% chance of qualifying in my model do typically make it to the Games around 50% of the time.  That doesn't mean the model is perfect, of course.  If it were, I'd give a 100% chance to the 5 athletes who were going to make it.  But then again, no model is perfect, and if there were a perfect model, well, there'd be no drama.  So keep in mind these are all in good fun.

For more background on the model and how it works, listen to the CFG Analysis Podcast Episode 8 or read up here.  In theory, the model should be getting better each year as we get more data from past years to help calibrate it.

Before we get to the predictions, here are some general thoughts on this year's Regionals:

• As far as loading goes, this year is roughly the average of the past four years.  The required weights in the metcons are lighter than last year, but there is more lifting overall (48% vs 37%).  Although the exact number depends on the numbers we see in the snatch event, the LBEL looks to be around 0.72, which is lower than 2012 (0.92) but higher than 2011 (0.68), 2013 (0.60) and 2014 (0.58).
• Olympic lifting movements play a huge role, even more so than in the past.  Every single lift falls into the "Olympic-Style Barbell Lifts" category that I've defined, and even if you restrict that definition to only snatches, cleans and jerks, that's 36% of the points.  Thrusters, overhead squats and sumo deadlift high pulls make up another 12%.  Historically, this entire group of movements is worth about 33% of the points at Regionals.
• Burpees are nowhere to be found for a second straight competition after appearing in every competition from 2008-2014.  Very curious on the reasoning from HQ here...
• I like the move to the Games-style scoring system.  This won't punish a single poor performance quite as much (Sam Briggs would have made it to the Games last year under this system), and it rewards elite performances on certain events more so than the past Regional system.  More old school thoughts on the scoring system here.  If I had my druthers, this is the scoring system I think I'd go with.

OK, well let's get to the predictions.  The only manual adjustments I made this week were to bump up Camille's chances a bit (despite her being in the highest-rated cohort already) and to put Sam Briggs into a slightly higher cohort (equivalent to athletes finishing below 15th at the Games last year but top 40 at Regionals last year).  Enjoy the weekend everyone!

Podcast Episode 8: How Do I Make Regional Predictions?

Today's podcast covers the method behind the madness of predicting each Regional athlete's chances of making the Games.  I'll post the week 1 predictions early-mid next week, and we'll have another podcast talking about the events themselves.

Since I never posted it last year, here's the final calibration plot of how last year's predictions ended up.  Things shaped up pretty well.  The mean-square error was 4.02%, compared with 4.43% in 2013 and 6.07% if you gave every athlete the same chance of qualifying.

Does Past Games Experience Matter? (And Other Thoughts On Predicting the Games)

Today, I'd like to tackle a topic that's been mentioned quite often in CrossFit Games commentary. It's basically an assumption that's been taken as fact: having experience competing at the CrossFit Games in the past gives an athlete an advantage over first-time Games competitors. I've generally believed this to be true, but without data to support it, we're all really just guessing.

But let's start with the reason I decided to look into the issue. About a week ago, I released my 2014 Games predictions, which were used in the CFG Analysis Pick 'Em that is going on right now. What I started to notice as picks came in is that people tended to wager much more often on past Games competitors. One reason for this is simply familiarity: people know these athletes and have seen them perform well (or perhaps they just like cheering for them). But I believe another reason is that people tend to assume that Games experience matters. And the reason that is a factor here is that my model does not take past Games experience into account.

Why? Well, in constructing these models, I wanted to be able to predict the chances that an athlete would win (or finish top 3 or top 10), not simply make a prediction about where they would finish on average. That meant I couldn't just set up some sort of linear regression model that could account for several variables (such as Regional placing, Open placing, past Games placing, etc.). I needed a model that could generate a range of outcomes, and I felt using this year's Games results was my best bet. This is a different approach than I took for the regionals predictions, for three reasons:

1. Entering the Regionals, there had only been 5 events thus far this season, which is not really enough for me to get a sense of what types of events might come at the Regional level.
2. Some athletes notoriously coast through the Open, so those results alone would not be a great predictor of Regionals success.
3. Because so many more athletes compete at Regionals each year compared to the Games (~1400 vs. 85), there was enough historical data for me to build a decision-tree-style model for Regionals. There is simply not enough past Games data to learn about what characteristics give an athlete a chance of winning. Basically what we would learn is: "In order to win, be Rich Froning."

So my solution was to build a pretty unique simulation model that took into account specific results for each athlete for all 11 events this season prior to the Games. It's at this point that I'd like to recite a quotation that one of my work colleagues (a predictive modeling guru if there was one) likes to bring up quite often:

"Essentially, all models are wrong, but some are useful." - George Box

Any model we come up with to predict the CrossFit Games will be wrong. Remember, a perfect model would predict with 100% certainty exactly who would finish in what positions. No one would have a 20% chance of winning - they would have a 100% chance or a 0% chance. But creating such a model is impossible. So with that in mind, I acknowledge that my model is wrong. But is it useful? I think so.

The chart below shows the calibration of this model on the 2012 and 2013 Games (combined men and women for both years). This shows how often athletes finished in the top 10, compared with the chances I gave them. A perfectly calibrated model (not necessarily a perfectly accurate one) would have the blue line follow the red line exactly, so that for the athletes I predicted with a 7% chance to finish top 10, exactly 7% of them did finish in the top 10.

As we can see, the model has been pretty well calibrated the past two years. Generally speaking, athletes with a low probability of finishing in the top 10 don't finish in the top 10. The model is also much more accurate than a dull (but perfectly calibrated) model that gives all athletes an equal chance of finishing top 10: my model's mean square error was 11.6% vs. 17.1% for the equal chance model.

But of course, my model is not perfect. And one area where it could be skewed is in how it accounts for (or rather, does not account for) past experience. If past experience is an advantage, then my model is understating (to some degree) the chances for returning Games athletes and overstating (to some degree) the chances for first-timers.

Which brings us back to the original question: Does past Games experience matter? To answer the question, I compiled the results from the Games and Regionals from 2011-2013 and tagged all athletes with Games experience prior to the year of competition (I went all the way back to 2007 to see if athletes had past experience). 

The simplistic way of looking at this is to compare the finishes of athletes with prior experience compared with first-timers. Looking at things this way, we find that returning athletes do finish approximately 8 spots higher than new athletes on average (18.3 vs. 27.8). However, this could simply be due to the fact that the returning athletes are just flat-out better, and their experience had nothing to do with their Games performance.

What we should do to account for this is compare Games performances to Regionals performances in the same year (using the cross-Regional rankings, adjusted for week of competition). In general, we expect athletes who fare better at the Regionals to perform better at the Games. So if Games experience is a factor, the returning competitors should perform better at the Games than their Regionals results would indicate. When we look at things this way, we see that returning competitors do indeed improve their placing by approximately 0.6 spots from Regionals, while new competitors dropped by approximately 0.8 spots in from Regionals.

Unfortunately, there is a still a problem with this comparison. Although Regionals performances are a good indicator of Games performance, there is still a tendency of athletes to regress towards the mean  in general. That is, athletes who finish near the top at Regionals don't tend to improve their placement at the Games, while athletes near the bottom at Regionals tend to improve slightly on average. Part of this is due to the fact that if you finish near the top at Regionals, there is basically nowhere to go but down (and the reverse is true for the athletes at the bottom of the Regional standings).

So to be fair, we need to compare returning athletes with first-timers who had similar Regional placements. Since we don't have a huge sample, I split the rankings into buckets of 10. Within each bucket, I found the average Regionals and Games placements of returning athletes and first-timers, as well as the average improvement or decline. The results are presented in the chart below.

For every level of competitors except those near the bottom, the athletes who had past Games experience showed an advantage at the Games over first-timers with similar Regional placements. While we can see that there is significant variation in how much this advantage is worth, if I had to put a number on it, I'd say that Games experience is worth between 4-5 spots at the Games. Remember, my current predictions assume all athletes have equal experience, so a reasonable adjustment might be to improve the average rank of past competitors by ~2 spots and drop the rank of new competitors by ~2 spots.

This analysis is, of course, not precise. It is likely that experience matters more for veterans like Rich Froning and Jason Khalipa than it does for someone who has only competed once at the Games before. Moreover, some veteran Games athletes have consistently struggled to match their Regionals performances, while newcomers have overachieved in the past (see Garrett Fisher last year).

I don't plan to adjust my predictions this year, for a few reasons:

• There is not a simple solution of how to implement this factor into the model framework I have set up;
• I feel that the predictions are still pretty reasonable on the whole (based on the calibration seen in the past two years);
• For the Pick 'Em contest, I committed not to change those predictions for reasons of fairness. I suppose I could produce a second set of predictions, but I think that's just creating unnecessary confusion. Anyone entering the contest is welcome to use the information in this post to their advantage if they wish.

Still, when it comes time to make my predictions again next year, I'm going to try to find a way to account for past Games experience. The model still won't be perfect, but hopefully it will be even more useful.

So Who CAN Win the 2014 CrossFit Games?

In just three short weeks, the rubber meets the road in Carson, California for the 2014 CrossFit Games. Until then, there is still plenty of time for speculation, and today marks the official speculation from CFG Analysis. Like last year, I've estimated the chances of each male and female athlete winning, placing top 3 and placing top 10.

For those who are interested, these predictions are used as the basis for the CFG Analysis Games Pick 'Em. For more information on that contest, see the original post. Please post your contest entries to comments on that post, NOT this one. Feel free to comment on this post, just don't put your contest entries there.

I won't bore you with too many details before we get to the picks, but here are some key points about the methodology:

• For the most part, the concept is the same as was done last year. For a full run-down, see my methodology post from last year. The basic idea is that we're using the regional results and the Open results from this season in a variety of combinations to simulate what might happen at the Games.
• In general terms, the picks are based 80% on the Regionals, 15% on the Open and 5% on last year's Games.
• The only information used from last year's Games were the athletes' results from the half-marathon row and the Burden Run. The reason for using these two is that there are no regional events of this length, so hopefully these events will give some insight into how the athletes will fare if (when) an event of that length comes up. For athletes that didn't compete last year, they get a random result on that event in each simulation.
• The regional results have been adjusted to reflect the advantage that athletes in later weeks have over athletes in the earlier weeks (whether it be from additional training or better strategy). I've done this the past two years, but this year the impact of each additional week was stronger than in past years. The one region that I made an extra adjustment for was the North East, since that event was held outside. I treated it as if it had been in week 1 rather than week 4.
• No advantage is given to returning Games athletes, even for Rich Froning. Certainly some will disagree with that, but we've seen time and again that athletes come out of nowhere (Garrett Fisher 2013, for instance) and past podium athletes can fall off (Matt Chan 2013, for instance). I'll probably devote a post in the next two weeks to looking at how much (if any) Games experience plays a role in predicting an athlete's success, beyond their performances so far this season.
• There are some athletes who are listed with a 0.0% chance of finishing in certain spots. Obviously every athlete has at least some chance of winning, but this method simply isn't going to account for such true long-shots. Last year, the longest shot on either side to finish in the top 10 based on my picks was Anna Tunnicliffe at 3%.
• As always, these picks aren't a personal judgment about any athlete, and of course, they are just for fun. Much respect for any athlete that even makes it to this level to begin with.

With those items out of the way, let's get on with the picks. These picks are subject to change in the event that athletes drop out prior to the competition (or if they add a late wildcard, for some reason), but otherwise, consider them final. I will make notes in this post of any changes that have occurred.

[UPDATE 7/8: HQ just announced they will be paying out prize money for the top 20 finishers, but anywhere you see the term "money" in this post, it refers to top 10.  I have re-posted these charts with the heading changed to say "Top 10", but I will not be re-doing the predictions to give odds for finishing top 20.] 

Regional Predictions, Week 4

Although last week's regional competitions had their share of drama, I'll admit it felt like a bit of a letdown after the amazing weekend prior. Thankfully, this fourth and final weekend looks like it has some fantastic stuff in store. I mean, how could Northern California's men's competition not be insane? We have seven former Games competitors vying for three spots, including three men who finished in the top 10 at the Games last year. Like we saw in the Central East, there will be some men not heading to the Games that probably would have gone in virtually any other region in the world.

With that in mind, let's get to some assorted topics before we move onto the predictions for week 4:

• Taken in a vacuum, I don't have a problem with Dave Castro's statement (speaking for HQ I presume) that there will not be any wild cards given out this year. However, in context of this season, I'm not a fan.
• Why even announce that wild card spots will be available (which they did earlier this year) if you are going to rule out that possibility before the Regionals have even finished? I cannot conceive of a scenario where wild cards would make more sense than they do for Sam Briggs this year. She is the reigning Fittest Woman on Earth, she had a single bad event in one of the most volatile events ever programmed at Regionals (1-attempt handstand walk) and she still finished fourth in a stacked region. If you're not going to use a wild card in that situation, then you're never going to use it.
• Talent is so clearly bunched in a few regions (and has been for a few years). I can understand the argument that the regionals are set up with a limited number of spots in each region to increase drama and make things more exciting. However, I find it difficult to accept the argument that this system is ideal for finding the fittest athletes in the world. I get that cross-regional comparisons are not perfect, but I challenge anyone to argue that Graham Holmberg (4th in Central East) is not among the 40 best CrossFitters in the world. As it stands now, he is ranked ahead of the champions from 9 other regions. For Castro to argue that "the right athletes" are going to the Games seems a bit disingenuous. If you're just setting it up this way for drama, that's fine, but let's just call it what it is.
• Although we have one week to go, the data from across all regions has allowed me to get a sneak peak at some interesting things from this year's regionals.
• In terms of correlation with success across all Regional and Open events, it appears that events 3 and 7 are the top events at this point. I'll admit when I was wrong, and I was wrong on event 7. The top athletes are all crushing it, and it is damn exciting. Event 3 is a bit surprising, but again, look at the athletes who are doing well there, and they're usually dominating across the board.
• On the other end of the spectrum, event 5 for the men actually has the lowest correlation with overall success. My guess here is that this is the one event this season that truly favors taller athletes, and so you are seeing some athletes with huge performances who otherwise are struggling. For the women, this event is not so bad, mainly because there are no athletes jumping 10-11 feet in the air and getting to the top of the rope in a couple pulls.
• Not surprisingly, the two single-modality events (1 and 2) are among the least correlated with overall success for both men and women. Event 2 is slightly worse than event 1, but not by as much as you might think.
• Events 4 and 6 are kind of middling in this respect. I expected event 6 to really bring out the top all-around athletes, but it might just be so grueling that it heavily favors the endurance specialists.
• If we look at Open events in this context, 14.3 has the lowest correlation with overall success among Regional athletes (as it did for the entire Open field). On the other side, 14.4 was the highest correlation with overall success among Regional athletes (as it did for the entire Open field). In fact, it is basically neck-and-neck with Regional event 3 for the top spot across all events this season.
• Some have suggested that results in the handstand walk might be correlated with success in event 4 (which has tons of handstand push-ups). It doesn't appear that way; ranks on those two events are not particularly correlated (52% for men, 44% for women - both of those figures are middle of the road this respect). The only combination of events that really stands out is events 1 and 7, which were 77% correlated for women and 68% correlated for men.
• Last week I posted a chart and some statistics regarding the accuracy of my predictions (I should note that these are after removing athletes who withdrew prior to event 1). After week 3, the calibration plot looks about the same, but the mean-square error has dropped from 4.38% to 3.93%. For reference, last year's model was 4.43% and a model giving each athlete an equal chance would be about 6.40%. Below is the calibration plot (read last week's post for an explanation):

Alrighty... with all that out of the way. Let's get onto the predictions. This week, the only athlete for whom I made a manual adjustment to the model was Jason Khalipa. This year's events might not really favor him, but the guy has been so freaking consistent over the past 6 years that I felt he warranted special consideration.

With that said, here you go. Enjoy the final week of Regionals, everyone!

[Update 5/31: I've made a couple fixes to account for women's name changes since last year, as well as making the adjustment for Andrea Ager that I suggested in the comments a couple nights ago. I treated her as if she did not compete at Regionals last year, rather than as if she finished very low. Her low finish was due to a DQ in the OHS event, not due to a poor performance overall.]

Note that Africa only has one qualifying spot. All other regions this week have three.

Also note that the pictures look prettier this week because I'm posting from a Mac. Excel is terrible on a Mac, but at least it exports nicely to pictures.

Regional Predictions, Week 3

I've said before that I have no doubt the CrossFit Games is a totally viable spectator sport. My opinion on the matter hasn't changed, but I'm beginning to think that it's the Regionals that should be on ESPN. And I doubt anyone who followed the coverage on the Games site this past weekend would disagree with me.

There was high drama all weekend, and with 10 different competitions to follow at various time zones across the globe, events were broadcast basically non-stop. We had the reigning fittest woman on Earth fighting just to make the Games in Europe, possibly the most competitive men's competition ever in Central East, a changing of the guard in Australia and the first real challenge to Camille and Michelle Letendre's dominance in Canada East. So before we move on to predictions for this week, let's get to some quick thoughts on week 2:

• The handstand walk claimed another victim this week in Sam Briggs. I'm sure HQ would never admit that the programming was anything less than perfect, but anyone following that competition knows that Briggs was one of the three fittest women there, and likely the fittest. She dominated events 3-6 and placed decently on the two heavier workouts (1 and 7).
• That being said, she will almost assuredly get a special invite, and the three women who did qualify certainly deserved it. Along with Southern California, Europe looks to be one of the two strongest women's regions in the world (Australia also looked sneaky-tough last weekend, too).
• I certainly hope that HQ also extends a special invite to Graham Holmberg. We'll have to wait to see how the rest of the competitions play out, but I'd be he would have qualified in every other region (and likely won many of them). He came through when it counted most, smashing the event record in event 7, but unfortunately, four other men in his region also beat the event record (including third-place Will Moorad). He was also the only man in that region other than Rich Froning to take an outright first place, and he did it twice.
• Super-impressive performance by Moorad to grab a spot in the Central East. As disappointed as I was to see Graham fall off, it is nice to see someone else break through in that region.
• If the Games were programmed like the Regionals, Camille Leblanc-Bazinet might be the most dominant athlete in the sport. If the event has barbells and gymnastics, she's basically guaranteed a spot in the top 10 in the world. Hopefully she can fare a bit better with the more unorthodox Games events this year.
• While I'm not convinced that event 7 is as good a test of fitness as, say event 4 or event 6, but I can't dispute that it is great for the viewers. Those 8 overhead squats have derailed more than one athlete's shot at the Games and it's given several others the chance to make up ground in dramatic fashion.
• Event 6 looks like an absolute beast of a workout. Unlike prior years, I haven't been able to test out the workouts (due to a back injury), but I don't recall seeing Rich Froning that gassed in a workout since the rope climb/sled push workout at the 2012 Games. 

So how did my predictions do last week? Well, despite a couple of shockers, things actually pretty well. The charts below show how well the predictions were calibrated this year compared to last year. The athletes were bucketed based on their predicted rank, and each bucket was plotted as a poitn on the blue line.  The location on the x-axis represents my predicted chances of qualifying and the location on the y-axis represents the actual chances of qualifying.  The red line represents perfect predictions, so when the blue line is below the red line, my predictions over-estimated the chances of qualifying for those athletes.  As you can see, this year, the blue line tracks much more closely to the perfect predictions.

Of course, calibration only tells half the story. We could easily create a well-calibrated model by predicting an even chance for each athlete. If there are 30 athletes in a region with 3 qualifying spots, we could estimate that each athlete has a 10% chance of qualifying, and indeed, we would be right in some sense. It is sure that 10% of them will qualify. But we are also trying to be accurate. A perfect model would predict 100% chances for the 3 athletes that do qualify and 0% for all others. To measure our accuracy, we can calculate the means square error across all our athletes. In that respect, I did about as well as last year, and considerably better than the perfectly calibrated model with even predictions for each athlete.

• 2014 Week 2, CFG Analysis Predictions - 4.38%
• 2013 Weeks 3-4, CFG Analysis Predictions - 4.43%
• 2013-2014 Equal Chance Predictions - 6.40%

So, with that in mind, I really didn't make any changes to the model this week. The only region where I had to deviate significantly was for the women in Asia. There are no returning Games qualifiers, hardly any returning regional competitors and just a handful of athletes in the top 2000 in the Open worldwide. My default model would have basically given all athletes the same chance, so I modifiied it as best I could, but the predictions are still pretty weak in that region.
[UPDATE 5/23/2014: In the Asia region, I just realized Candice Ford was Candice Howe last year, which meant I originally assumed she did not compete at the regionals last year. The predictions below are now fixed to account for this.]

Anyway, without further ado, here are my week 3 predictions. Enjoy the Regionals everyone!

Note that Asia only has one qualifying spot. All other regions this week have three.

Regionals Predictions, Week 2

Welcome back, everyone. The first week of the 2014 Regionals is in the books, and in many ways, I think it played out like I expected. The handstand walk derailed one heavy favorite (Stacie Tovar), all of the athletes I picked to fare well did (Lucas Parker, Elizabeth Akinwale, Talayna Fortunato) and the weekend as a whole seemed to favor athletes with strong gymnastic abilities and pure strength, rather than those with the biggest engines. I was pleasantly surprised by event 7, which appeared to be more balanced than I expected and provided some exciting shake-ups in a few regions.

This week looks to be one of the most (if not the most) exciting weeks of the Regional schedule. We all know that the Central East men's region is the toughest in the world, but the European women's region and the Central East women's region are also super-competitive as well this year. It's also a little intriguing to see which "records" will fall now that the bar has been set on all the events. Personally, I think every single one of the men's records will go down (potentially all in the Central East?) and many of the women's records will go down (doubtful that Akinwale's records in events 1 and 2 will fall).

With that in mind, let's get down to the business at hand. Last week, I made a bunch of excuses for why I wasn't able to get formal predictions finished in time, but this week I was able to make it happen. I've been able to estimate the odds of qualifying for each athlete in all 10 competitions, and at the bottom of this post I've shown the odds for the top contenders in each region.

But first, here's a recap of the methodology, which is largely similar to what was done last year:

• Learn from prior results
• Separate the 2013 Regional competitors into various categories, based on their performance in the 2013 Open, 2012 Games and 2012 Regionals.
• See how frequently athletes in each category posted a very high (top 20 worldwide) or relatively high (20-50 worldwide) regional performance in 2013 (based on the cross-regional rankings last year).
• Repeat the first two steps one year further back. Combine the results with what I came up with in the first two steps. This helped me get a bigger sample size and hopefully improve the predictions.
• Apply learnings to what we know this year to make predictions
• For each athlete this year, place them in one of the categories based on their performance in the 2014 Open, 2013 Games and 2013 Regionals.
• Depending on their category, randomly generate a worldwide ranking for each athlete this year. The category affects this randomized worldwide ranking, i.e. those who had better results in the past year will generally get a better randomized worldwide ranking this year.
• Re-rank all athletes within a region based on these randomized worldwide rankings.
• Repeat 200 times and see how often each athlete qualifies for the 2014 Games.

Now, for those so inclined, here are a few details on this process:

• To get a large enough sample size to build this model, I combined men and women.
• The process for creating the categories of competitors was not straightforward. There was quite a bit of judgment on my part to make sure that each category had sufficient athletes to be credible and that the categories produced results that made sense with each other. For instance, I wanted to separate out the top 2012 Games competitors (I chose top 15), but that meant I could not further break those athletes down based on 2012 regional rank, because there just would not be enough athletes there to get a credible sample.
• The process for randomly generating the numbers is as follows:
• Generate a uniform random number between 0 and 1 (=rand() in Excel). If the first is lower than the athlete's chance of finishing in the top 20, assign him or her to the top 20. If not, then if it is lower than the athlete's chances of finishing in the top 50, assign him or her to be between 20-50. Otherwise, the athlete is assigned to be between 50 and 100.
• Once we have assigned the athlete to the a range of ranks, generate another uniform random number between 0 and 1. Multiply this by 20 to get the athlete's exact place within the range (multiply by 30 if they are in the 20-50 range or multiply by 50 if they are in the 50-100 range). Generally, you'll need to be in the top 50 worldwide to qualify, but depending on how other athletes fare, it's possible to end up in the 50-100 range and still be in the top 3.
• Here is a lifting of the categories I used to break down the athletes:
• Top 15 at prior Games
• Below 15 at prior Games, top 40 worldwide at prior Regionals
• Below 15 at prior Games, below 40 worldwide at prior Regionals
• Did not make prior Games, top 50 worldwide at prior Regionals, top 100 in current Open
• Did not make prior Games, top 50 worldwide at prior Regionals, below 100 in current Open
• Did not make prior Games, 50-100 worldwide at prior Regionals
• Did not make prior Games, below 100 worldwide at prior Regionals, top 250 in current Open
• Did not make prior Games, below 100 worldwide at prior Regionals, below 250 in current Open
• Did not make prior Games, did not compete at prior Regionals, top 75 in current Open
• Did not make prior Games, did not compete at prior Regionals, 75-150 in current Open
• Did not make prior Games, did not compete at prior Regionals, below 150 in current Open

Last year, my predictions weren't bad, but they generally overestimated the chances for the athletes on the low end and high end, but I underestimated the chances for the athletes in the middle. Consider:

• Athletes predicted 0-10% - 2.5% expected to qualify, 0.4% qualified
• Athletes predicted 10-50% - 20.7% expected to qualify, 39.5% qualified
• Athletes predicted 50-100% - 66.3% expected to qualify, 57.1% qualified

I have more data to train the model this year, which should help to calibrate things a little better, and I was a bit more liberal in applying some manual adjustments to some elite athletes. For instance, Julie Foucher did not compete last year, but I treated her as if she had finished in the top 15 in the Games. As a three-time top 5 athlete, I think this is only fair. Other athletes for whom I made at least some adjustment included Rich Froning, Samantha Briggs, Annie Thorisdottir, Frederick Aegideus and Camille Leblanc-Bazinet.

So with that in mind, below are my predictions for week 2 (athletes with less than 5% chance are not shown). As always, keep in mind that this is all in fun, and it's all simply based on the numbers. I'm not making any sort of judgment about the effort these athletes have put in, I'm simply reflecting how athletes in similar situations have performed in the past. Enjoy week 2, everyone!

*Note that Canada East only has two qualifying spots. All other regions this week have three.

Quick Regional Programming Thoughts

As much as I was hoping to have my regional predictions set up to go this week, I wasn't able to make that happen. A confluence of events over the last month - a 5-hour actuarial exam, a trip out of town to have my son baptized, my wife's birthday and numerous trips to the chiropractor/ART to try to fix some back issues that flared up after the Open - pretty much made that impossible for me. By next week, I do expect to be able to produce my stochastic regional predictions (like I did for the final two weeks of last year's season). But for this week, your guess is as good as mine about who will claim the first 24 spots at the Games.

That being said, I did have time to do a bit of work assessing the programming for this year's Regionals. Here are my thoughts:

• Overall, I like the programming. In particular, I think events 3-6 seem to me to be well-balanced workouts that should be fun to watch.
• Events 1 and 2 introduce a ton of volatility into the situation. Only getting 3 attempts on the hang snatch means we could see some top athletes get burned by taking a gamble on those 2nd and 3rd lifts. And a max handstand walk on a single attempt gives plenty of opportunity for a catastrophic failure that could cost an otherwise fit athlete a shot at the Games. In my opinion, I don't think this event really helps us find the fittest athletes, and it may end up preventing some really stellar athletes from making it.
• Event 7 is really a wait-and-see event for me. It seems like a really weird design for a workout to have just 8 reps of the overhead squat, since it doesn't seem like it gives enough time for athletes to make up ground on that movement. But hopefully I'm wrong and this event turns out to be a better test of fitness than it appears to be on paper, especially considering it's the finale.
• This year's regional is in some ways the heaviest regionals to date and in some ways the lightest. When weights are involved, the average relative load (1.55 men, 0.98 women) is the highest in the past four years. However, the programming is only 37% lifting, the smallest percentage of any Regionals. In fact, the 2011 Games is the only HQ competition with a lower percentage (34%).
• When you combine those two factors, you get an load-based emphasis on lifting (LBEL) of 0.58 for men and 0.36 for women, both slightly lower than 2011 and 2013 and much lower than 2012. The chart below shows the progression of each of these metrics at the Regionals since 2011.

• All-in-all, I believe this year's Regionals look more like the Games than in any past year. And what that means to me is that the emphasis is on strength, both in terms of weightlifting and very challenging bodyweight movements (like legless rope climbs or strict handstand push-ups). Don't get me wrong, you can't do well here without a high level of conditioning, but you will be punished much harder for lacking in strength.
• Look for handstand push-ups and legless rope climbs to completely decimate the women's leaderboard. With the legless rope climbs, we saw how much variability there was at the Games last year. As far as handstand push-ups, keep in mind that before kipping became common (think 2011 and earlier), this was an extremely difficult movement for many top women, even at the Games.
• With that in mind, here are some athletes that should do well this programming: Lucas Parker, Josh Bridges, Lacee Kovacs, Chris Spealler, Matthew Fraser, Elizabeth Akinwale, Talayna Fortunato, Annie Thorisdottir, Camille Leblanc-Bazinet.
• Beyond those names, all the podium athletes from last year's Games should do fine. I'm just not sure they will fare any better because of this programming.

That's it for today. I hope you all enjoy the opening weekend of Regionals, and I'll see you again next week.

Fun with SWAGs: What Will 14.5 Be?

See? I told you. I knew last week was the week. For the first time in 8 tries (dating back to last year), I put up a SWAG that was pretty close to spot-on. Sure, I missed the row (who saw that coming?), but the combination of wall ball, toes-to-bar, 135/95 cleans and muscle-ups was pretty much spot-on. Will it ever happen again? Eh, probably not.

As for this week, things should be pretty straightforward with just a few movements left on the table. With that in mind, a few quick thoughts before we get to the pick:

• I've really enjoyed the programming so far this season.  Aside from this past week, it has not been in my favor, but still, I like that they have been willing to get creative and take some chances with the programming. They've also avoided any judging disasters like they had last season with 13.2.
• Having the video requirement for regional qualifiers seems to have increased the number of videos available for top athletes. This has been nice to see, as it's highlighted how completely legit most of the top athletes are. When I watch a video of a Games-level athlete, the reps are all crystal clear and the range of motion leaves no doubt (except in the head-to-head Open announcement WODs, which seem to have had some questionable reps, in my opinion).
• I have to say, I'm not sure why HQ felt they needed to add a row this year. In any other setting, I would have no problem with this, but if they really want the Open to be "the most egalitarian sport in the world," requiring the use of a $1,000 piece of equipment doesn't really make sense. Sure, this only impacts 0.1% of the athletes in the Open, but really, was it necessary?
• I'll have more on this later this week, but at first glance, it appears that my method for mid-week projections worked fairly well last week. Many thanks to Andrew Havko for pulling the data for me at several points this weekend, and thanks to the several others who offered up their services for helping me pull the data.

OK, let's get down to brass tacks. We all know what's left on the table for 14.5: burpees and thrusters. Yes, jerks are left as well, but I don't see them skipping burpees or thrusters, and I doubt they'll put jerks in a workout that already has thrusters. So I'm going to assume it's a couplet of burpees and thrusters.

Weight-wise, I think this needs to be relatively heavy, considering the LBEL and average weight load are still below their historical averages. As far as the time domain, I think they'll keep it short. We have been hovering around 10 minutes throughout the Open, but considering last week was 14 minutes and we're likely to have only two movements this week, I can't see them going long for 14.5.

Bearing all that in mind, let's get to it. My SWAG for 14.5 is:

AMRAP 7 of 3 bar-facing burpees, 3 thrusters (115/75), 6 bar-facing burpees, 6 thrusters, 9 bar-facing burpees, 9 thrusters, ...

I hate putting down something that looks so familiar, but we've seen that 7-minute, 3-6-9-... pattern in each of the past three Opens, so I feel like I had to go with it again this year. I figure the 115/75 thrusters are a bit of a diversion from prior years where it has typically been 100/65, but yeah, this one is kinda dull. Let's hope Castro proves me wrong.

Post SWAGs to comments, and good luck to everyone on 14.5!

Fun with SWAGs: What Will 14.4 Be?

For 14.1, I whiffed on the movements but at least got the time frame right. For 14.2, I did manage to call the overhead squats but basically missed on everything else. Last week? I pretty much got no part of 14.3.

But this week? THIS IS IT. I can feel it.

Obviously, things should be getting easier for us, since fewer movements are left on the table. Here is what we have left, in order of emphasis in past years: burpees, thrusters, jerks, toes-to-bar, cleans, muscle-ups, wall balls and push-ups. Of those, I'd expect them all to come up at some point, with the exception of push-ups. So that means we probably have a triplet coming up, or possibly even a workout with 4 or more movements, which has never occurred in the Open before.

You may have also noticed that the complexity has increased each week so far, along with the weight level. In fact, the last two weeks have been among the most unique workouts in the four years of the Open. My theory is that this trend will continue this week before HQ releases something simple and classic to close things out on 14.5. So I'm going to leave a couple classic movements on the table for 14.5: burpees and thrusters.

I also feel like they need to go long this week, since they really haven't gone beyond 10 minutes for all intents and purposes so far (on 14.2, the athletes lasting beyond 10 minutes probably were cruising for the first 3 or 6 minutes). Weight-wise, this could go either way, since the LBEL is very close to the historic average at this point. So for 14.4, I'm thinking something long and funky, with a bunch of movements.

With that all in mind, here's what I got for 14.4:

AMRAP 15 of 60 toes-to-bar, 60 wall balls, 30 clean and jerks (135/95), 30 muscle-ups

I wouldn't waste your time with your own pick. I just nailed it. Sorry, I know you wish you had thought of it first. Post congratulatory remarks to comments.

Fun with SWAGs: What Will 14.3 Be? (And More)

Before we get SWAG-y, I'd like to get some quick thoughts out about 14.2 and also give you the results of an interesting little study I did last week.

Without further ado, a few thoughts on 14.2:

• Really cool to see a new WOD format for the Open.  We all know the Open has typically been filled with very basic, classic CrossFit WODs, so it was refreshing to see a workout like this come out. I think it took us all a couple of days to figure out the right strategy for this one.
• Personally, I would have liked to see the overhead squats either progress in weight or start a little bit higher. For the top-tier athletes, this was pretty much all about the chest-to-bar pull-ups. When Castro announced that the rounds would change in each 3:00 segment, I was really hoping to hear him say the weights would increase. But still... cool workout.
• Good to see that they set the breakpoints such that more athletes were able to get past the first round, unlike 13.5. Based on a really quick look, it looks like about 70% of the men reached the second round and about 30% of the women reached the second round.
• Looks like we saw quite a few people drop out in week 2. About 15% of the men and 15% of the women dropped out. That was a bigger percentage drop than we saw in any week last year, except for the women between 13.3 and 13.4 (19%). [UPDATE 4/12/2014 - This previously said the largest drop was between women's 13.2 and 13.3. That was a typo.] 

Also last week, I decided to try something new. I picked out two athletes who submitted scores by 9 a.m. on Friday morning, and each 12 hours after that, I tracked their progress on the leaderboard. My goal was to understand how the ranking of a particular score will evolve over the course of the weekend. Obviously an athlete's position would continue to get worse, but would his or her rank decrease (or increase) as a percentage of the field? If not, then an athlete could quickly get a sense of where his or her score might end up simply by checking out the percentile rank early on.

So let's see what happened. These athletes each were ranked in the 90th percentile at the start of the weekend, and I tracked the rank of the score, not necessarily the athlete. If the athlete retried the workout, their new score did not count towards this study.

Looking worldwide, we can see that in fact, the athlete's percentile stayed relatively stable. The men's percentile improved slightly, which I think is actually more typical. The women's score I picked was 87 (one rep away from finishing the 12's), and I suspect that a lot of women worked just hard enough to get that past hump on their second try.

Quickly, let's take a look at the progression in the region, which is generally more important for most of us.

Here we see that the athletes actually started at a lower (worse) percentile in the region. In the North American regions, this is probably common, since the only people performing the workouts that early on are generally strong athletes. In the rest of the world, those scores early on could come in normal Friday classes. But by 9 p.m. Friday, the athlete's percentile was pretty indicative of how things would finish up.

Anyway... on to the SWAG. Briefly, here is what we know:

• The movements left on the table (in order of how much emphasis has been placed on them in the past three years): burpee, thruster, jerk, toes-to-bar, box jump, muscle-up, wall ball, clean, deadlift, push-up.
• The LBEL so far is 0.38 for men and 0.27 for women. That's fairly close to the historical average for women (0.30), but below the historical average for men (0.46). So I think it's likely that things will get a bit heavier over the final three workouts than they have been to this point. That may or may not be true for 14.3 specifically, though.
• Neither workout so far has been particularly long. 14.1 was 10:00, and 14.2 had at most 9:00 of challenging work for any particular athlete, when you consider that the early rounds were nothing more than a warm-up for the top athletes.
• The first two workouts have been couplets. In 2011, there were three triplets, and in 2012 and 2013 there were two triplets each, so I think it's fair to assume we're getting a triplet soon. There were no single-modality workouts in 2013, and I doubt we'll see one this year.

With that as our framework, here we go with the SWAG for 14.3:

AMRAP 16 of 8 thrusters (115/75), 10 toes-to-bar, 12 bar-facing burpees

Again, you can pretty much take it to the bank that I will be way off. But it's not like you can do better. Of course, if you think you can, post SWAGs to comments. Enjoy 14.3 everyone!

Fun with SWAGs: What Will 14.2 Be?

One week in the books. I don't know about you, but I am once again shocked by the jump in competition from last year. I know personally I feel fitter than ever, but I find myself lower on the leaderboard than in past years (~780 in the Central East). It's tough seeing the top of the leaderboard get further and further away, but it's also reassuring that the sport is headed in the right direction. With 6,500+ men in the competition here in my region, only the absolute best of the best will even make it to Regionals. Marcus Hendren, top 10 in the Games the past two years, is currently outside the top 100 in his own region. This is where we are these days. For the rest of us, there's nothing to do but work hard to keep up.

Anyhow, let's move on. I may not qualify for Regionals, but I still have a chance to fulfill another lifelong dream: correctly predicting an Open workout the week beforehand. Last week, with basically nothing to go on, I wasn't particularly close, except for the 10-minute time domain. But I have a feeling is my week. NOW IS THE TIME.

OK, so let's get started. Let's assume that snatches and double-unders are off the table (even though half of the field probably used clean-and-jerks instead of snatches). Let's go ahead and assume they'll hold thrusters and pull-ups until the end again (although I really hope they don't). I also don't think they'll put cleans or jerks in this week, since they were an option in 14.1. That leaves us with the following movements on the table, in order of how much emphasis was put on them in the past: burpee, toes-to-bar, box jump, muscle-up, wall ball, deadlift, push-up, overhead squat.

Last week was a couplet with relatively light weights, so I'm going to assume this week will be a triplet with considerably heavier weights. As far as the time domain, it could really go either way, but let's go with... long?

Well, unfortunately I haven't really narrowed it down a whole lot. But that's never stopped me from going bold with a prediction before. Here goes:

AMRAP 15 of 10 bar-facing burpees, 10 overhead squats (115/75), 10 toes-to-bar

Zero percent chance that happens, of course. But let's see you try to do better. Post SWAGs to comments, and good luck on 14.2!

Fun with (S)WAGs: What Will 14.1 Be?

The 2014 Open season is finally here, folks. That means another chance to test your fitness against 70,000 men and 40,000 women (and counting). But perhaps more importantly, it means another chance to test your ability to make modestly substantiated, vaguely scientific predictions about what each Open workout will be!

Welcome to Fun with SWAGs, 2014 Edition (SWAG: Scientific Wild-Ass Guess).

Today's prediction for 14.1 will be less SWAG, more WAG, since every movement is still on the table and we can't really rule much of anything out. However, that doesn't mean we can't narrow things down at all.

First, let's limit the movements to those that have been programmed in the Open in the last three years. Considering no new movements were added to the Open programming in 2012 or 2013, I think it's a pretty fair assumption that no new ones will be added this year. In any case, it's good enough for a SWAG.

Next, consider that either burpees or snatches (or both) have come up in the first Open workout in each of the past three years. Not coincidentally, these are also the two most heavily-programmed movements across the three Opens. So let's assume at least one will come up in 14.1.

How about time domain? Well, broadly, we can assume it will be between 4 and 20 minutes (the min and max of all Open workouts historically). Beyond that, I'm not sure we can narrow it down a whole lot, since the first Open workout has been 10, 7 and 17 minutes in 2011, 2012 and 2013 respectively. But since we need to make a specific guess, I'll assume that they back off the 17 minutes from last year and put this one in the 10-12 minute range.

Now, let's start to put something together. I do not think they will repeat 13.1, since that was basically a combination of two prior Open workouts already (12.1 and 12.2). I think there will be a repeat workout this year, but I believe it will be 13.4 (cleans and jerks and toes to bar). That one doesn't satisfy my criteria of having burpees or snatches, so that leads me to believe 14.1 will be brand new. I'll also go ahead and assume it will not have both snatches and burpees this time, but I do believe it will be a couplet. I think the weight will be light-to-moderate, in order to attract as many entrants as possible in the final few days.

OK, with all that said, it's time to make a pick. My SWAG for 14.1 is:

AMRAP 10 of 20 burpees (bar-facing), 20 deadlifts (155/105)

Think you have a better idea? Post your SWAG to comments. Enjoy the Open everyone!

After Day 1, Is Rich Froning Still The Favorite?

Anyone who has been following the CrossFit Games for the past few years probably knows that the results after the first couple events generally don't really look a whole lot like the results at the end of the weekend. For one, there are simply a lot of events left to shake things up. This year, it appears we have at least 8 left, but I'm guessing more. But also, the early events have typically involved some atypical CrossFit movements, particularly swimming. The best swimmers have had a big advantage in the early events in the past few years, but the best swimmers aren't necessarily the best CrossFitters, so they often fall off over the course of the weekend.

Still, if you're making predictions right now (and you can make them up until the first Friday event, in fact, at the contest at switchcrossfit.com), you can't simply ignore the results from Wednesday. Those points are in the bank, and guys like Dan Bailey (currently 34th) now have a lot of ground to make up if they want to make it back into contention. My stochastic projections prior to the Games had Bailey picked very high, but how high would I pick him right now? And what about Rich Froning, who was a heavy favorite coming in but is currently in 6th?

Well, I took a couple hours to look into this. What I did was pretty simple: I re-ran my stochastic projections, but I replaced three of the random events with the actual results from Wednesday. The events I replaced were the random event based on last year's "long event" and 2 events based on this year's Regionals. My model still assumes 15 scored events, so we have 10 left that are based on this year's Regionals and 2 left that are based on this year's Open. If we assumed this year will have fewer than 15 events, the results would a little bit different - the current leaders would have a bigger advantage. But I think there are still a lot of points left on the table.

To keep things short, I'm not going to reproduce the entire table here for men and women. Rather, I'll give a quick recap of the current favorites, as well as some of the biggest movers after day 1.

Men
Favorite: Rich Froning, 51% chance. Froning dropped from a 58% chance prior to the Games but still is close enough to be considered the favorite in the long run.
Biggest contender: Jason Khalipa, 34% chance. Khalipa was already in the discussion, but his dominant performance on day 1 moved him up from a 7% chance coming in.
Others still with a strong shot: Scott Panchik, 7%; Josh Bridges, 5%. Both lost some ground on the rowing events. For those who read my methodology, you'll recall Panchik and Bridges were expected to do well on day 1 because of strong showings on the long events in past years.
Other notes: Dan Bailey dropped from a 1.7% shot to an 0.6% shot after a rough day 1, and Ben Smith fell from 3.6% chance to a 1.0% chance. Even the guys like Garrett Fisher, Chad Mackay and Justin Allen who did really well on day 1 are still pretty big longshots based on their Regional performances. The fact that the leader is Jason Khalipa doesn't make it any easier for them to make up ground. However, I do now have Fisher (currently 2nd overall) with a 7% chance at the podium, up from 1% coming in.

Women
Favorite: Sam Briggs, 66% chance. She was the favorite coming in at 32%, and with a lead, she's got to be an even bigger favorite. She doesn't have a lot of holes in her game, but there are still a lot of unknown events that could shake things up.
Biggest contender: Lindsey Valenzuela, 8% chance. She had a strong day 1 and is always a threat to win some of the heavier events. She moved up from about a 5% shot coming into the Games.
Others still with a strong shot: Kaleena Ladeirous, 6%; Rebecca Voigt, 7%; Elizabeth Akinwale, 4%; Talayna Fortunato, 3%. Ladeirous and Fortunato moved into the mix with good showings on day 1. Akinwale was a big contender coming in but now has a good deal of ground to make up sitting in 20th.
Other notes: Camille Leblanc-Bazinet dropped from a 9% chance to just a 2% chance now that she's back in 28th. There are three other athletes still with at least a 1% chance: Michelle Letendre (2.4%), Alessandra Pichelli (2.3%) and Kara Webb (2.0%). Rory Zambard, relatively unknown coming in, is at a 0.3% chance of claiming the title, after a very solid day 1.

It's still early, and the key for the top athletes is just to keep themselves within striking distance. Nobody is truly out of it at this stage, but a few athletes certainly made things a bit harder on themselves, while others gave themselves a real shot.

I'll be in California watching the Games in person for the next few days, and I don't plan to post anything else until I get back into town next week. Until then, enjoy the Games everyone!

So Who CAN Win the 2013 CrossFit Games - Predictions

Just a few quick notes before getting to the picks:

• These picks are based almost entirely off the results from this season, and thus the order will be similar (but not identical) to the Cross Regional Comparison found at http://crossfitregionalshowdown.com/leaderboard/men.
• There are some Games veterans, like Matt Chan for instance, whose odds probably look lower than some would expect. That's because last year's Games played only a very minor role in these projections. Although there are some athletes for whom we could probably make an exception, I think that in general, the results from Regionals this season are the best predictors of what will happen at the Games this season. Regionals are competitive enough now that I doubt many athletes were holding much back.
• For full methodology, see the previous post. The general idea is to use the results from the events that have occurred this season and simulate Games events that would be similar to them.
• These are rounded to the nearest 1%, so some athletes listed with a 0% chance actually may have non-zero chance according to the model, but that chance is less than 0.5%. For instance, virtually everyone had a non-zero chance of finishing in the top 10. The list is sorted by chance of winning, prior to rounding, and in case of ties, it is sorted by average finish.
• This is all in good fun, so don't take it too seriously if your favorite athlete doesn't appear as highly as you'd like. I'm well aware that this model isn't perfect, but my goal is to make the best predictions I can with the data we have available. There's plenty going on behind the scenes for each of these athletes and plenty of other variables that I simply can't capture.
• I'm curious to hear who you guys are picking this year. I think it should be a blast to see how things play out given the level of competition we've already seen this season. Post to comments or shoot me an email to let me know your take.

OK, without further ado, here are the picks. For each athlete, I have the estimated chance of him/her winning, placing in the top 3 (podium) and placing in the top 10 (money), along with the average ranking he/she attained across all simulations.

So Who CAN Win the 2013 CrossFit Games - Methodology

In some ways, it seems like making predictions about the CrossFit Games should be relatively easy. After all, we have plenty of data to make direct comparisons between athletes. So far this season, these athletes all have completed the same 13 events. By this point, it seems like the cream should have risen to the top. Remember, the 2007 Games had only three events and the 2008 Games had only 4 events. With 13 events already, shouldn't the champion be fairly clear?

Of course, what we've seen is that competition has gotten much tighter in recent years. In 2008, there were only a handful of athletes of the caliber to even think about contending for the title. This year, if we compare the Games athletes, 14 different male athletes and 14 different female athletes have finished in the top 3 of at least one workout. So nearly a third of the field has shown the capability to be the close to the best in the world on a given workout.

So, obviously, the complicating factor with predicting the Games is that we don't know what the workouts will be. And even if we knew what they were (in fact, we likely will know some of the events within the next week or so), we can pretty much guarantee that they won't match any of the 13 workouts we've seen thus far. So what can we do?

Last year, I estimated the odds of each athlete winning the Games by randomly selecting 10 events from among the Regional and Open events that had occurred. As I looked back on that methodology, I noticed that it really only gave a small number of athletes a chance at winning or even placing in the top 3. The reason is that I implicitly assumed that each event of the Games would exactly mirror one of the prior events of the season. After some investigation, it turned out that most of the events from the Games did not match any one event from the Regionals or Open particularly closely.

• Of the 20 events from the 2012 Games prior to cuts (10 men's events + 10 women's events), I looked at the correlation between that event and each Regional and Open event.
• For each of those Games events, I took the maximum of those correlations.
• 3 of 20 were at least 60% correlated with one Regional or Open event.
• 10 of 20 were at least 50% correlated with one Regional or Open event.
• 5 of 20 were not more than 30% correlated with any Regional or Open event.

Certainly, the reason for this variation is due largely to the design of the workouts in the Games vs. the Regionals and the Open. But I also think part of it is due to the fact that the Games are simply a different competition than the Regionals and the Open. Athletes come in at varying levels of health, with varying levels of nerves, and so even if the events were identical to regionals, I think we'd have different results.

Either way, I felt that in estimating the chances for each athlete this year, I needed to account for how much variation we have seen from the Regionals/Open to the Games. I needed to simulate the Games using results that weren't identical to the Regionals/Open but were correlated. I also wanted to rely primarily on the Regional results, since we know that some top athletes tend to coast through the Open while others take it a bit more seriously. Still, I did include the Open results to a lesser extent, because I don't think it's fair to ignore it entirely as it provides insight into how athletes fare in events that are generally lighter than what we see at the Regional level.

Additionally, we know that historically, the Games has typically included at least one extremely long event (Pendleton 2, for instance). This event is generally very loosely correlated with anything at Regionals or in the Open. But, we can assume that athletes who did well on the "long" event the prior year will likely do well on the long event this year this year.

So I set up a simulation of 15 events, assuming no cuts (all athletes compete in all 15 events). Here is a description of how each event was simulated:

• For 12 events, I randomly chose one of the Regional events to be the "base" event.
• I started with the results (not the placement, the actual score) from that base event, then "shook up" those results enough so we'd get about new rankings that were roughly 50% correlated to the base event.
• To "shake up" the original results, I adjusted each athlete's original result randomly up or down. Exactly how much I allowed the result to vary depended on how much variation was involved in that event to begin with. So if Regional Event 4 was the base event, I might let the scores vary by 3 minutes, but if Regional Event 1 was the base event, they might vary by only 1 minute.
• I did testing in advance to see how much I needed to vary each individual's score to achieve about 50% correlation. It turned out to be about +/- 2.5 standard deviations. So each athlete's score could move from his/her original score by as much as 3 standard deviations in each direction.
• The athletes scoring well in the base event still have an advantage, but we allow things to shift around a bit.
• For 2 events, I used the same process, but I randomly chose one of the Open events to be the "base" event.
• For 1 event, I used the Pendleton 2 results from 2012 as the "base" event. For athletes who didn't compete in the Games last year, they were assigned a totally random result.
• Athletes who did well last year have an advantage, but I did "shake up" the results a bit in each simulation.
• Keep in mind that finishing poorly in Pendleton 2 last year was considered worse than not competing at all.
• I made two exceptions: Josh Bridges and Sam Briggs missed last year due to an injury but did extremely well on the long beach event in 2011. I treated them as if they had competed in Pendleton 2 and finished very highly.
• These events were simulated 5,000 times. The Games Scoring table was applied to determine the final rankings after each simulation.

Before applying this method to this year's field, I went back to see what type of estimates I would have gotten last year with this method. Some notes from those simulations:

• I looked at how good a job I did at predicting which athletes would finish in the top 10. The mean square error (MSE) of my model would have been 0.121 for women and 0.104 for men. Had I simply assumed the top 10 from Regionals would be top 10 at the Games with 100% probability, the MSE would have been 0.130 for men and 0.133 for women. If I had instead assumed all athletes had an equal shot at finishing in the top 10, the MSE would have been 0.254 for men and 0.259 for women. So I did have an improvement over those naive estimates.
• On the men's side, I would have given Rich Froning a 45% chance of winning, with Dan Bailey having the next-best chance at 30%. For the women, I would have given Julie Foucher a 53% chance of winning and Annie Thorisdottir a 22% chance of winning (remember, Foucher was the pick for many in the community last year, including me). No one else would have had more than a 7% chance on the women's side.
• For podium spots, I would have given Froning an 86% chance, Chan a 4% chance and Kasperbauer a 2% chance. For women, I would have given Thorisdottir a 61% chance, Foucher an 84% chance and Fortunato a 3% chance. While it would be nice to have given Chan, Kasperbauer and Fortunato a better shot, I don't recall many people talking these athletes up prior to the Games. None had ever reached the podium before, although Chan had been close.

My goal was to strike a balance between confidence in the favorites (like Froning) and allowing enough variation so that relative unknowns (like Fortunato) still have a shot. This largely comes down to how much I shook up those original results. The less I shook up the original results, the more confident I would have been that Froning would have won last year. But I also would have given someone like Matt Chan virtually no shot, because his Regional performance simply wasn't that strong compared to the other heavy hitters. But if I shook up the original results too much, things just got muddy and I allowed everyone to have a fairly even chance to win, which doesn't seem realistic either.

No model is going to be perfect with this many unknowns. Sure, you could argue that I am not taking into account other factors, like the advantage that Games "veterans" could have. But I would counter by pointing out that last year, Fortunato was a first-time competitor and Kasperbauer hadn't competed individually since 2009, and they both fared well. Other athletes like Neil Maddox simply didn't perform well at the Games despite experience at the Games and great performances at Regionals. A lot of it simply has to do with what comes out of the hopper, how each athlete manages the pressure and what little breaks go for or against each athlete throughout the course of the weekend. But at the end of the day, the fact is that the athletes who do well at Regionals and the Open generally fare well at the Games, and that's why I am using those results as the basis for my estimates.

With the methodology and assumptions out of the way, move ahead to my next post for the picks for the 2013 Games!

Week 4 Predictions: Let the Big Dogs Eat

OK, so I may be a little biased living in Indianapolis, but I think I'm justified in saying that the Central East men's competition is really the crown jewel of the Regional season. I'll be headed down to Columbus, OH to watch Saturday and Sunday in person, and I have to say I'm pretty pumped. The numbers for this region are staggering:

• The winner of the last three CrossFit Games has come from the Central East (Graham Holmberg 2010, Rich Froning 2011-12). Both men are competing this weekend.
• The Central East produced five men in the top 10 of the 2012 CrossFit Games. All five are competing this weekend.
• Two other men competing this weekend qualified for the Games in 2011: Nick Urankar and Joseph Weigel. Urankar was 6th at the regional last year.
• Four of the top 20 Open finishers worldwide and six of the top 43 are competing in this region this weekend.

Thank goodness for the rule allowing extra spots for former champions, because limiting this region to only three spots would be damn near inhumane. Five doesn't even seem quite enough, frankly. Eight competitors from this region were in the top 31 of the worldwide regional standings last year. There are some other great athletes competing this weekend (Ben Smith, Nate Schrader, Lucas Parker, Christy Phillips, etc.), but you can't deny the strength of the Central East men's region as a whole.

The competitiveness of this region also forced me into making some adjustments to my modeling this week. For background on the process of generating these projections, please see my prior post.

Change one this week was the fact that I decided I could not treat Froning like the other competitors. Realistically, the only way he's not making the Games is due to injury. My solution was to give him an 80% chance of finishing top 10 worldwide and a 20% chance of finishing 11-25 worldwide. This leaves open the chance that he could get beat, but only in a scenario where several other guys put up crazy numbers.

Change two was to adjust all my projections to give more of an advantage to the top competitors. My solution was to do the following:

• For each category, look at last year's results and calculate how often athletes in that category finished in the top 25 worldwide. Last week, I calculated how often they finished top 50.
• Additionally, calculate how often they finished between 26-75 worldwide. Last week, I calculated how often they finished 51-100.

The reason for making this change was that in really competitive regions, too many athletes were getting put into that top 50 range. From there, it was just a crapshoot to see who drew better random numbers and qualified for the Games. Now, it's less common for athletes to get into that top 25 category, so doing so gives you a big advantage. For instance, athletes in the top category (top 15 at Games last year) have an 83% shot at finishing top 50 and a 65% shot at finishing top 25. The next highest category (16-50 at Games last year, top 40 worldwide in Regionals last year) had an 81% chance at finishing top 50 but only a 41% shot at finishing top 25. This gives us more separation for the top athletes.

Still, the Central East has five men who all qualified for my top category. The fact is, it's tough to separate them: who would you pick not to make it from those five? But then again, you also have Nick Urankar, who is really solid (26th worldwide in the Regionals last year), and there are others who could legitimately slip into the top 5. As a result, my projections don't look that great for some men that you'd generally consider to be locks for the Games. I'm not sure that's necessarily wrong.

One final note: I made the assumption that if Froning or Holmberg is in the top four, the region gets four spots. If both are in the top five, the region gets five spots. This means if Froning is 5th and Holmberg is 4th, they both still get in. Technically, this isn't how the rules are laid out, but after HQ invited 4th place Kristan Clever from SoCal, I think it's a safe bet that they'd do the same for these guys.

Before I move onto this week's picks, let's look at how last week's picks turned out. Overall, I think pretty well:

• Of the 12 qualifiers from regions I projected, 4 were given more than a 50% shot at making it. There were also 3 athletes who I gave more than a 50% shot who did not make it.
• According to my picks, the longest shots to qualify were Alex Nettey at 16%, Zach Forrest at 18% and Matt Hathcock at 18%.
• If we look at the mean square error (MSE) for my picks compared to some other naive estimates, they did pretty well. 
• The MSE for my picks was 0.045.
• If you had given everyone in each region an equal shot, your MSE would have been 0.064.
• If you had given a 100% chance to the top 3 in the Open in each region, your MSE would have been 0.078.
• If you had given a 100% chance to my top 3 in each region, your MSE would have been 0.057.

So overall, a decent showing. As mentioned above, I made some adjustments, so we'll have to see how they turn out.

As I did last week, I'm limiting these picks to the four regionals that will be broadcast. In this case, that's the Central East men and women and the Mid Atlantic men and women. Without further ado, here are the athletes with the best shot at qualifying from those regions this weekend:

Enjoy the weekend, everyone!