Find out Etienne’s initial predictions by visiting last week’s World Cup blog post.

The World Cup is half-way through: the group phase is over, and the knockout phase is beginning. Let’s update the winning probabilities for the remaining teams, and analyze how our classifier performed on the group-phase matches.

From the 32 initial teams, 16 are qualified for the knockout phase:

There have been some surprises: from 10 of our favorite teams, 3 have been eliminated (Portugal, England, and, most surprisingly, Spain). But most of the main teams are still there.

Using our classifier, we compute again the winning probabilities of each team. To do so, we update the team features to include the last matches (that is, we update the Elo ratings and the goal average features), and then we run 100,000 Monte Carlo simulations of the World Cup starting from the round of 16. Here are the probabilities we obtained:

Again, Brazil is the favorite, but with a 32% chance to win now. After its impressive victory against Spain, the Netherlands’ odds jumped to 23.5%: it is now the second favorite. Germany (21.6%) and Argentina (8.6%) are following. There is thus, according to our model, an 86% chance that one of these four teams will be champion.

Let’s now look at the possible final matches:

The most probable finals are Brazil vs. Netherlands (21.5%) and Germany vs. Netherlands (16.7%). It is however impossible to have a final Brazil vs. Germany, since these teams are on the same side of the tournament tree. Here is the most likely tournament tree:

In the knockout phase, the position in the tournament tree matters: teams being on the same side as Brazil and Germany (such as France and Colombia) will have a hard time reaching the final. On the other hand, the United Sates, which is in the weakest side of the tree, has about a 6% chance to reach its first World Cup final.

Finally, let’s see how far in the competition teams can hope to go. The following plots show, for the 9 favorite teams, the probabilities to reach (in blue), and to be eliminated at (in orange), a given stage of the competition:

We see that Germany has a 35% chance to be eliminated at the semi-finals stage (probably against Brazil), while France and Colombia will probably be stopped at the quarter-finals stage (probably against Germany and Brazil).

Let’s now analyze how our classifier performed for group-phase matches. Forty-eight matches have been played, and it correctly predicted about 62.5% of them:

Which is close to the 59% accuracy obtained in the test set of the previous post. The accuracy is an interesting property to measure, but it does not reveal the full power of the classifier (we could have obtained a similar accuracy by always predicting the victory of the highest Elo-ranked team). It is more interesting to look at how reliable the probabilities computed by the classifier are. For example, let’s compute the likelihood of the classifier on past matches (that is, the probability attributed by the classifier to the sequence of actual match outcomes P(outcome₁) × P(outcome₂) × … × P(outcome₄₈)):

This value can be compared to the likelihood computed from commonly believed probabilities: bookmakers’ odds. Bookmakers tune their odds in order to always win money: if $3 has been bet on A, $2 on B, and $5 on C, they will set the odds (the amount you get if you bet $1 on the corresponding outcome) for A, B, and C a bit under:

Therefore, if we inverse the odds, we can obtain the outcome probabilities believed by betters. So, can our classifier compete with this “collective intelligence”?

We scraped the World Cup betting odds as they were right before each match from http://www.oddsportal.com/soccer/world/world-cup-2014/results and converted them to probabilities. We obtained a likelihood of 1.33209 × 10^-20, which is more than five times smaller than the likelihood of our classifier: there is thus about an 85% chance that our probabilities are “better” than bookmakers’. The simple fact that our classifier probabilities compete with bookmakers’ is remarkable, as we only used a few simple features to create the classifier. It is thus surprising to see that our classifier probably outperforms bookmakers’ odds: we might even be able to make money!

To test this, let’s imagine that we bet $1 on every match using the classifier (setting the value of UtilityFunction as explained in the previous post). Here are matches that we would have got right, and their corresponding gains:

The classifier only got 38% of its bets right. However, it often chose to bet on the underdog in order to increase its expected gain. In the end, we obtained $16 of profit, which is about 33% of our stake! Have we been lucky? To answer this, we compute the probability distribution of gains (through Monte Carlo simulations) according to our probabilities and to bookmakers’:

The average profit, according to our model, is $14. We have thus, for sure, been a bit lucky with this $16 of profit. Can we at least conclude that our probabilities outperform bookmakers’? Again we can’t be sure, but from computing the probability density to obtain a profit of $16 in both models, we see that there is a 65% chance that our model actually allows us to make money in the future…. To be tested on the next international competition!