Blanket statements made about statistics are dangerous. How any set of data is interpreted can lead to wildly different conclusions being drawn, and trust me, things are about to get wild. Yesterday, at the Games Developers Conference in San Francisco, Yauheni Hladki told his audience that: “The result that we’ve come with is that all the esports far surpass traditional sports in terms of skill”. And yes, that raised some eyebrows clean off their foreheads. “Why?” he thankfully went on to ask. The answer: "Because the sample size is huge and tremendous. For every single team, for every single player in esports, they play far more games than professional athletes."
By the sheer amount of games, the sample size becomes so big that the possibility for randomness almost goes to infinity.
—Hladki
It still sounds like an outlandish claim, but in his GDC talk entitled “Why It's So Much Harder to Predict Winners in Esports”, Hladki seemed extremely confident. He’s certainly well-qualified to speak on the subject, having built up plenty of esports experience in his role as the StarSeries commissioner at StarLadder. He’s been involved with running leagues for games across the entire spectrum of esports, from mainstream favourites CS:GO, Dota 2 and Hearthstone to the somewhat less well-known World of Tanks scene. Hladki also boasts an impressive academic background, having studied both theoretical physics and political science, both fields in which you need to know your way around an equation.
Hladki explained that his study was inspired by the work of Michael J. Maboussin, whose book The Success Equation sought to place traditional sports on a continuum between pure luck and skill. According to Hladki, these are the two components which determine the outcome of any competitive game. (If you’re mathematically inclined, F(x) and F(y)). Examining the luck factor first, Hladki notes that some games contain more of it than others. Chess, for example, features a lot less luck than ice hockey, which Hladki says is actually one of the most random professional sports. We should note also that as the sample size gets bigger (read: the number of games played increases) this factor becomes less significant as the luck evens out. Similarly, the extent to which players can demonstrate their skill will vary between games.
Hladki thinks the larger sample size we see in esports mean that luck is less of a factor. "By the sheer amount of games, the sample size becomes so big that the possibility for randomness almost goes to infinity". I'm a little skeptical at this claim. After all, is the number of games teams play in an esports league really all that different from, say, the number of games in a Premier League football season? It might be fairer to assume that here Hladki is talking about the online ranking systems you’d find in games like League of Legends or Overwatch, where players can grind away at the ladder all day in pursuit of the top ranks. There is no analogue to these in conventional sports. Players do train outside of competition, of course, but they are not formally ranked for doing so.
Perhaps even more controversially, Hladki says that esports are inherently more skillful. "To score one try in American football is very difficult." he says, "Whereas in CS:GO, the guy just comes and sprays. So every single bullet is potentially considered a try." Again, this raises some obvious questions. Perhaps he’s just simplifying things for the sake of clarity, but it’s hardly difficult to conceive of equivalent actions in sports that increase the chance of winning without the scoreboard actually being altered. A defence-splitting through-ball in soccer, for example, or perhaps a dominant first serve in tennis that puts the opponent on the back foot.
RNG is one of the challenges that I think we have to add one more variable for in esports.
—Hladki
Along similar lines: why does this have to be true of all esports? Take Hearthstone, for example, a game frequently criticised for its use of random effects. This is a game where a professional player can lose to a rank 25 (the game’s lowest rank) playing a budget deck. Faced with a question about RNG from our Hearthstone-obsessed global editor, Tim Clark, Hladki said this: "The RNG is one of the challenges that I think in an esports game we have to add one more variable than what we’re basing it on right now, which is RNG factor." Which seems a little confusing, as one would assume that’s already accounted for in any model that factors luck in the first place.
With all that said, Hladki’s thesis does still ring true to a certain extent. The idea that esports games have more opportunities to show skill than conventional sports (well, maybe not Hearthstone) does have some degree of truth to it. Think about how many variables there are in a game like Dota 2—hero choices, item builds, team compositions, etc—it’s clear that there are a huge number of decisions to make, and with each of those comes an opportunity for a player to demonstrate skill. The argument Hladki may also be making is that the sheer number of people playing these competitive videogames means that those who reach a level good enough to turn pro must have inherently displayed more skill over their peers than, say, a kid who makes a college football team.
While some of Hladski’s conclusions might seem a little far-fetched, it is important to emphasise that they are all based on data, and he actively encouraged the audience to debunk his work, arguing that what esports needs is more peer-review analysis based on data. And even though his data is not publicly available yet, he promises it will be up on his Linkedin page once he has permission from the relevant game publishers. So we should exercise some caution before tearing down his conclusions. In any case, it’s great to see this kind of research into variance and skill being done for esports. It might take a few more studies before you can convince me that Pavel is more skillful than Messi, though.
