“Game developers don’t like me,” said Sabina Hemmi. “Before I came around, there was no insight into how balanced a game was.”
Hemmi is joking. Partially. As the co-founder of DotaBuff, a site that provides statistics about Dota 2 gameplay, she can show with hard data if certain characters are stronger or weaker, and in what areas. As access to such data is becoming more commonplace, she says, players are starting to expect it, and are reacting to it. “If I’m focusing hours of my time into the game, I want to focus on playing the characters that are actually competitive,” she said.
Just as it did with traditional sports, the collection, analysis, and use of all kinds of data is beginning to change the way that competitive games are played and understood. But while many scenes like Dota 2, League of Legends, and Overwatch are coming around to the value of statistics, the breadth and sophistication of statistical coverage in general is still in its beginning stages.
The numbers that Sabina Hemmi’s team have pulled down from Dota 2 over the years are, in a word, expansive. You can view stats from your own match history stretching back years. I can see how good I am at playing Venomancer, what items I tend to buy, where I tend to place vision-providing wards, and my average damage output.
Pulling up the esports tab, I can scroll through hundreds of professional Dota matches, each recorded and compiled by the website’s myriad systems. Each is applied labels and tags, ordered by series and team, with player names automatically assigned so I can back-search their individual histories. (Dota players can opt out of having their data displayed by request, Hemmi said, leaving their stats but no account or personal information.)
The challenge, Hemmi noted, isn’t necessarily providing the data, but putting it into context. “Often, when people are looking to use a stats site, their first question when they see the data is, ‘Where is this data from?’” she said. “Is this data from this week, this patch, this month? Is it just high-skill data? Is it just esports data?”
Sorting the data of thousands of Dota matches by hand is like trying to take a sieve to a waterfall, so tools that can automate that process are important. Dota 2 has built-in skill markers, which can sort games as being played in certain tiers of matchmaking rating, or to only draw from games from certain tournaments. It’s important to sort the signal from the noise, not just to find data from the subset of players you might be targeting, but to also deal with the sheer breadth of numbers.
The DotaBuff team goes through about a million matches per day. Roughly 76 percent of that is normal skill level Dota, the average player. It stores over 100 terabytes of data on its servers. The wealth of data is staggering. But DotaBuff is ahead of the curve, and other esports are still in the process of catching on.
Sabermetrics, But For Dota
Before stats started to take over esports, they took over traditional ones.
For much of the 20th century, simple counting stats dominated our understanding of a baseball player’s value. For hitters, it was runs batted in. For pitchers, it was wins. If you’ve seen the film Moneyball, you know what happened next: Starting in the middle of the 20th century, “sabermetrics”—derived from the acronym SABR, or Society for American Baseball Research—applied empirical analysis to baseball, changing almost the very definition of how to play the game. Coined in the 1977 book Baseball Abstracts by Bill James, it paved the way for collecting and summarizing data to better calculate the contribution a player might offer to a team’s win.
The change was slow to come, as only a few front offices flirted with the idea of hiring advanced analytics types. The Oakland Athletics took a major bet on analytics in the late ‘90s, and ended up putting together a 2002 season that at one point chalked up 20 wins in a row. It drew national attention to the concept, and nowadays statistics are seen as a vital tool in understanding sports.
It might be surprising, considering that video games are literally made of numbers, but the use and collection of statistics in esports wasn’t much more advanced than box scores just five years ago. Kills and deaths would get tabbed up by a human viewer on a physical notepad, or through a basic punch-in client, rather than being pulled directly out of the game’s data.
“Back in the day, if you didn’t physically go to an esports event, it would be really difficult sometimes to find a [video] of the tournament, or to get a demo file or replay file,” said Sabina Hemmi. “You either had to know someone or rely on the journalists’ summary coverage of it, which might just be box scores, to see high-level play.”
Now, we have something that Hemmi calls the “democratization of data”—tools like recorded replays and match histories that create an abundance of data, all instantly available for public consumption.
The data process collection is simple, in theory: As in-game events occur, they are recorded and logged, and different programs or models can scan those logs for the information they’re looking for. Every swing of an axe, movement around the map, and coordinate location can be noted, as long as your game of choice has the tools to map it out.
As these data points became more public and players got more interested in accurate stat tracking, some of the ways of measuring achievement in games progressed as well. As the market for information grew with the amount of new competitive genres emerging, the understanding of a single player’s contribution to a win has evolved significantly.
Take that classic stat, the kill-death ratio. It’s been around ever since two players first fired rockets at each other in Quake. K-D was a simple tool for a simple job. You had to out-kill your opponent, so a good kill-death ratio meant you were performing better and winning more often than the other player.
Then shooter games became team-oriented, and the first MOBAs started to come out. Now, multiple people might participate in a kill, even if only one player lands the final blow. That person is awarded the point, but what about the other one, two, four players that might have contributed? The kill-death ratio became KDA, a number still used in most multiplayer games today: kills-deaths-assists. Now, if you were laying down cover fire for the star sniper, you still got some recognition even if Mr. NoScope landed the killing shot.
It’s the same as a basketball player getting credit for their assistance in a big play—the scorer isn’t the only person who made something happen. To break down the greater questions of how victory is achieved and maintained throughout a season or career, these minor advancements portray a more honest view.
Welcome To The Jungle Proximity
When the Houston Rockets basketball organization expanded into esports with a League of Legends team called Clutch Gaming, it said at its launch that the team’s approach would be centered on data and analytics.
“Given that historically there are very few things in the world that can’t be quantified in some manner or form,” said the team’s VP of esports Sebastian Park, “We’re like, okay, let’s work on seeing if we can make this better and see if we can gain an edge that we can keep over other teams, and field a highly more competitive team in the long run.”
The Rockets themselves are famously analytically oriented. The team’s general manager Daryl Morey is a cofounder of MIT’s Sloan Sports Analytics Conference, an annual forum for discussion about the role of analytics in the sports industry.
What do metrics and analytics look like for an esport like Riot Games’ League of Legends? Moving beyond the basics of KDA, there’s also something called the “gold differential.” In League, two teams of five players each have to seize gradually increasing advantages in order to destroy the heart of their opponent’s base. Part deathmatch, part strategy game, League is like a digital version of keeping several plates spinning in the air, securing enough simultaneous objectives to eventually lead to victory. During a game, you build up strength in an RPG-like fashion, gaining levels through battle experience, or buying new items with gold earned from battles.
The “gold differential,” then, is another example of a base, raw statistic that can provide a sense of the state of a match at a glance. Usually when a team has more gold than the other team, it means their champions are carrying more powerful items and are more likely to win if they get into a scrap. Combined with individual KDA numbers and the overall number of kills a team has accrued, it’s an easy way to let the viewers at home know who is winning. When you turn on an LCS broadcast, gold differential is one of the first numbers you see at the top of the screen. But statistics like these don’t tell the whole story.
It’s easy to convey and understand concepts like a kill tally or gold coins. But what about more esoteric stats that, while a bit less grokkable, might explain a lot more about the current state of a team? Doug Watson, a Riot Games employee that studies statistics, brings up “jungle proximity.”
Like a basketball team, the five players on a League of Legends squad are playing specific positions on the map. The “jungle” position splits their time between killing ambient wildlife in the “jungle” for gold and experience, and trying to kill or hinder enemies on the main “lanes.” In practice, the “jungler” is a roaming source of pressure that often dictates their team’s focus and pace. When a jungler “camps” a lane, they’re taking up residence to ensure their teammate in there is going to be successful and ahead of the opponent.
Watson’s team analyzes the proximity of the jungler to each lane throughout the course of the game, in order to quantify how good a jungler is at jungling. During games, his team tracks each champion’s X and Y coordinates on the map each second. Using their exact positions over a stretch of time, they can determine their proximity to each other and begin to analyze that connection.
“What lanes do junglers prioritize?” said Watson. “Do they play to snowball a specific lane versus focus on shutting down the biggest threat on the opposing team? Do they have heavy lane presence or focus more on farming their own jungle?” Through the numbers, they can begin to note trends and nuances specific to players, teams, and the general state of the game. Analysis like this can give Watson and the rest of the spectators, potential opponents included, an idea of what makes a strategy successful.
“There’s a time and place for [advanced metrics], to be able to understand the game further, and we try to make sure we intro those things appropriately, as well as build up the context over time,” Watson said.
Numbers like this can help you see whether League of Legends stars like Heo “Huni” Seung-hoon or Lee “Faker” San-hyeok are relying on jungle assistance to enable their playstyles, or if a team prefers to have their support roaming the map or stay with the carry. If a team knows that the opposing jungler likes to spend their time in the middle lane, ensuring that player’s success, then they can be ready and prepare to counter that, picking champions that can play against heavy pressure or having their own support spend more time roaming to assist. It can open up other lanes to pressure their opponents harder, forcing the enemy jungler to either leave their comfort zone or double-down on a strategy. And it also provides a storyline for the viewers, helping them to follow the action and get inside the minds of the players.
Scraping The APIs
Data can also tell us how a team might play or react to any given situation. In games like League and Dota 2, ten players on the field means thousands of data points to scrub, each one ranging in importance from telltale to humdrum. Finding the right hooks and points is important to uncovering effective data. This is made significantly easier when the game has an open API, or tools for scraping the game’s code to find relevant data points.
Getting info from an API requires some basic coding. You input a query, and it outputs the response. What level and depth of response you might get can vary, and tools like DotaBuff and datDota have been built to expedite the process and make it a little more user-friendly. While the base level could be something as simple as “how many kills in a given game,” it can escalate higher and higher. What’s the win rate of a team for a given set of games, if that team took the first tower? Which heroes benefit most from being on one side of the map versus the other? Which items would most affect a team’s chances of winning or losing?
Of all the games out there right now, Dota 2’s API is considered to be the most powerful, allowing users to grab a number of certain data points during a match. Simple in concept, complex in execution, these tools are what drive DotaBuff and countless other stat sites, and even the in-house broadcasts for competitions. Imagine if a computer could instantly inform you of the exact motion of every limb of a player during a key moment in a sports game.
One major statistic that’s easy to see, visualize, and understand is in Dota is “warding.” Wards are items that a player can place on the map to get a look at where the enemy team is for a limited amount of time.
This is critical, because the map in games like League and Dota is covered by a fog of war—you can only see in an area around your team’s units, structures, heroes, and wards. According to analyst Alan “Nahaz” Bester, who runs a site called datDota, Dota teams have been using tools like ward maps since at least the 2016 International, Dota’s multimillion-dollar world championships. Some teams use them to counter each other’s playstyles—but some have found ways to use other players’ over-reliance on information gleaned from statistics to their own advantage.
Bester told me about a recent incident in which a player called out Dota squad Virtus Pro for using the same ward spots over and over again in an extremely predictable way. But just because you’re predictable doesn’t mean your strategy sucks; Virtus Pro has already clinched its invite to the 2018 International.
“They are warding areas of the map that are very obvious,” Bester said of Virtus Pro, “But because they’re keeping such consistent pressure on those areas of the map, dewarding them becomes a risk in and of itself because they’re generally going to be in position to attack those areas of the map as a team, if you’re sending somebody out there unescorted.”
In other words, just tracking one statistic might lead to a flawed conclusion. And so teams aren’t just developing one metric but several, all in an attempt to answer the greater questions around competition in games like Dota 2.
For Bester, an analyst often on the commentator’s desk for broadcast Dota matches, it’s about finding the right metrics to tell a greater story. Even in the earlier days of Dota data, Bester and his fellow stat hounds were searching for answers using the API. One early problem, he said, was item timing winrates.
Remember that gaining strength in League and Dota by gathering items is paramount to success. In fact, several characters in Dota 2 rely directly on items to become the asset their teams need them to be. A (carry) Naga Siren needs Radiance, a burning sword that lets her destroy enemy units. An Anti-Mage needs Battle Fury, a weapon that sends the hero’s power into the stratosphere.
Bester’s concern was with a hero called Weaver, and an item called Linken’s Sphere. It gives a hero a nice overall boost to strength, damage, and mana, as well as some regeneration. It also blocks single-target spells, limited by a short cooldown—crucial for the slippery, mobile Weaver to move around fights without getting locked down by pesky stuns or silences. The more a Weaver can move about unhindered in the enemy’s backline, the better, and Bester wanted to identify how important that asset was to a team’s probability to win, especially when bought before or after a certain minute mark.
When folks like Bester first brought up these concepts, there was some pushback, just like when analytics in traditional sports first started reaching conclusions that ran against the conventional wisdom of the time. Bester told me a story of a commentator asking him why certain things, like an item timing compared to the global average, even mattered in a single game. In the commentator’s view, it should be obvious that when a player gets one of their best items early on, they have a better chance of winning. It’s the magnitude that’s important, was Bester’s reply.
“Most of the time, everybody sitting in the seats as a fan can tell when LeBron James just did something good,” said Bester. “But the point is some of those actions are more important than others, and as you start to look at them, it becomes really surprising which of those actions have a larger significance.”
Another, more subtle, concept is support level timing.. Gold is a primary meter for strength in a core hero, one that needs big, high-ticket items to be successful for a team. Support heroes are less needy. They tend to do more early on, especially with early points in their big spells.
So, as Bester as found through data, teams are now finding ways to get their supports to an early level six. Early rotations of a mid player into the jungle to find more gold can leave a lane open for an otherwise-starved support to soak up experience, securing that crucial level. That’s the point where the supports’ ultimate abilities come online, big-impact moves like Earthshaker’s Echo Slam or the global-range Ice Blast from Ancient Apparition. It sounds simple on paper, but teams that share the experience rather than hoard it on a few heroes are finding success through this strategy.
Even bigger problems are out there, though. If kills and assists don’t tell the whole story well enough, other metrics are trying to do it better. One stat like that is teamfight data—a stream that tells you when a certain major event goes down, which players involved did what, and how much that contributed to the team’s success. While Alan Bester’s group has been working on this for a while, someone sent them a packet of data back in 2015 after that year’s International with their own calculations, which contained found some pretty telling results.
It turned out, Bester said, that the presence or absence of a single player named Universe on the team Evil Geniuses was key to determining if the players would win a teamfight. The difference in terms of the favorability of the outcome for EG between him being involved in a teamfight and not was ridiculous,” said Bester. “It was like, by far larger than any other player in the tournament at any position.”
All of this, again, seems fairly straightforward. If you were watching the games live, listening to the commentary, and following the action, you would easily know who Saahil “Universe” Arora was and his contribution to the team. But over time, the data shows a pattern and attributes a level of value to said player. He was always contributing to the win, but this helps quantify how much and in what capacity. It tells the story: Arora isn’t just a good player, but a crucial component of Evil Geniuses’ success that year.
How Data Will Change The Game
What are the other practical applications of techniques like this? For one, it’s extremely useful in broadcasts. Statistics help tell a story, accentuating different aspects of a player or team in order to help you understand what makes them operate at such a high level. Doug Watson’s Insights team at Riot tries to balance that with an educational aspect as well. These aren’t just games you watch, but ones that many spectators likely play in as well. What a pro player does in a match last night, you could do today in your own games. The latter is where metrics differ the most, in practical use, from typical sports shows. You might see how Tom Brady throws a spiral, but it’s not like you can replicate that exactly the same way you can insta-lock the hot new champion of the month in a game of League of Legends and imitate a pro’s build.
“We have certain things around champion selection and what people are doing on champions, which would never relate back to traditional sports because there’s not that kind of concept,” said Watson. “But that’s super important and helps us understand a different view of the world... The champion does this type of thing, this champion spikes in performance at these points in time, this champion has something I can learn from as a player. Those are things we would focus in on that would never come from traditional sports.”
In the case of DotaBuff co-founder Sabina Hemmi, her team is looking into technology like computer vision—teaching a computer to watch and analyze events on a screen—to mesh with the comprehensive Dota API. Her site has a tool that links a computer vision stream of a match with its combat log. Using visual data, it can determine when a crucial fight is happening and clip a replay of it. Computer vision has seen a fair amount of use in traditional sports, but its development in the esports realm is still in its early stages, and costly.
A prevalent use for metrics has been scouting. Hundreds of matches are played every day, across different games, servers, and regions. Finding the next star player simply by watching games haphazardly would be like looking for a needle in an infinitely large haystack. Good statistics can get you some of the way there, identifying not just who has good KDAs, but what kind of other contributions they’re making. There’s still something to be said for traditional scouting and tryouts, but APIs make it easy to identify the good data points your team is looking for, and start combing.
Identifying those metrics will become key to growing professional esports organizations. Park posed an interesting notion to me about another competitive game, Overwatch. If you’re looking for a Tracer player, you might start with eliminations (kills). But hitscan accuracy can also be a marker of a great Tracer. “Hitscan” refers to weapons whose bullets have little to no travel time. Tracer uses hitscan weapons, twin machine-gun pistols that unload bullets in rapid succession.
The more accurate a Tracer player, the more damage they’re doing, and the more ultimate charge they’re building. The more ultimate charge they have, the more they can use Tracer’s powerful Pulse Bomb ultimate. Pulse Bombs might lead to eliminations, which is good. But they might also force an enemy team to use precious support ultimates, like Zenyatta’s or Mercy’s, to mitigate or recover from the damage done by one. That second example wouldn’t show up on eliminations, but it’s still a moment when a Tracer is making an impact on the game, contributing to a win.
How close we are to finding more metrics like that varies from game to game. Dota 2’s API is open and comprehensive, and so it’s attracted a great deal of attention from stat-hungry analysts like Alan Bester. League of Legends’ API is also out there for open use, and it’s already producing interesting innovations like “jungle proximity.” Other games like Overwatch are more closed-off in terms of how much data they let outside sites access, though sites like Winston’s Lab are making strides in the scene and the official site has been adding more statistics options over the last few months.
In the future, artificial intelligence could be fed millions of previously-played Dota matches to analyze, all to find what the next great team composition or strategy might be. There’s still the human element, since interpretation is key to using any of these advanced metrics. Visualization is important too, rendering data in a more easily-understood manner.
“Nobody wants to sit through a two-minute explanation of where a number comes from,” Bester said. “The current generation of players that are approaching these titles and looking at possibly playing them at the competitive level, many of them don’t care about numbers at all. But all of them care about data.”
Measuring, analyzing, and reacting to esports statistics can still feel like fumbling in the dark sometimes, but it’s also ripe for exploration. Tools are open and available, teams are starting to hire on dedicated statisticians and analysts, and the role that numbers will play over the long-term for these competitions is becoming clear. Even for casual players, scoreboards are becoming more detailed, and built-in features like Dota Plus offer advanced metrics and algorithms to promote improvement, even in average-skill public matchmaking. It’s pretty certain that at this point, statistics are becoming inseparable from the games.
Compete is Deadspin and Kotaku’s joint site dedicated to competitive gaming.